Multi-Touch Flash, flosc : Flash OpenSound Control

04.22.2008 // News, Research //

We were using this method to track and migrate data from Processing to Flash… it’s not extremely efficient but can get the job done. I imagine this could come in handy if you wanted to utilize a single machine/camera setup that feeds data to multiple exhibit stations…

We figured It was possible to do multi-touch within Flash using this method but it required some heavy work within the application. At the moment, I have not read up on any implementation of Flash with multi-touch/multi-track or gesture via the Apple multi-touch touchpad. I have asked the question on some forum groups and emailed some inside contacts at Adobe Flash but no one has officially said that they support the multitouch classes/methods and functionality through Flash or that they have had a reason/functional applet to do so.

Some garbage collection should be implemented on the Flash side otherwise it tends to crawl to a complete stop eventually. We are hoping to do more with implementing it in Flash AS3 this summer. Do let us know if you have any success or another way to implement this approach to motion tracking and gesture recognition.

Software/Applets Needed:

Adobe Flash (AS2, AS3 not tested at the moment)
http://www.adobe.com

flosc : Flash OpenSound Control
http://www.benchun.net/flosc/

OSC P5
http://www.sojamo.de/oscP5

Processing Vs.0124
http://processing.org/

BlobDetection for Processing
v3ga <http://processing.v3ga.net>

http://www.v3ga.net/processing/BlobDetection/index-page-download.html

Edge Detection class is extended to implement Blob Detection. The
idea to compute blobs is to go over each cell of the grid until we
find a voxel which is cut by an edge (green segment). Then, we
propagate over the surface by ‘surfing’ along the green segments.
Thus, when we’ve made a loop, we know that we have found a blob, we
can compute its properties (center, dimensions) because during the
propagation, we saved the min/max values of the bounding box. Note
that this algorythm does not perform blobs tracking, it simply tries
to find blobs each new video frame.

BEGIN

Plug in IR modified webcam to firewire port
Turn on IR Emitters

Initiate Terminal Window

CD (change directory focus) to FLOSC Directory
newg5-05:~ labuser$ cd /Users/labuser/Desktop/multitouch/flosc-0.3.1/

Initiate FLOSC Java Gateway
newg5-05:~/Desktop/multitouch/flosc-0.3.1 labuser$ java Gateway 1250 3000

Open/Start Processing PDE file
(/multitouch/Processing 0124/oscP5/examples/osP5webcam/bd_webcam.pde)
PDE file must have ports set to 1250 default.

Open[flosc032107.fla] file
(/multitouch/flosc-0.3.1/flosc032107.fla)

Start Flash file, [Connect] to server (IP 127.0.0.1, Port 3000)

PDE file should start datastreaming the dataset/coordinate information over the open ports so Flash can consume and execute… Only one webcamera object can be access at a time (in Flash AS2). So it’s good to keep in mind that either Processing is using the webcam feed or Flash is using the webcam preview feed but not both.