2202 – Hand tracking in Flash – nothing to install

When I finally got my hands on a Kinect a few days ago, I already knew that getting it to work with Flash on OSX wouldn’t be the easiest of tasks… but it was worst than I expected.
I got scared on the first half of any page giving instructions: how to install OpenNI, NITE, drivers, OSCeleton, messing with terminal, developer tools, Java servers for hours or even days…

Since I tend to think “plug A to B, plug B to C, plug C to D, it’s working, I did it!”, I started testing different ways to get the skeleton values from the Kinect into Flash EASILY, by combining different softwares as bridges… and it worked. Free and friendly softwares that bring the message from Kinect to Flash.

As the title says: nothing to install. Just download and run.

Ok, following certain steps. But don’t be scared, this post is long just because I use too many words.
By the way, for those interested in the long, detailed version, click here.
And remember: it’s only been tested on Snow Leopard but should work in Lion as well.


1. You need to download:
*Synapse: http://synapsekinect.tumblr.com
*OSCulator http://www.osculator.net/download
*An OSCulator file that keeps the communication flowing + the .fla source tracking both hands: http://www.zeladores.net/2202.zip
*The NETLab toolkit: http://www.netlabtoolkit.org/

2. Copy the contents of the “NETLabFlashWidgets_3_2_1” folder to
** Flash CS3 folder /Configuration/Components
** Flash CS4 or CS5 /Common/Configuration/Components
2b. Do a GET INFO (command-I) on the NETLabHub.app + in the info window check the “Open in 32 bit mode”!! (sorry, I forgot to mention it on the video)


Hand tracking in Flash is now a 1′ operation:
– Plug your Kinect and be sure you downloaded all files and followed the steps for the NETLab toolkit (copy widgets, check The Hub to run “32-bit”)
– Open the file “trackhands.oscd” in OSCulator.
– Run Synapse.
– Run The Hub.
– Open the trackhands.swf (you published, remember?)
– Do the cactus position and hold it for a few seconds until Synapse detects your skeleton: the trackers in the swf should start following your hands.

I hope you enjoy being able to play with your Kinect almost inmediately. If you’re able to do it on other OS versions, please let me know so I can update the post.
And please, don’t forget to leave your comments and as many ideas as possible.


2202 – Hand tracking in Flash – extended version

If you’re eager to start hand tracking in Flash (OSX), right away, click here.

Otherwise, let’s talk about how it happened

The only problem with the otherwise fantastic Synapse (besides its lack of RGB image) is that it needs “keepalives” to remind her every 2-3 seconds to keep sending us the joints information through OSC.
So, for the right hand position on screen, I need to send an OSC message “/righthand_trackposition 3″ to the port 12346 (where Synapse listens) every 2-3”
I achieved it “crafting” a configuration file in OSCulator with two “loops”. One to keep Synapse sending information, and another to re-send that info to my port of choice: 2202.
Where you can listen to the values of these joints:


In my example, I pick the 3D position of both hands, and use that info to move and scale two “trackers” in Flash.
I also chose a naming based on the assumption of a repeating task of 6 NETLab Widgets per joint

For the right hand x position
1. Drag an AnalogIn Widget from the “Components” Panel and give him the instance name of s0x (“s” for source, “0” for being the first joint, “x” because this widget gets the x value for us)

2. Using the “Component Parameters” panel, set:
controller: osc
controllerInputNumber: 0 (for the x value, since there are 3 values coming. x is 0, y is 1, z is 2)
controllerIP: (I always work localhost)
controllerPort: 2202 (from OSCulator, remember?)
oscMultiplier: 1
oscString: /rhand (which is the name I gave to that message when rerouting from OSCulator)

3. Drag an Insert Widget (to make that “x” value available in Actionscript). Give it the instance name of i0x (“i” for insert, etc)

4. In the Insert widget’s “Component Parameters” panel, set:
inputsource: s0x

Repeat this process for the right hand’s “y” position, just changing any “x” in the instances names and source references for a “y”, and moving the controllerInputNumber to 1 for y.
Repeat again to get the z position. Changed for “z”, controllerInputNumber to 2.
And for the left hand, same process another 3 times. Just changing the oscString from “/rhand” to “/lhand”, and the names of the instances from “s0x” to “s1x”, from “s0y” to “s1y” and so on…
This way, after another 84 Widgets (i14z should be the one) you’d be tracking the full skeleton.

Anyway, I “got” the positions in 3D space for my left and right hands in Flash. Cool.

But I needed to call all 6 Insert Widgets (rightxyz, leftxyz…) before I could use their values to manipulate my “trackers” (2 movieclips named tracker0 and tracker1)
I used the x and y values to move the trackers, and the z value to scale them.

actionscript code:
// Pull the values from each Insert widget and transform the trackers “tracker0” and “tracker1” on stage.
var trackeditems:Number = 2;
var nome:String;
var j:Number;

for (var i:uint = 0; i<trackeditems; i++) {
var ix:String = “i”+i+”x”;
this[ix].insertInput = posx;
var iy:String = “i”+i+”y”;
this[iy].insertInput = posy;
var iz:String = “i”+i+”z”;
this[iz].insertInput = posz;
function posx (inputValue, id) {
j = id.substr(1,1);
nome = “tracker”+j;
this[nome].x = inputValue;
function posy (inputValue, id) {
j = id.substr(1,1);
nome = “tracker”+j;
this[nome].y = inputValue;
function posz (inputValue, id) {
j = id.substr(1,1);
nome = “tracker”+j;
this[nome].scaleX = this[nome].scaleY = (400-inputValue)/100;

Did I mention that I’m not a programmer but a humble copypaster? 😉

Enjoy and please, don’t forget to post your comments and share your ideas.

Blog at WordPress.com.