Use Processing and Kinect in Ubuntu

I'm working on a project that uses Kinect depth camera to detect a person silhouette as I have mentioned before in this blog.

Part of the development was done using a Mac because I found a library that enabled me to use Kinect directly from Processing (which I've found very convenient for prototyping).

It did not hurt that an OpenCV library for processing was also available.

What was missing was a version of the Kinect's Processing library to be used with GNU/Linux. Fortunately, someone took the time to tweak the Mac version to make it work and to post the process online. It did not work for me exactly as it was but it was close. So just in case you want to experience with it, here are the steps that worked for me:

  1. Get a copy of the library:
    git clone git://github.com/shiffman/libfreenect.git
  2. Create a and move to a build directory inside libfreenect folder
    cd libfreenect
    
    mkdir build
    
    cd build
    
    cmake ..
    
    make 
    
    sudo make install
  3. Go to wrappers/java and edit build.sh file (remove -m64 compiler flag unless your platform is 64-bits)
  4. Make sure the folders for libfreenect.a file and for the Java directory containing jni.h are correct in the build.sh file
  5. Run build.sh:
    sh build.sh
  6. Make sure the step above was error free.
  7. Now copy the folder openkinect from wrappers/java/processing/distribution to your sketchbook/libraries folder
  8. Copy file libOpenKinect.so from wrappers/java/dist to sketchbook/libraries/openkinect/library/
  9. Almost there, create a symbolic link in that last folder:
    ln -s libOpenKinect.so libkinect.so
  10. Fire up Processing IDE and try any of the examples from the openkinect library once you have connected your Kinect to the USB port.
Update: In case of trouble do not forget all the install details provided here.If you're using Ubuntu 11.04, the following packages need to be installed: git-core, cmake, libusb-1.0, g++, libxmu-dev, bixi-dev, libglut3-dev. If it happens to be a 64bit Ubuntu it won't work (I'm still figuring out how to make it work).

Fixed! In case you have 64 bit ubuntu, make sure your processing folder does not include java binaries (if it does, it'll likely be 32 bit, as it was the case with my processing 1.5.1). Once you're using 64 bit java, the 64 bit library created with the instructions above will work nicely too.

Comments

Philipp said…
Miguel - thank you very much for your step-by-step tutorial.

I get an error running the build.sh

"build.sh: 16: JDK_HOME=: not found
In file included from OpenKinectJNI/org_openkinect_Context.cpp:32:0:
OpenKinectJNI/org_openkinect_Context.h:2:17: fatal error: jni.h: No such file or directory
compilation terminated.
In file included from OpenKinectJNI/org_openkinect_Device.cpp:29:0:
OpenKinectJNI/org_openkinect_Device.h:2:17: fatal error: jni.h: No such file or directory
compilation terminated."


I would really appreciate, if you could post an example how the Java directory containing jni.h are correct in the build.sh file.

Thank you very much in advance!!
Miguel Sánchez said…
Hi Philipp,

I guess the JDK_HOME variable is not properly set.

Edit build.sh and make sure the proper value is set after it says:

then JDK_HOME="/usr/lib/jvm/java-6-sun-1.6.0.24/"

(the value shown above is ok for my system).

I did another change to the build.sh file:

LIBFREENET_LIBRARY=/home/misan/Kinect/shiffman/libfreenect/build/lib/libfreenect.a

(just put the proper path for libfreenect.a in your system).
Philipp said…
Miguel - thank you for the help with the build.sh. Everything worked without error.

I try to run an example in processing but it give me an error in the line of code:

kinect.enableDepth(true);


error:
libusb couldn't open USB device /dev/bus/usb/001/004: Permission denied.
libusb requires write access to USB device nodes.
Exception in thread "Animation Thread" java.lang.NullPointerException
at org.openkinect.processing.Kinect.enableDepth(Kinect.java:82)
at AveragePointTracking$KinectTracker.(AveragePointTracking.java:111)
at AveragePointTracking.setup(AveragePointTracking.java:40)
at processing.core.PApplet.handleDraw(Unknown Source)
at processing.core.PApplet.run(Unknown Source)
at java.lang.Thread.run(Thread.java:662)



I reinstalled libreenect, added myself to video, created the udev rules.

Any hint what the problem above could be? Do i have to run processing with sudo?
Miguel Sánchez said…
Hi Philipp:

Did you get C++ examples from libfreenect working?

I think I have forgotten to mention one important step. To make the

sudo make install

(that would create the udev rules).

Please try to do that to see if works.
Philipp said…
Miguel - SOLVED.

Thank you very much for your help.

Everything works fine now.

I followed your instructions (yes the glview example worked perfect).

I also re-installed the Java Sun JDK for Ubuntu 11.04 from with this link:
http://www.webupd8.org/2011/03/install-java-in-ubuntu-1104-natty.html

Again, thanks a lot!
Anonymous said…
Miguel, thanks a lot for the help, everything worked after I re-installed the Java sun jdk.

I followed you instructions above and in the latest post.
Miguel Sánchez said…
You're welcome. Glad to be useful.
Anonymous said…
Hi, thanks for your guide.
I've gotten everything compiled without errors, on 64-bit Ubuntu 11.10, but can't get the kinect examples to work in processing. I get this error:

java: symbol lookup error: /home/trix/sketchbook/libraries/openkinect/library/libOpenKinect.so: undefined symbol: libusb_init

Does anyone have any ideas?
Miguel Sánchez said…
I'm assuming you have libusb-1.0 already installed in your system. The complaint is about not being able to find libusb init code.

You can try #openkinect channel on IRC.
Mike Beardmore said…
I found a solution to the libusb_init error at http://libusb.6.n5.nabble.com/Compile-in-GCC-td7048.html

i.e. adding
-lusb-1.0 to the gcc command.
Miguel Sánchez said…
Thanks a lot for sharing it Mike!

Popular posts from this blog

VFD control with Arduino using RS485 link

4xiDraw: Another pen plotter

Software I2C for Arduino