Use Processing and Kinect in Ubuntu

-
I'm working on a project that uses Kinect depth camera to detect a person silhouette as I have mentioned before in this blog.

Part of the development was done using a Mac because I found a library that enabled me to use Kinect directly from Processing (which I've found very convenient for prototyping).

It did not hurt that an OpenCV library for processing was also available.

What was missing was a version of the Kinect's Processing library to be used with GNU/Linux. Fortunately, someone took the time to tweak the Mac version to make it work and to post the process online. It did not work for me exactly as it was but it was close. So just in case you want to experience with it, here are the steps that worked for me:

  1. Get a copy of the library: 
    git clone git://github.com/shiffman/libfreenect.git
  2. Create a and move to a build directory inside libfreenect folder 
    cd libfreenect

mkdir build

cd build

cmake ..

make 

sudo make install
  3. Go to wrappers/java and edit build.sh file (remove -m64 compiler flag unless your platform is 64-bits)
  4. Make sure the folders for libfreenect.a file and for the Java directory containing jni.h are correct in the build.sh file
  5. Run build.sh:
    sh build.sh
  6. Make sure the step above was error free.
  7. Now copy the folder openkinect from wrappers/java/processing/distribution to your sketchbook/libraries folder
  8. Copy file libOpenKinect.so from wrappers/java/dist to sketchbook/libraries/openkinect/library/
  9. Almost there, create a symbolic link in that last folder: 
    ln -s libOpenKinect.so libkinect.so
  10. Fire up Processing IDE and try any of the examples from the openkinect library once you have connected your Kinect to the USB port.
Update: In case of trouble do not forget all the install details provided here.If you're using Ubuntu 11.04, the following packages need to be installed: git-core, cmake, libusb-1.0, g++, libxmu-dev, bixi-dev, libglut3-dev. If it happens to be a 64bit Ubuntu it won't work (I'm still figuring out how to make it work).

Fixed! In case you have 64 bit ubuntu, make sure your processing folder does not include java binaries (if it does, it'll likely be 32 bit, as it was the case with my processing 1.5.1). Once you're using 64 bit java, the 64 bit library created with the instructions above will work nicely too.

Comments

Philipp said…
Miguel - thank you very much for your step-by-step tutorial.

I get an error running the build.sh

"build.sh: 16: JDK_HOME=: not found
In file included from OpenKinectJNI/org_openkinect_Context.cpp:32:0:
OpenKinectJNI/org_openkinect_Context.h:2:17: fatal error: jni.h: No such file or directory
compilation terminated.
In file included from OpenKinectJNI/org_openkinect_Device.cpp:29:0:
OpenKinectJNI/org_openkinect_Device.h:2:17: fatal error: jni.h: No such file or directory
compilation terminated."


I would really appreciate, if you could post an example how the Java directory containing jni.h are correct in the build.sh file.

Thank you very much in advance!!
9:20 pm
misan said…
Hi Philipp,

I guess the JDK_HOME variable is not properly set.

Edit build.sh and make sure the proper value is set after it says:

then JDK_HOME="/usr/lib/jvm/java-6-sun-1.6.0.24/"

(the value shown above is ok for my system).

I did another change to the build.sh file:

LIBFREENET_LIBRARY=/home/misan/Kinect/shiffman/libfreenect/build/lib/libfreenect.a

(just put the proper path for libfreenect.a in your system).
10:02 pm
Philipp said…
Miguel - thank you for the help with the build.sh. Everything worked without error.

I try to run an example in processing but it give me an error in the line of code:

kinect.enableDepth(true);


error:
libusb couldn't open USB device /dev/bus/usb/001/004: Permission denied.
libusb requires write access to USB device nodes.
Exception in thread "Animation Thread" java.lang.NullPointerException
at org.openkinect.processing.Kinect.enableDepth(Kinect.java:82)
at AveragePointTracking$KinectTracker.(AveragePointTracking.java:111)
at AveragePointTracking.setup(AveragePointTracking.java:40)
at processing.core.PApplet.handleDraw(Unknown Source)
at processing.core.PApplet.run(Unknown Source)
at java.lang.Thread.run(Thread.java:662)



I reinstalled libreenect, added myself to video, created the udev rules.

Any hint what the problem above could be? Do i have to run processing with sudo?
11:29 pm
misan said…
Hi Philipp:

Did you get C++ examples from libfreenect working?

I think I have forgotten to mention one important step. To make the

sudo make install

(that would create the udev rules).

Please try to do that to see if works.
12:07 pm
Philipp said…
Miguel - SOLVED.

Thank you very much for your help.

Everything works fine now.

I followed your instructions (yes the glview example worked perfect).

I also re-installed the Java Sun JDK for Ubuntu 11.04 from with this link:
http://www.webupd8.org/2011/03/install-java-in-ubuntu-1104-natty.html

Again, thanks a lot!
6:09 pm
Anonymous said…
Miguel, thanks a lot for the help, everything worked after I re-installed the Java sun jdk.

I followed you instructions above and in the latest post.
10:09 pm
misan said…
You're welcome. Glad to be useful.
10:55 pm
Anonymous said…
Hi, thanks for your guide.
I've gotten everything compiled without errors, on 64-bit Ubuntu 11.10, but can't get the kinect examples to work in processing. I get this error:

java: symbol lookup error: /home/trix/sketchbook/libraries/openkinect/library/libOpenKinect.so: undefined symbol: libusb_init

Does anyone have any ideas?
2:17 pm
misan said…
I'm assuming you have libusb-1.0 already installed in your system. The complaint is about not being able to find libusb init code.

You can try #openkinect channel on IRC.
4:57 pm
MikeTheBee said…
I found a solution to the libusb_init error at http://libusb.6.n5.nabble.com/Compile-in-GCC-td7048.html

i.e. adding
-lusb-1.0 to the gcc command.
4:43 pm
misan said…
Thanks a lot for sharing it Mike!
12:01 am
Post a Comment

Popular posts from this blog

VFD control with Arduino using RS485 link

-
Image
Variable Frequency Drives ( VFD ) allow the control of spindles so speed can accurately be controlled and a detailed acceleration profile for the spindle and reverse rotation can all be handled. In essence a VFD is an three phase inverter for three-phase AC motor. I am using a popular (I mean cheap) Chinese VFD and though the reference manual is not great, I could see there is a built-in RS-485 port. I usually control the start, stop and speed selection using the keyboard on the unit but I thought it will be more useful if I could control everything from the same Arduino is doing our CNC table control. Some cheap RS-485 off eBay and some lines of code later, I can start, stop and change the speed from an Arduino. What a cool thing to have! Some configuration of your VFD are needed before you can use it like that.  You need to set PD163=1 (I am using address 1 in the code). PD=164=1 (for setting serial to 9600bps) and PD165=0 (for using ASCII and 8N1 character format)
Read more

How to get sinusoidal s-curve for a stepper motor

-
Image
After some experimentation, and delving a bit into the math of motion, I realized there can be a significant difference on the motion of a machine by just making small changes. A common and popular approach to smooth one axis of motion is to use a trapezoidal speed pattern, where acceleration and deceleration are uniformly accelerated movements (aka constant acceleration motion) separated by a portion happening at cruising speed. The problem comes with the fact that, while that approach is simple to understand and to code, it does contain sudden changes in the acceleration, that show as spikes on the acceleration derivative, called jerk. These sudden changes in the acceleration translate into undesired vibration in our machines. Most of that can be eliminated by selecting a motion pattern that does not contain sudden changes of acceleration. If we chose a mathematical function for our speed whose second derivative is continuous, then we reduce system vibration quite a lot. One ma
Read more

Importing OpenSCAD designs into Onshape

-
Image
After using Onshape CAD software for a while I am really fond of it. So one natural thing to do, specially once they have released the Instructor's kit is to start using it in the classroom too. But one thing I was not so sure how to do was to use old designs I made using OpenSCAD. It does not mean I am completely quitting using OpenSCAD but suddenly I can see I can do assemblies in Onshape that will look quite nice for showing and documentation purposes if only I could easily import from OpenSCAD. I am glad to report that the route OpenSCAD --> FreeCAD --> Onshape worked brilliantly. Even more when a few tricks were applied. While some people suggest to import CSG files from OpenSCAD to FreeCAD, I have had more success by directly importing SCAD files into FreeCAD. One trick that worked well for me was to get rid of $fn references, so now circles become real circles (or cylinders) in FreeCAD. If a part keeps some $fn references then your cylinders or circles wi
Read more