I went a slightly different route. In org.openscience.jmol.app, I
created a tiny class ImageFeedback extends JPanel. Its JFrame isn't
visible until setImage(BufferedImage) is called. JmolPanel creates
an ImageFeedback when the entry in Tools is selected and passes it
on to the viewer via the new
JmolViewer.setImageFeedback(ImageFeedback). JmolOpenNIAdapter now
has access to viewer.imageFeedback and can call setImage() in the
OpenNI update loop.
This is ready to go - I delivered a beta version to our chemistry
department with everything except ImageFeedback. My work is based
on Jmol svn commit 7ae3f6dee82a027a8437a4edc760c6c53aac937a
(September 8). My changes to existing code are pretty minimal, so I
don't expect any issues when I update and rebase. I've read the
Jmol Developers' Guide; what else do I need to know about
One issue: in JmolOpenNIAdapter.setMultiTouchClient(), I call
viewer.setSelectionHalos(true), but this doesn't appear to work.
The code has much more documentation than the last time you saw it,
but it's lacking in end-user documentation. A formal user manual is
on my todo list. I'm also planning to write an announcement and
call for testers to post to Jmol-users; this will contain an updated
version of the below instructions.
1) To see the live image captured from the sensor, go to
Tools->OpenNI Image Feedback.
2) Stand in front of the sensor and perform a Click - move open hand
straight toward the sensor and back. This hand is the primary
hand. Use it to select the current action by making gestures.
3) Move your other hand into the scene until it's recognized - NITE
can be finicky about this and sometimes gets into a create-destroy
cycle with the second hand. It takes some practice to find optimal
distances from the sensor and between your hands*.
4) Perform a Push (same as Click) with the primary hand to enter
selection mode. A cursor will appear in the Jmol viewer. Move the
secondary hand to the desired target and perform a Push with the
primary hand to select - default is to select all atoms within 20
Jmol units of the selection cursor.
5) Swipe the primary hand up to do a selectAll.
6) Swipe the primary hand down to do a selectNone.
7) Wave the primary hand (a Wave is at least l-r-l or r-l-r) to
enter translation mode. Move the secondary hand to translate all
8) Move the primary hand in a Circle to enter rotation mode. Move
the secondary hand to rotate all selected atoms.
9) Repeat 4, 5, 6, and 7 as desired.
Note: You must install OpenNI and NITE first. I highly recommend
using the unstable versions. I successfully followed this guide
for OS X. The process is similarly documented for Windows and
Linux elsewhere on the web. For Gentoo Linux users, I've got some
ebuilds that I'll try to get into an appropriate overlay soon.
Note: You must find the file Hands_1_5_2/Nite.ini (location
differs by platform) after installing NITE and remove the leading
semicolons from AllowMultipleHands=1 and TrackAdditionalHands=1.
This software will not work without that change.
*Note: HandTracker spits out a lot of useful information when its
field info==true (default: true). Open up Help->Jmol Java
Console to see it.