The OS X installer puts libturbojpeg.dylib in /opt/TurboVNC/lib/, but the vncviewer executable looks for it in /opt/VirtualGL/lib/. A minor issue for a slightly technical person (which most users will have to be anyway, to even try to run this software), but nevertheless something that should probably be fixed.
Oops. Thanks for pointing that out. I've checked in a fix, and it will be in the upcoming 0.4 release.