ofxWorkbench version 1.8 as of 20.March.2020
--------------------------------------------
The ofxWorkbench framework is free software. It is licensed under the
GNU GPL 3.0 license.
CONTENTS
--------
* Introduction
* Requirements
* Installation
* Usage
* FAQ
* Documentation
* Developers
* Credits
INTRODUCTION
------------
This software provides the source code for running virtual reality
applications on a 3D workbench. A 3D workbench (often referred to as a
holobench) is a virtual work station which displays three-dimensional
objects on a virtual workbench table by employing 3D active stereo
projection techniques. It is possible to interact with the displayed
scenes and objects via user tracking. For that purpose the Kinect and
Leap input devices are utilized.
The software also supports non-stereo applications with multiple
projectors like tiled wall or floor projections. In those settings the
projectors are calibrated so that the alignment of the projection
tiles is seamless. The Kinect is used to track the viewer's head. This
is neccessary to compensate the perspective distortions introduced by
varying eye positions of the viewer.
REQUIREMENTS
------------
This software requires the following prerequisites and libraries:
* Base installation of Ubuntu 16.04 LTS or 18.04 LTS
* NVidia OpenGL graphics drivers
* Openframeworks (http://openframeworks.cc/)
* Qt (https://download.qt.io/archive/qt/)
* glVertex (https://sourceforge.net/projects/glvertex/)
* libbullet
* libnvstusb
* libfreenect2
* liboscpack
* OpenCv
* VVV (Versatile Volume Renderer)
This software requires the following matching hardware setup:
* a workbench construction or projection canvas
* a Linux computer with NVIDIA graphics card driving a three display stereo setup
* e.g. NVIDIA GTX 1070
* two DLP-Link 120 Hz stereo projectors for active stereo setups
* e.g. Optoma W304M
* DLP-Link stereo glasses
* or a set of projectors for tiled non-stereo projections
* e.g. NEC P554U
* a Kinect V2 (with flashed firmware for developers using the Microsoft SDK)
* optional: a Leap Motion controller
INSTALLATION
------------
For installing Openframeworks see: http://openframeworks.cc/download/
* make sure you have the following folder structure:
~~~~
base_folder > Openframeworks > addons
> apps
> ...
> ofxworkbench > trunk
> tags
> ...
~~~~
* "base_folder" is a location of your choice
* "Openframeworks" is the folder which contains the unpacked Openframeworks framework and must be located as shown above
* "ofxworkbench" is the repository which contains this software
--- Installation on Ubuntu 16.04 LTS ---
For installing NVIDIA OpenGL graphics drivers
* install the latest NVIDIA driver package
* reboot for the kernel module to load
For installing OF
* download OF v0.9.8 for Linux 64bit
* from https://www.openframeworks.cc/versions/v0.9.8/of_v0.9.8_linux64_release.tar.gz
* unpack OF and rename it to "Openframeworks"
* make sure that it is located beneath this software (see folder structure above)
* run the following script with sudo:
* sudo ./Openframeworks/scripts/linux/ubuntu/install_dependencies.sh
* when prompted during installation press Enter to proceed
* to install the sound codecs (mp3 etc.) run the following script with sudo:
* sudo ./Openframeworks/scripts/linux/ubuntu/install_codecs.sh
* when prompted during installation press Enter to proceed
For installing Qt
* download the respective Linux installer for Qt 5.7.1
* from https://download.qt.io/archive/qt/5.7/5.7.1/qt-opensource-linux-x64-5.7.1.run
* set the executable bit: chmod +x *.run
* and run the installer with sudo: sudo ./*.run
* install Qt in /opt/Qt5.7.1
For installing glVertex type the following in the ofxworkbench/deps/glvertex folder:
* sudo ./install.sh
For installing libbullet
* download bullet3 2.83
* from https://github.com/bulletphysics/bullet3/archive/2.83.7.tar.gz
* and type the following in the bullet folder:
* cmake -DUSE_DOUBLE_PRECISION=ON -DINSTALL_EXTRA_LIBS=ON . && make && sudo make install
For installing libnvstusb type the following in the ofxworkbench/deps/libnvstusb folder:
* sudo apt-get install libusb-1.0-0-dev libdevil-dev
* sudo apt-get install automake libtool
* ./autogen.sh && ./configure && make && sudo make install
For installing libfreenect2 type the following in the base folder:
* sudo apt-get install libglfw3-dev libturbojpeg libjpeg-turbo8-dev
* git clone https://github.com/OpenKinect/libfreenect2.git
* cd libfreenect2/
* cmake -DBUILD_SHARED_LIBS=OFF -DGLFW3_LIBRARY=/usr/lib/x86_64-linux-gnu/libglfw.so . && make && sudo make install
* sudo cp platform/linux/udev/90-kinect2.rules /etc/udev/rules.d/
* reboot for updated usb permissions to take effect
* optionally: plug in the Kinect and run bin/Protonect to see if the Kinect is detected
For installing liboscpack type the following in the base folder:
* git clone https://github.com/MariadeAnton/oscpack.git
* cd oscpack
* make COPTS="-O3 -fPIC" && sudo make install
For installing OpenCV type the following:
* sudo apt-get install libopencv-dev
For compiling and installing the VVV type the following in the ofxworkbench/deps/vvv folder:
* cmake . && make
* sudo make install
--- Installation on Ubuntu 18.04 LTS ---
For installing NVIDIA OpenGL graphics drivers
* make sure that the kernel is from the 4.0.x series
* install the latest NVIDIA driver package for this kernel
* sudo apt-get install nvidia-384
* reboot for the kernel module to load
For installing OF
* download OF v0.11.0 for Linux 64bit gcc6
* from https://openframeworks.cc/versions/v0.11.0/of_v0.11.0_linux64gcc6_release.tar.gz
* unpack OF and rename it to "Openframeworks"
* make sure that it is located beneath this software (see folder structure above)
* run the following script with sudo:
* sudo ./Openframeworks/scripts/linux/ubuntu/install_dependencies.sh
* when prompted during installation press Enter to proceed
* to install the sound codecs (mp3 etc.) run the following script with sudo:
* sudo ./Openframeworks/scripts/linux/ubuntu/install_codecs.sh
* when prompted during installation press Enter to proceed
For installing Qt
* download the respective Linux installer for Qt 5.12.7
* from https://download.qt.io/archive/qt/5.12/5.12.7/qt-opensource-linux-x64-5.12.7.run
* set the executable bit: chmod +x *.run
* and run the installer with sudo: sudo ./*.run
* install Qt in /opt/Qt5.12.7
For installing glVertex type the following in the ofxworkbench/deps/glvertex folder:
* sudo ./install.sh
For installing libbullet
* download bullet3 2.83
* from https://github.com/bulletphysics/bullet3/archive/2.83.7.tar.gz
* and type the following in the bullet folder:
* cmake -DUSE_DOUBLE_PRECISION=ON -DINSTALL_EXTRA_LIBS=ON . && make && sudo make install
For installing libnvstusb type the following in the ofxworkbench/deps/libnvstusb folder:
* sudo apt-get install libusb-1.0-0-dev libdevil-dev
* sudo apt-get install automake libtool
* ./autogen.sh && ./configure && make CFLAGS="-Wno-error" LIBS="-lm" && sudo make install
For installing libfreenect2 type the following in the base folder:
* sudo apt-get install libglfw3-dev libturbojpeg libjpeg-turbo8-dev libturbojpeg0-dev
* git clone https://github.com/OpenKinect/libfreenect2.git
* cd libfreenect2/
* cmake -DBUILD_SHARED_LIBS=OFF -DGLFW3_LIBRARY=/usr/lib/x86_64-linux-gnu/libglfw.so . && make && sudo make install
* sudo cp platform/linux/udev/90-kinect2.rules /etc/udev/rules.d/
* reboot for updated usb permissions to take effect
* optionally: plug in the Kinect and run bin/Protonect to see if the Kinect is detected
For installing liboscpack type the following in the base folder:
* git clone https://github.com/MariadeAnton/oscpack.git
* cd oscpack
* make COPTS="-O3 -fPIC" && sudo make install
For installing OpenCV type the following:
* sudo apt-get install libopencv-dev
For compiling and installing the VVV type the following in the ofxworkbench/deps/vvv folder:
* cmake . && make
* sudo make install
--- Installation on Ubuntu 19.04 - 19.10 ---
same as above
COMPILATION
-----------
For compiling the ofxworkbench software:
* open the ofxworkbench/trunk/HoloPlayground.qbs file with QtCreator
* and click at the green play symbol for building and running the software
* in release mode the default scene is shown with reprojection
* in debug mode a test scene is shown without reprojection
Optionally, we may compile the "sendosc" command line tool for sending osc commands:
* git clone https://github.com/yoggy/sendosc.git
* cd sendosc
* cmake . && make
* sudo make install
* example usage:
* sendosc localhost 12345 /command s reset
Optionally, we may compile and run the "RemoteOSC" UI for sending osc commands:
* qmake remoteosc.pro && make
* remoteosc/RemoteOSC
USAGE
-----
On the command line the workbench application is started from the bin
directory in the trunk as follows:
~~~~
./playground
~~~~
By default, the start scene is the main menu and the projection
geometry is setup for a workbench with a bottom and back projection
plane and two top projectors being directed at those two planes. The
start scene can be overriden by specifying additional command line
arguments:
~~~~
./playground [<start scene>] [<default scene>] [<teaser scene>]
~~~~
For example:
~~~~
./playground drawgame menu teaser
~~~~
or simply
~~~~
./playground drawgame
~~~~
To run the playground application with dual projectors in stereo mode,
a proper xorg.conf triple head configuration needs to be setup, where
the first logical screen is connected to the LCD monitor and the
second logical screen is a side-by-side twin-view feeding the two
stereo projectors. Both projectors need to run with 120 Hz in DLP Link
mode. Then the software drives the dual projectors, once it is started
from the command line as follows:
~~~~
DISPLAY=:0.1
./playground
~~~~
The following environment variables control the application behavior:
* WB_LANGUAGE overrides the language for text translations (e.g. EN, DE)
* WB_GEOMETRY_TYPE overrides the type of the projection geometry setup (e.g. workbench, floor, dual_side_floor, etc.)
* WB_PROJECTOR_TYPE overrides the type of the projector (e.g. "NEC P554U" for WB_GEOMETRY_TYPE = floor)
* WB_MONITOR_TYPE overrides the type resp. size of the monitor (e.g. "27" for WB_GEOMETRY_TYPE = monitor)
* WB_START_SCENE overrides the start scene (e.g. ballroom)
For example, setting WB_GEOMETRY_TYPE to "dual_side_floor",
WB_PROJECTOR_TYPE to "default" (Optoma W304M) and WB_START_SCENE to
"railway_demo" will show a virtual railway scene using a floor
projection with a stack of two side projectors. Floor projections
would benefit from active 3D stereo rendering, but will also show well
without a stereo projection setup.
The calibration and alignment of the projectors is done by starting
the "calibration" application, which also controls the display of the
calibration markers and the stereo mode. For the calibration to work
properly, the projection geometry type (as defined by WB_GEOMETRY_TYPE
and WB_PROJECTOR_TYPE) needs to match the setup used by the workbench
application.
The position of the tracked user will be forwarded to the workbench
software while the "tracking" application is running concurrently (on
the same or a different computer on the local network). The tracking
application utilizes the Kinect to track the head positions. If no
Leap device is present, it will track bright red objects like red
gloves to determine the position of the virtual hand. The projection
geometry type needs to match the setup used by the workbench
application.
DEVELOPMENT
-----------
If you want to develop your own scene, you need to create a new C++
module that is derived from WorkbenchScene and implement the following
virtual methods:
* workbenchSceneName: e.g. returning "My first little scene"
* workbenchSceneId: e.g. returning "my scene"
* workbenchSceneGeometryType: e.g. returning "floor"
* workbenchSceneProjectorType: e.g. returning "NEC P554U"
* setupScene -> create, position and add workbench scene objects (objects derived from WorkbenchObject)
* for example add a bouncing sphere at the workbench center 1 meter above the ground:
* addObject(new WorkbenchObjectSphere(ofVec3f(0,0,1), 0.1f, 0.5f, "my sphere"));
* the above sphere has a radius of 10cm and weight of 500g
* updateScene() -> update scene objects (usually empty, if all objects are derived from WorkbenchObject)
* renderScene() -> render scene (usually empty, if all objects are derived from WorkbenchObject)
Then compile in release mode and start the program with the above
scene id as command line argument and your scene will show up
automatically after program start.
Your scene will be using a floor projection for a single top-down
projector positioned about 4 meter off the ground. To see the
unprojected scene for developing and debugging purposes, press 'd'.
As a reference, have a look at the "Ballroom" scene as defined in
the scenes/simpleScene.cpp module, which just shows a single bouncing
football.
To see multiple spheres interacting physically, have a look at the
"Holo Playground" scene as defined in the scenes/holoPlayground.cpp
module. This scene is using a dual projector setup.
For a more elaborated projection demo, have a look at the
scenes/railwayDualPlayerRace.cpp module. You can load that scene by
starting the program as follows:
bin/playground railway_race
Note: For the latter scene language translations are available for
"EN" and "DE". To add your own translations, you need to pass the
English text to be translated through the TRANS() method, and add the
foreign translations to the workbench dictinary.
FAQ
---
Q: Can I use the holo playground software without the workbench
construction and its hardware?
A: For proper immersive usage and interaction the hardware modules are
required. However, you can also run the application without the 3D
input devives and navigate through the VR scenes by using mouse and
keyboard shortcuts. The movement of the 3D virtual hand can be
simulated with left resp. right mouse drags. Key 'g' simulates a grab
gesture. Key 'w' simulates a wipe gesture. A scene is entered or
exited with a wipe gesture.
List of basic key commands:
* Space : show teaser scene
* Enter : enter or exit scene
* Arrows : choose scene in menu
* Key q : quit application
* Key t : toggle kinect tracking
* Key 0 : disable stereo rendering
* Key a : enable anaglyph stereo mode
* Key 3 : enable NVIDIA 3D Vision stereo mode
* Key s : enable DLP Link stereo mode
* Key f : toggle fullscreen mode
* Key w : wipe gesture
* Key g : grab gesture
* Key d : toggle graphical debug mode to show unprojected virtual scene
* Key + : in graphical debug mode zoom into the scene
* Key - : in graphical debug mode zoom out of the scene
List of special key commands:
* Key S : select nearest object
* Key I : show info and properties of selected object
* Key D : delete selected object
* Key U : place grabbed, touched or selected object upright
* Key P : reset position of touched or selected object
* Key T : toggle selected object
* Key R : reset actual scene
* Key H : toggle virtual hand
* Key M : mute sound
* Key 1 : toggle shadow rendering
* Key 2 : toggle black & white rendering
* Key 9 : toggle contrast enhancement
* Key F : show actual frame rate
* Key C : show projected calibration crosses
Q: Can I add my own key commands in my own scenes?
A: Yes, the lowest row of the key board is free: <>yxzcvbnm,;.:-_
Q: I can't get the 120 Hz stereo mode to work - it always runs at 60 Hz.
A: You have to run a non-compositing window manager like TWM!
Compositing window managers like Unity limit the frame rate to 60 Hz.
DOCUMENTATION
-------------
To generate the class documentation of the software,
type "doxygen" in the "trunk" folder.
This requires doxygen and graphviz to be installed:
* sudo apt-get install doxygen graphviz
Then the HTML class documentation can be found
in the "Documentation/html" folder.
DEVELOPERS
----------
This project was supervised by Prof. Dr. Stefan Roettger (Georg-Simon
Ohm University of Applied Sciences, Nuremberg) as part of the major
Media-Engineering.
Project developers (v0.1 as of 10/2016 to 07/2017):
* Helena Bayerl
* Fabian Geyer
* Esther Kleinhenz
* Yvonne Laugisch
* Alex Neugebauer
* Chi Lan Nguyen
* Melanie Schoesser
* Stefanie Steding
* Emin Tavli
* Stefan Roettger
Project developers (v0.2-v0.9.10 as of 08/2017 to 04/2018):
* Stefan Roettger
Project developers (v0.9.11-v1.0 as of 05/2018 to 07/2018):
* Stefan Roettger
* Lisa Gruner
* Julia Bott
* Ksenia Huber
* Lena Huebner
* Johanna Hartmann
Project developers (v1.0-v1.7 as of 08/2018 to 10/2019):
* Stefan Roettger
Project developers (v1.7-v1.8 as of 11/2019 to 03/2020):
* Stefan Roettger
CREDITS
-------
* steam whistle sounds from http://www.crosby-steam.com
* steam locomotive sounds from http://www.greatwestern.org.uk/sounds.htm
* misc sounds from
* http://www.grsites.com/archive/sounds
* http://www.orangefreesounds.com
* http://soundbible.com
* 3D models from
* http://www.blendswap.com
* http://www.yobi3d.com
* http://www.turbosquid.com
* http://www.models-resource.com
* transparent billboard textures from
* http://www.kisspng.com