Kinect_Server_OpenNI

Summary

Program to transfer camera data to virtools directly from the kinect camera or from recorded files (use [PCL_Recorder]). Includes skeleton tracking and the AR Toolkit for camera tracking. Data can be returned in a number of ways.

Parameters

• "--help" - shows help message
• "--world_centric" or "-w" - return ARTK tracking data relative to the marker (instead of relative to the camera).
• "--mirror" or "-m" - broadcast image and all positions mirrored along the vertical image axis
• "--config_file STRING" or "-x STRING" - path to the OpenNI configuration file (.xml format)
• "--calibration_file STRING" or "-c STRING" - path to the camera calibration file (.dat format). Not mandatory, leaving this flag enables broadcast without ARTK patterns.
• "--pattern_file STRING" or "-p STRING" - path to one or more ARTK pattern files (.cfg files). Not mandatory, leaving this flag enables broadcast without ARTK patterns.
• "--use_recording STRING" or "-r STRING" - provide the path to a previously recorded image sequence (use [PCL_Recorder]). Overrides the kinect camera."
• "--recording_fps INT" or "-f INT" - if a recorded image sequence is used, set the framerate with which it was recorded for correct playback speed.
• "--pipe" - Use a Named Pipe instead of the TCP connection to transfer images on the same computer. Note that both the Kinect Server and the receiving thread need to run on the same permissions (e.g. both with Admin permissions) to avoid "access denied" errors on the client side.

Usage

For broadcasting data directly from the Kinect camera, using one ARTK pattern:

Kinect_Server_OpenNI.exe -x KinectAR.xml -c KinectCalib.dat -p marker_large_ID0.cfg

For broadcasting recorded data from a directory, using two ARTK patterns and world-centric tracking data (add list of patterns last!):

Kinect_Server_OpenNI.exe -x KinectAR.xml -r "F:\FullCap3" -f 20 -w -c KinectCalib.dat -p marker_large_ID0.cfg marker_config_2.cfg

Note that the images are received in Virtools using the Virtools_ImageStreamClient Manager. There is still an issue with image lag.

NB: Make sure that both the sender and the receiver are started with the same access rights when using the Named Pipe (i.e. both started with/without Administrator priviledges). Otherwise, the pipe will fail upon reading access!

If you broadcast pattern tracking data in camera-centric format (position of pattern relative to camera, without "-w" flag), use the following VRPN settings:

vrpnTracker_0 ARTK@localhost:3883
neutralPosition_0   0.0 0.0 0.0
neutralQuaternion_0 0.707106781186548 0.0 0.0 0.707106781186548
axisPermute_0 0 1 2
axisSign_0 1 -1 1
trackerScale_0 1

We assume the following conventions with regards to Virtools: The world extends in the x-z plane, gravity acts along the -y axis. Keep this in mind for the following VRPN settings.

If you broadcast pattern tracking data in pattern_centric format (position of camera relative to pattern, "-w" flag, use the following VRPN settings:

vrpnTracker_0 ARTK@localhost:3883
neutralPosition_0   0.0 0.0 0.0
neutralQuaternion_0 0.707106781186548 0.0 0.0 -0.707106781186548
axisPermute_0 0 2 1
axisSign_0 1 1 1
trackerScale_0 1

Skeleton data is currently provided only relative to the camera. Use the following VRPN settings:

vrpnTracker_1 skeleton_pose@localhost:3883
neutralPosition_1   0.0 0.0 0.0
neutralQuaternion_1 0.707106781186548 0.0 0.0 0.707106781186548
axisPermute_1 0 1 2
axisSign_1 1 1 1
trackerScale_1 1

A typical ARTK marker file (marker_config.cfg)

# multimarker definition file for ARToolKit (format defined by ARToolKit)
# dataset for test Painting application

# number of markers
1

# marker 0
0
120.0
0.0 0.0
1.0000  0.0000 0.0000 0.0
0.0000  1.0000 0.0000 0.0
0.0000  0.0000 1.0000 0.0

Typical Kinect camera calibration data (KinectCalib.dat)

ARToolKitPlus_CamCal_Rev02
640 480 320.0 240.0 523.7521 523.7964 0.2238 -0.3194 0.0049 -0.0069 0.0 0.0 0.0

Typical OpenNI configuration file (KinectAR.xml)

<OpenNI>
    <Licenses>
        <!-- Add licenses here 
        <License vendor="vendor" key="key"/>
        -->
    </Licenses>
    <Log writeToConsole="false" writeToFile="false">
        <!-- 0 - Verbose, 1 - Info, 2 - Warning, 3 - Error (default) -->
        <LogLevel value="3"/>
        <Masks>
            <Mask name="ALL" on="true"/>
        </Masks>
        <Dumps>
        </Dumps>
    </Log>
    <ProductionNodes>
        <!-- Normal Image -->
        <Node type="Image" name="Image1">
            <Configuration>
                <!-- <MapOutputMode xRes="640" yRes="480" FPS="30"/> -->
                <Mirror on="false"/>
            </Configuration>
        </Node>

        <!-- Depth Image -->        
        <Node type="Depth" name="Depth1">
            <Configuration>
                <!-- <MapOutputMode xRes="640" yRes="480" FPS="30"/> -->
                <Mirror on="false"/>
            </Configuration>
        </Node>

        <Node type="User"/>

        <!-- Normal IR -->
        <!--
        <Node type="IR" name="IR1">
            <Configuration>
                <MapOutputMode xRes="640" yRes="480" FPS="30"/>
                <Mirror on="false"/>
            </Configuration>
        </Node>
        -->

        <!--
        <Node type="Audio" name="Audio1">
        </Node>
        -->
    </ProductionNodes>
</OpenNI>

Output

Serves as a network image streaming server to the Virtools_ImageStreamClient manager. Acts as a VRPN server broadcasting skeleton and marker tracking data

Skeleton Data:
6 - (XN_SKEL_RIGHT_HIP)
12 -(XN_SKEL_LEFT_HIP)
2 - (XN_SKEL_NECK)
1 - (XN_SKEL_TORSO)
3 - (XN_SKEL_RIGHT_SHOULDER)
9 - (XN_SKEL_LEFT_SHOULDER)
0 - (XN_SKEL_HEAD)
4 - (XN_SKEL_RIGHT_ELBOW)
5 - (XN_SKEL_RIGHT_HAND)
10 - (XN_SKEL_LEFT_ELBOW)
11 - (XN_SKEL_LEFT_HAND)
7 - (XN_SKEL_RIGHT_KNEE)
8 - (XN_SKEL_RIGHT_FOOT)
13 - (XN_SKEL_LEFT_KNEE)
14 - (XN_SKEL_LEFT_FOOT)

Planned Changes

• Add a "Virtools" flag to provide all data in Virtools-compliant coordinate system
• Provide skeleton tracking data relative to a specified ARTK marker
• Fix bug with boost program options: Only the first marker specified is used.
• Migrate to the Concurrent Data and Concurrent Queue classes for thread-shared memory

Requirements

Compiling:

• boost 1.46.1
• OpenNI 1.3.3
• NITE 1.3
• OpenCV 2.3
• ARTK+
• VRPN
• Google Snappy compression library

Running:

• OpenNI 1.3.3
• NITE 1.3
• OpenCV 2.3
• ARTK+
• VRPN