Gstreamer

Input via GStreamer

Snowmix takes live video as input from video feeds primarily through shared memory interfacing with the GStreamer plugin module shmsink. The module shmsink is responsible for allocating a shared memory area, communicating the filename representing the shared memory area to Snowmix and feeding video into the shared memory. The video must be in the ARGB format or rather the BGRA format on little endian CPU systems such as the x86 CPU architecture. This means that each pixel is send to SNowmix in the order Blue, Green, Red and Alpha.

The image below shows 3 different types of input.

The GStreamer module needs a control channel to communicate with snowmix. This control channel is a named pipe (a fifo file) and both GStreamer and snowmix need to know the name of this pipe for this to work. This page will show and example of GStreamer feeding a video stream to snowmix.

GStreamer pipeline example: Input from File

Below is shown an example for inputting video to Snowmix using GStreamer version 1.0 tested with GStreamer version 1.0 to 1.3.1

    #!/bin/bash
    # These settings are the settings defined for video feed 1 in Snowmix
    width=704
    height=576
    # This is the framerate you defined for Snowmix
    framerate='24/1'
    CONTROL_PIPE=/tmp/feed1-control-pipe
    MIXERFORMAT='video/x-raw, format=(string)BGRA, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive'
    MIXERFORMAT=$MIXERFORMAT", framerate=(fraction)$framerate, width=(int)$width, height=(int)$height"
    SRC="filesrc location=../test/LES_TDS_launch.mp4 ! decodebin "
    SHMSIZE='shm-size='`echo "$width * $height * 4 * 13"|bc`
    SHMOPTION="wait-for-connection=0"
    SHMSINK1="shmsink socket-path=$CONTROL_PIPE $SHMSIZE $SHMOPTION"
    SCALENRATE='videoconvert ! videorate ! videoscale ! videoconvert'
    while true ; do
        # Remove the named pipe if it exist
        rm -f $CONTROL_PIPE
        gst-launch-1.0 -v    \
            $SRC            ! \
            $SCALENRATE     ! \                
            $MIXERFORMAT    ! \
            $SHMSINK1
        sleep 2
    done
    exit

Below is shown an example for inputting video to Snowmix using GStreamer version 0.10 tested with GStreamer version 0.10.36.

    #!/bin/bash
    # These settings are the settings defined for video feed 1 in Snowmix
    width=704
    height=576
    # This is the framerate you defined for Snowmix
    framerate='24/1'
    CONTROL_PIPE=/tmp/feed1-control-pipe
    MIXERFORMAT='video/x-raw-rgb, bpp=(int)32, depth=(int)32, endianness=(int)4321, red_mask=(int)65280, green_mask=(int)16711680, blue_mask=(int)-16777216, pixel-aspect-ratio=(fraction)1/1, interlaced=(boolean)false'
    MIXERFORMAT=$MIXERFORMAT", framerate=(fraction)$framerate, width=(int)$width, height=(int)$height"
    SRC="filesrc location=../test/LES_TDS_launch.mp4 ! decodebin2 "
    SHMSIZE='shm-size='`echo "$width * $height * 4 * 6"|bc`
    SHMOPTION="wait-for-connection=0"
    SHMSINK1="shmsink socket-path=$CONTROL_PIPE $SHMSIZE $SHMOPTION"
    SCALE='ffmpegcolorspace ! videorate ! videoscale ! ffmpegcolorspace'
    while true ; do
        # Remove the named pipe if it exist
        rm -f $CONTROL_PIPE
        gst-launch-0.10 -v    \
            $SRC            ! \
            $SCALENRATE     ! \                
            $MIXERFORMAT    ! \
            $SHMSINK1
        sleep 2
    done
    exit

In this example 'filesrc' reads from the file ../test/LES_TDS.mp4, decodes it, scale it to 704x576 and converts it to 32 bit RGBA (BGRA actually) before writing it to shared memory and signaling to Snowmix via the named pipe '/tmp/feed-control-pipe'. The example will stream the file over and over again with a 2 seconds pause until interrupted. The frame rate is set to 24fps. The videorate element is not strictly necessary as SNowmix will insert extra or drop frames automatically upon need.

Snowmix Configuration

For snowmix to receive the video feed from GStreamer configured as shown above, the following entries need to be added to snowmix either interactively or to the configuration file snowmix reads upon starting. The entries are

    feed add 1 Camera 1
    feed geometry 1 704 576
    feed live 1
    feed idle 1 100 ../frames/dead-704x576.rgba
    feed socket 1 /tmp/feed-control-pipe

These entries first add feed number 1 and name it 'Camera 1'. Then the geometry of the feed is defined. Then the feed is defined as live. Last but not least, snowmix is informed that the name of the named pipe used as control channel is /tmp/feed-control-pipe. The 'feed idle' command optionally tells snowmix to use and image (found in the RGBA dead-704x576.rgba as replacement for the live feed, should the feed be missing for 100 frames.

Other GStreamer examples:

GStreamer pipeline example: Input from a camera using V4L2 (Video for Linux)

The example below uses live video from web camera connected to the computer:

    #!/bin/bash
    CONTROL_PIPE=/tmp/feed-control-pipe
    MIXERFORMAT='video/x-raw-rgb, bpp=(int)32, depth=(int)32, endianness=(int)4321, red_mask=(int)65280, green_mask=(int)16711680, blue_mask=(int)-16777216, width=(int)704, height=(int)576, framerate=(fraction)24/1, pixel-aspect-ratio=(fraction)1/1, interlaced=(boolean)false'
    SRC='v4l2src device=/dev/video1 '
    SHMSIZE='shm-size=10000000'
    SHMOPTION="wait-for-connection=0"
    SHMSINK1="shmsink socket-path=$CONTROL_PIPE $SHMSIZE $SHMOPTION"
    SCALE='ffmpegcolorspace ! videoscale ! ffmpegcolorspace'
    while true ; do
        # Remove the named pipe if it exist
        rm -f $CONTROL_PIPE
        gst-launch-0.10 -v      \
            $SRC              ! \
            $SCALE            ! \
            $MIXERFORMAT      ! \
            $SHMSINK1
        sleep 2
    done
    exit

GStreamer pipeline example: Input from a MPEG-4 UDP stream

The example below uses a MPEG-4 stream sent to a UDP port on the computer from an external IP based web camera.

    #!/bin/bash
    CONTROL_PIPE=/tmp/feed-control-pipe
    MIXERFORMAT='video/x-raw-rgb, bpp=(int)32, depth=(int)32, endianness=(int)4321, red_mask=(int)65280, green_mask=(int)16711680, blue_mask=(int)-16777216, width=(int)704, height=(int)576, framerate=(fraction)24/1, pixel-aspect-ratio=(fraction)1/1, interlaced=(boolean)false'
    SRC=" udpsrc port=4444 caps="application/x-rtp, media=(string)video, payload=(int)96, clock-rate=(int)90000, encoding-name=(string)MP4V-ES, profile-level-id=(string)245, config=(string)000001B0F5000001B50900000100000001200088401928B02240A21F, a-framerate=(string)30.0, a-mpeg4-esid=(string)201, clock-base=(uint)1918588220, seqnum-base=(uint)40098, npt-start=(guint64)0, play-speed=(double)1, play-scale=(double)1" ! rtpmp4vdepay ! mpeg4videoparse ! ffdec_mpeg4"
    SHMSIZE='shm-size=10000000'
    SHMOPTION="wait-for-connection=0"
    SHMSINK1="shmsink socket-path=$CONTROL_PIPE $SHMSIZE $SHMOPTION"
    SCALE='ffmpegcolorspace ! videoscale ! ffmpegcolorspace'
    while true ; do
        # Remove the named pipe if it exist
        rm -f $CONTROL_PIPE
        gst-launch-0.10 -v       \
            $SRC               ! \
            $SCALE             ! \
            $MIXERFORMAT       ! \
            $SHMSINK1
        sleep 2
    done
    exit