Re: [gst-devel] Video 3D support

[David S. <ds...@en...>]

On Tue, May 18, 2010 at 07:12:26AM +0000, Martin Bisson wrote:
> My work is based on the proposal made for Google Summer of Code :
> http://gstreamer.freedesktop.org/wiki/Video3DSupport , and I would like to
> discuss the issue discussed in
> https://bugzilla.gnome.org/show_bug.cgi?id=611157.  I'm actually trying to
> get input on what needs to be added to GStreamer in order to provide proper
> 3D video support, by adding info to the 3D stream.  The different options
> could be any combination of:
> 
> 1) doing nothing : the resulting stream would just be treated as a "normal"
> video stream
> 2) adding caps : the caps could have information about the 3D video
> information (left-right, top-bottom, red-green, etc.)
> 3) adding buffer flags : the info would be in the buffer flags, like audio
> streams (number of "channels" (left/right video streams seen as left/right
> sound channels), etc.)

The goal, imo, is to define how GStreamer handles stereo video
*natively*, that is, elements can automatically differentiate
between normal and stereo video, caps negotiation works as one
would expect, playback of a stereo video on a normal monitor
would allow for the option of viewing in mono or with red/green
glasses, etc.  A lower goal would be to create elements that
manipulate stereo video, but entirely manually.  I'm only
concerned with the former.

There are two main options: Use pairs of well-known raw image
layouts (i.e., I420, YUY2, UYVY, etc.), probably consecutive
in memory, for the left and right images.  Or, define a bunch
of new fourccs that mean "stereo video" that correspond to
existing layouts, but enhanced for stereo.

Using existing layouts: I recommend packing the two pictures
consecutively in memory, left then right.  The main rationale is
that conversion to a normal picture is simply changing the size
and/or pointer of the buffer and changing caps.  Other packing
arrangements might be important in the future, so having a
manditory caps field marking the packing would be a good idea.

I recommend using a new caps type, perhaps video/x-raw-yuv-stereo
or some such, instead of using video/x-raw-yuv,stereo=true.  Using
video/x-raw-yuv leads to endless compatibility problems: elements
that currently handle video/x-raw-yuv would silently do the wrong
thing with stereo video.  Using x-raw-yuv would mean width/height
in the caps would be double the *actual* width/height of the mono
video, which is hacky.  Also, converting from stereo to mono in
many cases would require copying.

Defining new fourccs:  This has the obvious disadvantage that
we'd either need to keep these internal to GStreamer, or make them
well-known enough for other people to use them.  Integrating with
existing elements (and libgstvideo) is straightfoward.  Adding
new layouts (side-by-side, top-bottom, memory consecutive, etc.)
is simple, although adds *lots* more fourccs.

I think that overall, using new fourccs would involve writing
less code and be less prone to bugs.  It is my preference.

Oh yeah, other methods such as dual pads for left/right, or buffer
flags, are non-starters: our attempts at dual pads for float audio
failed miserably, and we don't have any buffer flags available.
And there are other reasons which I don't feel like enumerating
right now.



dave...