[GD-Windows] streaming video into textures

[Andras B. <and...@gm...>]

I'm not sure if this is the right list to ask this question (it spans
across Windows, DirectShow, OpenGL and a bit of D3D even), so if you
think you know of a better place to ask, then please don't hesitate to
redirect me. :)

I'm trying to stream video files and real-time captured video frames
(eg. from webcam, tv tuner, etc..) into OpenGL textures.

First, I've looked into Video For Windows. The API is simple, the docs
is good, implementation is a snap. So now I can play AVI files
anywhere there is a texture. It looks great! :)

Video for Windows is far from perfect though. It only supports AVI
files (no MPEG, WMV or other fancy formats), it decompresses the
frames into system memory (which is somewhat inefficient), and it
doesn't support capturing frames from external sources at all.

So I turned my attention towards DirectShow. This solution is waay
more complex, and after intensive Googling, browsing the MSDN, and the
SDK samples for two days, there are still some unanswered questions.
So here I am. :)

I understand the part where I have to create a filter graph, which is
very flexible, and enables me to play back basically anything that I
have a DirectShow codec or WDM driver for.

The part where I get confused is how do I get my hands on the frame
image data itself? There seems to be a couple ways to do it. One of
them is using the ISampleGrabber filter. One thing I don't understand
about this method is I couldn't put this filter after the decoder in
graphEdit. It would always put an extra AVI decompressor plus a color
space converter afterwards. Why is that? I feel that the documentation
is lacking, I don't have a clear undersanding about what this filter
actually does. Also, the doc says that its header file is incompatible
with newer directx headers, and that made me feel a bit uneasy. Is
this some deprecated API?

Then I've found the VMR-9. Everyone is raving about it, HW accelerated
deinterlacing, color space conversion, muxing multiple sources,
renders straight into VRAM, etc... Seems like exactly what I need!
Unfortunately, this is the most complicated solution of all. I have
found samples that demonstrate how to decode video into DirectX
surfaces, but I don't know too muh about DX, I'd like to avoid that as
much as possible.. I've read that I have make my custom implementation
of VMR's allocator interface, but it seems like that this interface
only takes a pointer to a DX surface. What I'd love to be able to do
instead, is to just pass an arbitrary pointer (that would probably
point straight into a PBO, from where texture updates should be
lightning fast).

Question #1: Do I really need to create a DirectX surface, and render
the video frames into that? Is this the only way? If yes, how can I
copy this data over to OpenGL efficiently?

Question #2: I read somewhere that VMR-9 does not work with captured
video?? Can anyone confirm/deny this? If that's the case it would be
pretty much useless for me..

Question #3: What other options are there that I've missed?

As a sidenote, I did see the nVidia samples, but the OpenGL version is
a mess and the precompiled version they provided in the SDK does not
even work, so I didn't look too hard there. If anyone knows a better
VMR-9/GL sample that would be great! But all I'm really looking for is
some pointers in the right direction. I just don't want to spend days
on figuring out something, when that something might not even work for
me at all.


Thanks,


Andras