Opengl support for vdr-sxfe

Is the following note about VDPAU still valid:

> - OpenGL acceleration for video and OSD display. If the option --opengl-always is used in
> addition to --hud, opengl is used to draw the video and the OSD. The advantage is a video size
> independent, full transparent, and fast OSD. No window manager is required. However, this
> method seems only to work with the VDPAU video out driver, not XV.

Was it because of failed XGetWindowAttributes ?
It seems xine-lib xshm/xv/opengl video drivers initialize unconditionally x11osd, and that calls XGetWindowAttributes() for the drawable. It does not work as it is not window ... ?

Could the error / XGetWindowAttributes call be trapped to make this work ?

> And I was also not able to get a playback without occasional frame drops. However, I found a compromise by
> adding the second option below

Solution to both could be using raw video driver ; that provides raw frame data that can be copied to OpenGL texture. But it can't use VDPAU decoder.

Another solution could be to hook to video driver interface and request frame data there. That should be very simple too ; there are already other similar hooks. That should work with VDPAU too.

If this patch does not work without VDPAU, someone needs to test modified version before it can be committed. I've done some modifications (I had ex. some problems with dlsym() with recent mesa versions).

But I don't see much point using this feature only with VDPAU driver, as it already implements similar video-independent OSD functionality. Instead vdr-sxfe VDPAU OSD support should be fixed. It is not enabled currently because I can't test it. There's also OpenGL video driver in xine-lib ; maybe that could be extended to support OSD extent (video-resolution independent OSD).

Honestly speaking, I don't remember what the exact problem with XV was. Sorry...

I can test your changes with my vdpau installation. Is the patch in the repository? Do I need to enable something?

For getting XV to work, I would suggest to ask the xinelib community how to get raw video frames from the library for OpenGL use, without much overhead. I don't think that the OSD support in xinelib is a better choice because I tried it before. I even looked in the VDPAU driver source code in xinelib how the OSD was implemented. It was using the native VDPAU OSD support to implement the xinelib OSD feature. The implementation looked ok but it was too slow. There was a lag about half a second before the cursor followed the remote click. Therefore, I decided to try the OpenGL way that turned out much faster.

> I can test your changes with my vdpau installation. Is the patch in the
> repository? Do I need to enable something?

It is now, just use --opengl or --hud=opengl command-line options.

I've commented out some code that looked unused (frame duration calculation - the result was not used anywhere).
Also I disabled xshape + opengl combination (opengl disables xshape at runtime).

> For getting XV to work, I would suggest to ask the xinelib community how
> to get raw video frames from the library for OpenGL use, without much
> overhead.

It is already possible with raw video driver, but not with xv driver (with hardware overlay it might be impossible even in theory).
I could add the code that provides frame data in RGB format. We can then add OpenGL YUV->RGB conversion later. Note that frames are unscaled ; we'll need also some code to calculate scaling factors based on frame size/aspect ratio and window size/display aspect ratio.

> I don't think that the OSD support in xinelib is a better choice
> because I tried it before. I even looked in the VDPAU driver source code in
> xinelib how the OSD was implemented. It was using the native VDPAU OSD
> support to implement the xinelib OSD feature. The implementation looked ok
> but it was too slow. There was a lag about half a second before the cursor
> followed the remote click. Therefore, I decided to try the OpenGL way that
> turned out much faster.

Ok, that explains it.

BTW, it seems to leave (vdpau) video window under the opengl window. Could it reuse the existing window or should it be unmapped ? (at least when using opengl always, when it never draws to the window).

I tested the latest cvs version. I really like the way you integrated the opengl support into the command line options. Thank you very much for taking care!

I found some minor problems during testing (patch will be uploaded):

1. The hud field in the sxfe_s struct was just one bit wide. But you are using now flags to indicate what kind of hud is requested. I changed the field to a byte to handle this. Otherwise, the new --opengl option was not working
2. The video window now uses the extension CWBackPixel. I don't know the reason why this is required, but it causes one black frame when the HUD is unmapped if --hud=opengl is used. I commented out this extension in my patch. Why is this extension required?
3. The opengl initialization was not done anymore in the drawing thread. Opengl must be initialized from the same thread that does the drawing. I moved the code accordingly.
4. I did same changes to the opengl drawing thread to support the --opengl option (initial redirecting of xine frames into pixmap and mapping of opengl window)

Regarding the unmapping of the video / vdpau window when --opengl is used: I think this should work. But I would only unmap the window if the --opengl option is used.

Regarding the raw video driver: I don't think this is a good solution because no hardware acceleration will be used for the decoding of the video. Do you already have experience with it? Better would be if the xv implemenation in xine would allow frame redirection into pixmaps. I will write the xine developers an email. Can you send me your email in a private message such that I can keep you updated?

I created a bug report for xine-lib regarding the XV issue. See https://bugs.xine-project.org/show_bug.cgi?id=409.

> Regarding the raw video driver: I don't think this is a good solution
> because no hardware acceleration will be used for the decoding of the
> video.

Yes, color space conversion and scaling should be implemented using OpenGL.
There's some code for that in xine-lib opengl video driver.

> Do you already have experience with it?

I tested raw video driver and it seems to work. I can post the code if someone is interested to develop it further.

> Better would be if the xv
> implemenation in xine would allow frame redirection into pixmaps.

Unfortunately XvPutImage() accepts only window - it can't output to pixmap.

But I might have a solution to this. When using texture-based xvideo implementation it is possible to redirect the Xv target window to pixmap.

LOGDBG("redirecting video to opengl frame texture");
+#if 0
xine_port_send_gui_data(this->x.video_port, XINE_GUI_SEND_DRAWABLE_CHANGED,
(void*) this->video_frame_pixmap);
+#else
+ XCompositeRedirectWindow(this->display, this->window[1], CompositeRedirectAutomatic);
+ this->video_frame_pixmap = XCompositeNameWindowPixmap(this->display, this->window[1]);
+#endif

Of course this should be implemented before binding pixmap to opengl texture.

> Of course this should be implemented before binding pixmap to opengl texture.

And off-screen window instead of current output window :)