I have used the new version's example VMR renderless but it doesn't work for my application. I'm making presentation software and i need to render different blocks. I first render an image and this part is ok. But when the video is rendered all items before and afther it are gone. What is my problem. I removed the scene.begin and end and moved the device.present. Please help me.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Some weird things happen to me too. I was able to solve some of the problems, but I still have some troubles. Maybe something of what I did may help you: I use DirectShow .Net to capture video stream from a camera, and then render it to a texture, with VMR9. I also draw some 3D meshes in the scene.
The fist problem I had was some 'flicking' in the VRM9 generated texture. In the example the scene is rendered when the PresentHelper callback is called, but in my application I modified the code so it only updates the texture. So, the flickering seemed to be caused because DirectShow runs in separate threads, and it was updating the texture while it was being rendered. So I add some Mutex here and there. So:
Lock Mutex
Draw everything
Unlock Mutex
Let the VMR9 callbacks update.
But now I have some other problems: First: The textures on the mesh get rendered completely wrong. The mesh is textured with garbage EVEN if I deactivate texturing before drawing them. (look to the link below)
Second, VMR9 seems to use the main buffer to capture the image ?, as can be seen in
I render a mesh in the upper left corner, a bitmap in the lower left, and the captured image in the upper left corner. Notice the 'shadow' of the rendered mesh in the streamed image! :? If I do a Clear() before the VMR9 callbacks, insthead in the beginning of the drawing, this shadow disappears.
It is like zbuffer is affecting the generated textures. When you initialize the Allocator, you have to give it your Direct3D device, or you have to create another one?
Finally, I am having some random crashes, witch Visual Studio 2003 dosen't let me to debug. Sometimes, VMR9 stops to call the callbacks, and I don't receive new frames anymore. But I think it is because some wrong event handling in my code.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Your URL is invalid for me so i can't really help you.
What kind of video is provided to the VMR (did you have to deal with an YUV private texture) ?
You also have to know that i take some shortcuts in the VMRAllocator sample. In this sample, unmanaged texture must never change since managed object are only created once and stored in an Hashtable with the unmanaged pointer as a key. So you must be sure that your modifications havn't modified this requierement.
The sample also play with the D3D device's RenderTarget. Does this ok for your application ?
Finally, if you just want to retrieve the video as texture you should also considere using a custom Compositor instead of the of a custom Allocator. It's easy to write a compositor with a VMR with just 1 pin...
The Compositor is a VMR's componant positionned after the Mixer and before the Allocator / Presenter. It have the responsability to mix video streams from all pins into an unique texture (then used by the A/P).
I have write a small and dirty sample to test this fonctionality but i havn't time to make it more sexy to be in the v1.1... I can send it to you if you want...
Eric
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Can you send me please this sample too? (marco at alphasoft dot ch)
I have a question: How can I render multiple Video-Streams to multiple 3D Objects? Do I have to do this with a Compositor or an Allocator/Presenter? Is there an example anywhere?
Thank you
Marco
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
You have multiple video sources. So you have 2 choices :
1) Add a VMR9 for each sources and using custom A/P for each one (seem complicated).
2) Use only one VMR9, connect all your source to it (VMR9 support up to 16 input pins) and add a custom Compositor.
During the compositing, it's possible to access to each video streams as textures. Like previous posts stat it, you will probably have to deal with Mutex and Multithreads problems...
Eric
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Choice 2 sounds good. In a previous post of this thread you wrote, that you made a small and dirty sample for a compositor. Can you send it me please? At the moment I have no idea how to write a custom Compositor.
Thank you
Marco
Sorry for the wrong posting below...
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I have send it this morning... Do you have receive it ?
Regarding how implementing a compositor, the best source of information IS the DirectShow documentation and the samples from the DirectShow SDK (now in Platform SDK)...
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
In your sample I tryed to render the second movie (or pin or source) with a second texture. It works with some movies, but with the most movies it does not work. When it does not work, the output of the last few lines is
SetStreamMediaType : 0 : False
InitCompositionDevice : 1895200
SetStreamMediaType : 0 : False
SetStreamMediaType : 1 : False
SetStreamMediaType : 1 : False
SetStreamMediaType : 1 : False
SetStreamMediaType : 1 : False
SetStreamMediaType : 1 : False
SetStreamMediaType : 1 : False
SetStreamMediaType : 1 : False
SetStreamMediaType : 1 : False
SetStreamMediaType : 1 : False
SetStreamMediaType : 1 : False
SetStreamMediaType : 0 : False
SetStreamMediaType : 1 : False
'VMRCompositor.exe': Loaded 'c:\windows\assembly\gac\mscorlib.resources\1.0.5000.0_de_b77a5c561934e089\mscorlib.resources.dll', No symbols loaded.
When it does work, there are much more outputs of SetStreamMediaType.
Do you have a tip? Thank you.
Marco
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
In the compositor I added a second texture, a vertex buffer, a surface and vertices. In CompositeImage I get the second movie with pVideoStreamInfo[1].pddsVideoSurface which I mapped to the second texture.
I don't know if the technique with a custom compositor is the right one. Can I start/stop each pin of the vmr?
Thank you
Marco
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I have start changing my sample to display up to 4 pins in a quarter of the screen (not finish yet) and found that the StretchRectangle method is very restrictive. For exemple, it seem to be a problem if sources have not the same size. In this case, an exception is thrown and nothing at all is drawn (CompositeImage use a try/catch statement)...
I found that my implementation is really not optimal. I will probably have to redo it entirely...
Finaly, just a D3D optimisation, changing the vertex buffer of a device is costly. You should have just 1 VB with all your vertices and use DrawPrimitives to specify which one need to be drawn...
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
For your pins problem, i think the best way to do this is to implement it yourself in the compositor. If you don't want a pin being active, you just have to not draw its texture...
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
This is not what I want. I think I have to explain my problem: I want to play a movie and render it to a 3D object. During playing this one, I click on a button. Then a second movie should start play and rendered to another 3D object. For creating transition effects, like DES. But DES is not the right way for this, because I want to add some text over the movies. And it should also be possible to render live streams.
Is the composer the right way for this?
Thank you for tips.
Marco
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
If i have understand correctly, you want to be able to independently start and stop video sources connected to the VMR9. There is only two possibilities to do that :
1) Use 2 different graphs, each with a VMR. In this case the A/P scenario is the best. As a side note, this is exactly what the Video object (from Microsoft.DirectX.AudioVideoPlayback) does.
2) Using "Dynamic Graph Building" (see docs). I never use this tricky fonctionality but it could possibly work...
To make the right choice between writing a custom Compostor or a custom A/P you have to understand how the VMR works. You MUST read the "Using the Video Mixing Renderer" section in the documentation !
Eric
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I will read the docu... anyway a question: I think as first I have to make an allocator sample which renders from the application main loop like the sdk tutorials and not from the allocator:
while(frm.Created)
{
frm.Render();
Application.DoEvents();
}
Do you know what I mean? Your allocator calls the DrawScene in the PresentHelper function. I think I have to call a Render function (eg DrawScene) from the main app loop. I don't know how to "share" the texture.
Thank you
Marco
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
A first very important thing about Managed D3D loop : NEVER USE Application.DoEvents(); in a long loop.
Application.DoEvents(); make a lot's of allocations and in a busy application (3d app are generaly in this category) Garbage Collector don't have time to do its job.
You should instead use this loop (by the maker of Managed DirectX) : http://blogs.msdn.com/tmiller/archive/2005/05/05/415008.aspx
Now, how the VMR Allocator sample must be modified to use the video texture in an external loop ?
1) All PlaneScene stuff can be remove.
2) The IVMRSurfaceAllocator9 part can remain mostly unchanged.
3) You must neutralize the IVMRImagePresenter9 part. This part must draw nothing. Instead, the PresentImage must take the texture and make it accessible to the rest of your code throw a public property or a method returning it for exemple.
4) This texture must not be modified by PresentImage during your render loop. There are variety of ways to do that, from mutex to a small boolean value indicating that you are in your rendering loop and that this frame must be skip.
Eric
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I'm sorry but i don't understand what you mean. You must set your boolean variable to true when begining your render loop (before device.Begin()) and reset it after (after device.Present())
You should start a new thread because this one (with its 22 answers) is now unreadable...
Eric
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Sorry about the broken link, It seems the home user's server has been down during the weekend. The link works now.
The video provided to VMR9 come from the "Microsoft DV Video and VCR" component, and connected 'manually' (dealing with pins and connections) with VMR9 component. After connecting them, I do a 'run' from the media control. The rest of the allocator code is practically the same of the DirectShowNet VRM9 example, with modifications in the presentHelper function to assign the result texture to another object. The texture is displayed correctly, with some blinking if I don't use mutex. I think i don't have to deal with any private YUV texture.
About the render target, It doesn't seem a problem, as it is restored at the beginning of the present helper function. But maybe there might be some problems, because I can't know if VMR9 filter switch the render target meanwhile I am doing the drawing, in the main loop thread. My application has the typical 'game loop', wich maybe not adecuate with VRM9 'random' callbacks in separate threads.
It may be really useful if you can send me the 'non-sexy' example :-) I have written a private message to you with my email.
Another little question: Does VMR9 change some DX3D Device render state flags?
Thanks,
Alex.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Writing Direct3D applications is not an easy task. Before add pieces of the VMRAllocator in your application, be sure to understand it and point where it's code can affect your code. The VMRAllocator change a lots of Direct3D device's states like the View and the Projection Matrix and the Vertex format...
Where is your rendering loop (in the VMR's PresentImage or in your own loop) ?
Eric
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Choice 2 sounds good. In a previous post of this thread you wrote, that you made a small and dirty sample for a compositor. Can you send it me please? At the moment I have no idea how to write a custom Compositor.
hHank youi
Marco
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I have used the new version's example VMR renderless but it doesn't work for my application. I'm making presentation software and i need to render different blocks. I first render an image and this part is ok. But when the video is rendered all items before and afther it are gone. What is my problem. I removed the scene.begin and end and moved the device.present. Please help me.
Hi,
Some weird things happen to me too. I was able to solve some of the problems, but I still have some troubles. Maybe something of what I did may help you: I use DirectShow .Net to capture video stream from a camera, and then render it to a texture, with VMR9. I also draw some 3D meshes in the scene.
The fist problem I had was some 'flicking' in the VRM9 generated texture. In the example the scene is rendered when the PresentHelper callback is called, but in my application I modified the code so it only updates the texture. So, the flickering seemed to be caused because DirectShow runs in separate threads, and it was updating the texture while it was being rendered. So I add some Mutex here and there. So:
Lock Mutex
Draw everything
Unlock Mutex
Let the VMR9 callbacks update.
But now I have some other problems: First: The textures on the mesh get rendered completely wrong. The mesh is textured with garbage EVEN if I deactivate texturing before drawing them. (look to the link below)
Second, VMR9 seems to use the main buffer to capture the image ?, as can be seen in
http://www.salleurl.edu/~is06764/bug.jpg
I render a mesh in the upper left corner, a bitmap in the lower left, and the captured image in the upper left corner. Notice the 'shadow' of the rendered mesh in the streamed image! :? If I do a Clear() before the VMR9 callbacks, insthead in the beginning of the drawing, this shadow disappears.
It is like zbuffer is affecting the generated textures. When you initialize the Allocator, you have to give it your Direct3D device, or you have to create another one?
Finally, I am having some random crashes, witch Visual Studio 2003 dosen't let me to debug. Sometimes, VMR9 stops to call the callbacks, and I don't receive new frames anymore. But I think it is because some wrong event handling in my code.
Hi,
Your URL is invalid for me so i can't really help you.
What kind of video is provided to the VMR (did you have to deal with an YUV private texture) ?
You also have to know that i take some shortcuts in the VMRAllocator sample. In this sample, unmanaged texture must never change since managed object are only created once and stored in an Hashtable with the unmanaged pointer as a key. So you must be sure that your modifications havn't modified this requierement.
The sample also play with the D3D device's RenderTarget. Does this ok for your application ?
Finally, if you just want to retrieve the video as texture you should also considere using a custom Compositor instead of the of a custom Allocator. It's easy to write a compositor with a VMR with just 1 pin...
The Compositor is a VMR's componant positionned after the Mixer and before the Allocator / Presenter. It have the responsability to mix video streams from all pins into an unique texture (then used by the A/P).
I have write a small and dirty sample to test this fonctionality but i havn't time to make it more sexy to be in the v1.1... I can send it to you if you want...
Eric
Hello Eric
Can you send me please this sample too? (marco at alphasoft dot ch)
I have a question: How can I render multiple Video-Streams to multiple 3D Objects? Do I have to do this with a Compositor or an Allocator/Presenter? Is there an example anywhere?
Thank you
Marco
Hi,
You have multiple video sources. So you have 2 choices :
1) Add a VMR9 for each sources and using custom A/P for each one (seem complicated).
2) Use only one VMR9, connect all your source to it (VMR9 support up to 16 input pins) and add a custom Compositor.
During the compositing, it's possible to access to each video streams as textures. Like previous posts stat it, you will probably have to deal with Mutex and Multithreads problems...
Eric
Choice 2 sounds good. In a previous post of this thread you wrote, that you made a small and dirty sample for a compositor. Can you send it me please? At the moment I have no idea how to write a custom Compositor.
Thank you
Marco
Sorry for the wrong posting below...
I have send it this morning... Do you have receive it ?
Regarding how implementing a compositor, the best source of information IS the DirectShow documentation and the samples from the DirectShow SDK (now in Platform SDK)...
Yes, thank you. I will try it at home.
In your sample I tryed to render the second movie (or pin or source) with a second texture. It works with some movies, but with the most movies it does not work. When it does not work, the output of the last few lines is
SetStreamMediaType : 0 : False
InitCompositionDevice : 1895200
SetStreamMediaType : 0 : False
SetStreamMediaType : 1 : False
SetStreamMediaType : 1 : False
SetStreamMediaType : 1 : False
SetStreamMediaType : 1 : False
SetStreamMediaType : 1 : False
SetStreamMediaType : 1 : False
SetStreamMediaType : 1 : False
SetStreamMediaType : 1 : False
SetStreamMediaType : 1 : False
SetStreamMediaType : 1 : False
SetStreamMediaType : 0 : False
SetStreamMediaType : 1 : False
'VMRCompositor.exe': Loaded 'c:\windows\assembly\gac\mscorlib.resources\1.0.5000.0_de_b77a5c561934e089\mscorlib.resources.dll', No symbols loaded.
When it does work, there are much more outputs of SetStreamMediaType.
Do you have a tip? Thank you.
Marco
What have you change in my sample because my custom compositor never mix anything other than the first pin ?
In the compositor I added a second texture, a vertex buffer, a surface and vertices. In CompositeImage I get the second movie with pVideoStreamInfo[1].pddsVideoSurface which I mapped to the second texture.
I don't know if the technique with a custom compositor is the right one. Can I start/stop each pin of the vmr?
Thank you
Marco
I have start changing my sample to display up to 4 pins in a quarter of the screen (not finish yet) and found that the StretchRectangle method is very restrictive. For exemple, it seem to be a problem if sources have not the same size. In this case, an exception is thrown and nothing at all is drawn (CompositeImage use a try/catch statement)...
I found that my implementation is really not optimal. I will probably have to redo it entirely...
Finaly, just a D3D optimisation, changing the vertex buffer of a device is costly. You should have just 1 VB with all your vertices and use DrawPrimitives to specify which one need to be drawn...
This sample would be nice...
Thank you for the tip with 1 vertex buffer.
Is it possible to play/stop each pin or can I only play/stop the entire vmr?
I havn't time to work on this sample today...
For your pins problem, i think the best way to do this is to implement it yourself in the compositor. If you don't want a pin being active, you just have to not draw its texture...
This is not what I want. I think I have to explain my problem: I want to play a movie and render it to a 3D object. During playing this one, I click on a button. Then a second movie should start play and rendered to another 3D object. For creating transition effects, like DES. But DES is not the right way for this, because I want to add some text over the movies. And it should also be possible to render live streams.
Is the composer the right way for this?
Thank you for tips.
Marco
If i have understand correctly, you want to be able to independently start and stop video sources connected to the VMR9. There is only two possibilities to do that :
1) Use 2 different graphs, each with a VMR. In this case the A/P scenario is the best. As a side note, this is exactly what the Video object (from Microsoft.DirectX.AudioVideoPlayback) does.
2) Using "Dynamic Graph Building" (see docs). I never use this tricky fonctionality but it could possibly work...
To make the right choice between writing a custom Compostor or a custom A/P you have to understand how the VMR works. You MUST read the "Using the Video Mixing Renderer" section in the documentation !
Eric
I will read the docu... anyway a question: I think as first I have to make an allocator sample which renders from the application main loop like the sdk tutorials and not from the allocator:
while(frm.Created)
{
frm.Render();
Application.DoEvents();
}
Do you know what I mean? Your allocator calls the DrawScene in the PresentHelper function. I think I have to call a Render function (eg DrawScene) from the main app loop. I don't know how to "share" the texture.
Thank you
Marco
A first very important thing about Managed D3D loop : NEVER USE Application.DoEvents(); in a long loop.
Application.DoEvents(); make a lot's of allocations and in a busy application (3d app are generaly in this category) Garbage Collector don't have time to do its job.
You should instead use this loop (by the maker of Managed DirectX) : http://blogs.msdn.com/tmiller/archive/2005/05/05/415008.aspx
Now, how the VMR Allocator sample must be modified to use the video texture in an external loop ?
1) All PlaneScene stuff can be remove.
2) The IVMRSurfaceAllocator9 part can remain mostly unchanged.
3) You must neutralize the IVMRImagePresenter9 part. This part must draw nothing. Instead, the PresentImage must take the texture and make it accessible to the rest of your code throw a public property or a method returning it for exemple.
4) This texture must not be modified by PresentImage during your render loop. There are variety of ways to do that, from mutex to a small boolean value indicating that you are in your rendering loop and that this frame must be skip.
Eric
Thank you for the tip with the Loop.
I solved the 4) with a boolean. Then every frame will be skipped. Is the priority of the main loop too high?
Thank you
Marco
I'm sorry but i don't understand what you mean. You must set your boolean variable to true when begining your render loop (before device.Begin()) and reset it after (after device.Present())
You should start a new thread because this one (with its 22 answers) is now unreadable...
Eric
Hi,
Sorry about the broken link, It seems the home user's server has been down during the weekend. The link works now.
The video provided to VMR9 come from the "Microsoft DV Video and VCR" component, and connected 'manually' (dealing with pins and connections) with VMR9 component. After connecting them, I do a 'run' from the media control. The rest of the allocator code is practically the same of the DirectShowNet VRM9 example, with modifications in the presentHelper function to assign the result texture to another object. The texture is displayed correctly, with some blinking if I don't use mutex. I think i don't have to deal with any private YUV texture.
About the render target, It doesn't seem a problem, as it is restored at the beginning of the present helper function. But maybe there might be some problems, because I can't know if VMR9 filter switch the render target meanwhile I am doing the drawing, in the main loop thread. My application has the typical 'game loop', wich maybe not adecuate with VRM9 'random' callbacks in separate threads.
It may be really useful if you can send me the 'non-sexy' example :-) I have written a private message to you with my email.
Another little question: Does VMR9 change some DX3D Device render state flags?
Thanks,
Alex.
Hi,
Writing Direct3D applications is not an easy task. Before add pieces of the VMRAllocator in your application, be sure to understand it and point where it's code can affect your code. The VMRAllocator change a lots of Direct3D device's states like the View and the Projection Matrix and the Vertex format...
Where is your rendering loop (in the VMR's PresentImage or in your own loop) ?
Eric
Heay would be great if you could send some samples.
Choice 2 sounds good. In a previous post of this thread you wrote, that you made a small and dirty sample for a compositor. Can you send it me please? At the moment I have no idea how to write a custom Compositor.
hHank youi
Marco