Artifacts when rendering H.264 encoded RTSP stream with VPP's RTSP source filter

  •  onkel_keks

    onkel_keks - 2013-04-25

    first of all, thank you for your work, it is of great help. I have a project where I want to receive video from a webcam that transmits a H.264 encoded stream over RTSP and write the decoded pixel data into a buffer for further processing. (Audio support is supposed to follow, but that's a problem for later). Disclaimer: I am rather new to DirectShow.

    So far, I've managed to use your RTSP filter for receiving the H.264 stream and can play it in GraphEdit (I had to force UDP instead of TCP but I am getting a picture). The problem now is that the picture is flawed; there are many artifacts and large, depending on the decoder used either green or black areas. When I use VLC to view the very same stream, the picture is flawless. I know that it's not the fault of the decoder, since I have tried with both the DivX H.264 and the built-in decoder from Microsoft.
    Any hints as to what could be wrong here?

    P.S.: I'm not sure if it's a bug or I did something wrong, but in RtspSourceFilter::stopLiveMediaSession() you have this line:

    DWORD result = WaitForSingleObject(m_hLiveMediaStopEvent, INFINITE);

    which causes GraphEdit to hang indefinitely when closing the session, or when no session was ever created.

    P.P.S.: Sorry, this should have been in the "Help" section.

    Last edit: onkel_keks 2013-04-25
  • Ralf Globisch

    Ralf Globisch - 2013-04-25


    Firstly, let me just say that the filter was just intended as a sample showing how to combine the live555 with DirectShow. I primarily tested it with audio and added support for the H.264 video format afterwards. The H.264 was only briefly tested using live555 RTSP server and .264 test videos.

    When I use VLC to view the very same stream, the picture is flawless. I know that it's not the fault of the decoder, since I have tried with both the DivX H.264 and the built-in decoder from Microsoft.
    Any hints as to what could be wrong here?

    The usual suspects are either packet loss, or that the media isn't being processed fast enough. If you sniff the traffic you could check if the live555 module is reporting any/many packets being lost. If that is the case, you could try to increase the UDP receiver buffer size of the application, though IIRC the live555 does increase it.

    Out of interest, what happens when you stream via RTP over RTSP/TCP (i.e. no packet loss unless there is insufficient network bandwidth for the stream)?

    If you have ruled out loss as the culprit, perhaps there could be inefficiency in the RTSP source filter wrt how media is passed downstream to the decoder. I ran into the following issue when trying something similar: streaming H.264 via RTSP and passing it to the Win7 decoder resulted in strange artifacts. However passing the same media using our H.264 source filter resulted in flawless playback. (FYI:

    Other than that, please post back your findings so that we can improve the sample. Thanks!

    Thanks for the bug report, I will take a look at it when I get a chance. The event is supposed to be set when the live555 thread finishes.

  •  onkel_keks

    onkel_keks - 2013-04-26

    Thank you very much for your quick and helpful reply,
    I will try what you suggested on Monday (got other plans for this weekend) and let you know the outcome.
    If my memory serves me right, the webcam does not support TCP transmission, or at least I couldn't establish one (it's a cheap china import). Packet loss was my first guess too, but I didn't know live555 had the capability to report one. Well, I guess we'll see on Monday.

    Thanks again!

  •  onkel_keks

    onkel_keks - 2013-04-29

    Small update: I tried the openRTSP client from live555 (latest version). There was not a single packet lost, and connection to the cam via TCP was also possible.
    I will now try to get the packet statistics from live555 within VPP and see if there are any lost packets there. I also tried increasing the UDP receive buffer up to 50MB, but to no avail. If I'll get around to it, I'll try to compile the VPP source with the latest live555 sources.

    EDIT: ok, the RTP stats that live555 gives me from within the VPP sources do not indicate packet loss either (packets received/expected/lost: 2819/2819/0) - yet I still have those artifacts. I guess that leaves said "downstream inefficiency"...

    EDIT2: just fyi, I now tried the openRTSP.exe built from the live555-sources coming with VPP - works flawlessly, i.e., the problem is not an old live555 version.

    Last edit: onkel_keks 2013-04-29
  • Ralf Globisch

    Ralf Globisch - 2013-04-29

    Ok, thanks for the update. Just a few points off the top of my head:

    • There was some locking on the shared queue between the live555 thread and the source filter pin threads.

    • Without wanting to send you on a wild goose chase, one thing to try that shouldn't take too much time is to output all the NAL units to a .264 file and then to play back the "raw" file and check for the same artifacts. If you connect the RTSP source filter to a file writer (IIRC there was a dump filter in the DirectShow samples) and then play the file, that could give some further clues. I'm just in the process of relocating so I don't have time to look at the source myself at the moment.

    • Another thing could be the pre-buffering code or code that aligns video and audio streams.

    I am interested in solving the issue though (especially if the problem is in our sample filter) so please let me know if you find anything. I will try to find some time as soon as possible....

  •  onkel_keks

    onkel_keks - 2013-05-02

    Hello Ralf,
    I am happy to tell you that I finally found the solution to my problem: in RtspClientSessionManager::SetupMediaSession, when you create the RtvcRtpSink, you specify 20000 as buffer size, which is about 20KB. When I increased this buffer to 80KB, the picture was perfect without any artifacts. I guess this has something to do with the webcam sending rather high-res pictures, but that's just a gut feeling.
    I set the buffer (uiSinkBufferSize) for my application to 100*1024 (i.e., 100 KB) just to be safe.

    Quick follow-up question, if I were to add audio support for H.264 video with, say, AAC audio over RTSP, how much effort would that be (in terms of Lines of Code or work hours, very roughly)? Is it something that can be done in a day or is it a rather lengthy endeavour?

  • Ralf Globisch

    Ralf Globisch - 2013-05-02


    I'm glad you found the issue, I'll update the default buffer size in our code as well!

    Generally it's relatively quick to add new media types as long as live555 has implemented the RTP payload format. IIRC there is support for AAC hbr in live555.

    The main thing you then have to do to extend our code is to set the output pin media type correctly, based on the SDP retrieved via the RTSP setup. Then it depends entirely on what the downstream AAC decoder is expecting/requires.

    With respect to how long it takes, I can't say, it depends entirely on the format. I would suggest taking a look at how an encoder sets it's output pin media type: The source is (used to be) available via SVN and there's a decoder too. That usually gives you a good starting point. At the very least you'll see roughly what you have to do. Of course if you use a different decoder, you might have to set it slightly different. Hope that helps.

    Also, if you have permission to and want to, feel free to contribute that code back to VPP and I'll add the changes into the filter.



Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:

No, thanks