unable to play the received h.264 encoded video stream from an IP camera via RTSP

  • Sowmya

    Sowmya - 2013-08-06

    I'm fairly new to Directshow and trying to implement H.264 video support for our product. Currently our product supports MJPEG video reception.

    I'm trying to receive video from an IP camera that transmits H.264 encoded stream over RTSP and I'm using your RTSPSourceFilter for receiving it and then feed it to the Microsoft Video DTV-DVD Video decoder or the built in IP camera H.264 Video decoder for playback.

    Not sure what is going wrong, but all I get is a black screen in the video playback in Graphstudio.
    1. When connected to Microsoft DTV-DVD Video Decoder - No frames are being delivered when I right click on the properties of the Video Renderer
    2. When connected to IP camera H.264 Video Decoder - Frames are being delivered but still it is a black screen with no video.

    But I can use VLC player to connect to the IP camera url and play video successfully.

    During SDP exchange, I get this line from the IP camera
    a=fmtp:96 packetization-mode=1; profile-level-id=420029; sprop-parameter-sets=Z0IAKeKQFAe2AtwEBAaQeJEV,aM48gA==

    So I know that SPS and PPS parameters need to be sent to the decoder.
    1) But I'm not sure how to feed SPS and PPS to the decoder?
    2) what should the mediasubtype be set to?
    3) Any other things that need to be looked into for successful playback.

    Any examples of setting mediatype properties and sending SPS and PPS values to decoder will be of great help.

    Many thanks

  • Ralf Globisch

    Ralf Globisch - 2013-08-06

    Hi Sowmya,

    As far as I remember the filter was configured with the DS media type where the MS decoder expects the SPS and PPS in-band. See http://msdn.microsoft.com/en-us/library/dd757808%28VS.90%29.aspx. The width and height settings in the filter code don't matter.

    As a first step, can you test the RTSP source filter using the live555 media server with the .264 files available at http://live555.com/liveMedia/public/264/.

    Then things to check are:

    • is the RTSP exchange ok? E.G use wireshark. AFAIR the output pin on the filter is only generated once the RTSP is ok so I'm guessing that it's not a problem in your case.

    • Does the media reach the client? I.e. check that the packets are received and not blocked by a firewall. If UDP is blocked, try use RTP over RTSP/TCP.

    • Does the media reach the DS pipeline: set a breakpoint inRtspSourceOutputPin::FillBuffer at the line copyMediaDataIntoSample(pSample, pSampleData );

    If the breakpoint is hit, then at least you know that the data is getting into the DirectShow media pipeline.

  • Sowmya

    Sowmya - 2013-08-06

    Thank you for the reply.
    The url http://msdn.microsoft.com/en-us/library/dd757808%28VS.90%29.aspx doesn't seem to open.

    Yes I'm able to successfully play foreman.264 file or other sample *.264 files that I downloaded from your project or live555 website. I first launched live555MediaServer.exe that streams .264 files and then gave the correct url to RTSPSourceFilter and it played successfully in GraphStudio.

    I'm seeing that data is actually being copied in copyMediaDataIntoSample(pSample, pSampleData) and the data is also going to the Deliver(..) function in DoBufferProcessingLoop().

    1. Not sure how to check the firewall issue and also how to use RTP over RTSP/TCP
    2. Also how do we send SPS and PPS in band?
  • Sowmya

    Sowmya - 2013-08-06

    Hi Ralf
    The above url opened but i'm still confused what mediasubtype to use? how to send SPS and PPS in band? Could you also let me know how to check the firewall issue and how to use RTP over RTSP/TCP?

    Many Thanks

  • Ralf Globisch

    Ralf Globisch - 2013-08-06

    The url http://msdn.microsoft.com/en-us/library/dd757808%28VS.90%29.aspx doesn't seem to open.

    Ok, strange, it opens from here. Google '"H.264 Video Types" Directshow' (including the double quotes) to find the article.

    1.) If data is being copied then there is no firewall issue.
    2.) They need to be in the bitstream itself. Alternatively use the other media type (MEDIASUBTYPE_AVC1). IIRC you add the SPS/PPS to the DS media format block used during media type negotiation. And then you need to format the data differently i.e. without start codes (search for that MSDN article). I've never tried this approach though.
    3.) You can also try changing the buffer size as mentioned in https://sourceforge.net/p/videoprocessing/discussion/933075/thread/98ac9692/. The OP found it to be an issue with high-res issues.

  • Sowmya

    Sowmya - 2013-08-06

    I will increase the buffersize and see if it works.

  • Sowmya

    Sowmya - 2013-08-07

    Hi Ralf

    Letting you know the options that I have tried. The IP camera that I'm connecting to is available at and i'm connecting to it using the url rtsp://

    Not sure if you can also connect to it and test?

    To connect to Microsoft-DTV-DVD Video Decoder,the following lines of code need to be changed.
    In the function, void RtspSourceOutputPin::initialiseMediaType(...), for H264 Video, Change the lines.(Original code is in comments)

    const unsigned uiWidth = 640;
    const unsigned uiHeight = 480;

    1. pvi2->bmiHeader.biSize = sizeof(BITMAPINFOHEADER);// uiWidth * uiHeight * 3;
    2. pvi2->bmiHeader.biSizeImage = uiWidth * uiHeight * 3; // DIBSIZE(pvi2->bmiHeader);
    3. const REFERENCE_TIME FPS_30 = UNITS / 30; //FPS_25 originally
      pvi2->AvgTimePerFrame = FPS_30;

    4. In RTSPClientSessionManager, in setupMediaSession(), I also changed buffersize
      unsigned uiSinkBufferSize = 1000000;//20000;

    But still in vain. The video just shows a black screen and no frames are coming to renderer.

    To connect to Axis IP camera H.264 Decoder, commented the last line

    In the function setSampleTimestamps(...)
    // RG: setting timestamps according to the above link causes artifacts in video: still need to determine why
    // Remove the following line once issue is sorted out
    //pSample->SetTime(NULL, NULL);

    The video is still a black screen but I can see that frames are being rendered when I click on renderer properties.

    Could you please provide any further ideas? Could you also provide information on what is the difference in sending SPS and PPS in band and out band?

    Warm Regards

    • Ralf Globisch

      Ralf Globisch - 2013-08-13

      Firstly, FYI that looks like a local IP to me :-), and I also unfortunately do not have the time to help you debug.

      The video is still a black screen but I can see that frames are being rendered when I click on renderer properties. Could you please provide any further ideas?

      Something you could check is to connect the RTSP source filter to a Dump filter. IIRC one of the DirectShow samples contains the Dump filter, but you'll have to search for it. The dump filter basically writes the received data to file as is: in this case raw H.264 NAL units. You could take a look at this file to see what is actually being passed to the Axis decoder. That may or may not give you more clues as to what is going wrong. You could then try to decode the "dumped" file with the HHI H.264 reference decoder. That could also give you some more clues, but requires that you have some H.264 knowledge....

      Could you also provide information on what is the difference in sending SPS and PPS in band and out band?
      It's a different way of communicating the video parameters, it really depends on what a specific decoder expects. The decoder needs to know what kind of video it is decoding i.e. how it has been encoded, it needs to know the width and height, etc.


Log in to post a comment.