Re: [linux-uvc-devel] Logitech C920 dropped/duplicated frames / decreasing timestamp problem (h.264)

[Rucio D. <sf-...@un...>]

A quick update.

I reversed the changes introduced in bisect work that Peter Rabbitson
posted in the link here:
http://sourceforge.net/p/linux-uvc/mailman/message/33164469/ and was able
to eliminate the problem entirely.

He is correct in saying that the bug was introduced in this commit:

https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=66847ef

Beyond that, I have no idea what the additional code is doing or the reason
behind it, but by simply extracting it from the current source the problem
goes away.

The video in the link below was captured using gstreamer and the v4l2src
(same command found below in my original email).
https://drive.google.com/a/ungulate.net/file/d/0B2VWNnMtikioaHhXV2xVM3BZR3c/view?usp=sharing

In reviewing avprobe -show_packets, the pts nor the pts_time value were
found to be decreasing in value or out of order.

Not sure where we go from here since I'm not a developer. ?

Rucio

On Sun, Mar 8, 2015 at 12:40 AM, Rucio Donk <sf-...@un...> wrote:

> Greetings,
>
> I am having an issue where some of the h.264 timestamps from a v4l2 source
> appear to decrease in time or are out of order roughly ever 2.5 seconds of
> captured video. The result is that the video frames are duplicated or
> dropped causing the appearance of paused frames or missing frames.
>
> This issue appears to have been reported from others already, but sadly
> the threads all seem to die with no solution given. At the bottom I have
> included a collection of references to other posts about this issue. If
> anyone has any suggestions on how to resolve this issue, I would be very
> grateful. If I can provide more information to help get to the bottom of
> this issue, please let me know and I will provide it.
>
> Issue -
>
> Captured H.264 video from the Logitech C920 camera in the H.264 format in
> either VLC or gstreamer using the v4l2 source reports dropped/duplicated
> frames and reports a decreasing timestamp error - sometimes reported as
> 'too late to be displayed' in the case of VLC. This causes the appearance
> of a "pause" / frame duplication when played back live or from the captured
> file. This pause in the video occurs approximately once every 2.5 seconds.
>
> However, when video is captured using the uvch264src from gstreamer, this
> problem does not occur and the video produced plays back perfectly with no
> pauses or playback problems.
>
> I have captured two mp4 videos using gstreamer 1.2 and have linked to them
> below. The first was captured using the uvch264src source and the second
> using the v4l2src. Although the uvch264src is sometimes unreliable, it
> produces the most consistent and error-free video I can acquire from the
> Logitech C920 camera. The second video however when played back using vlc
> -v produces four "picture is too late to be displayed (missing ### ms)"
> errors.
>
> Good capture -
>
> gst-launch-1.0 -vv -e uvch264src device=/dev/video0 name=src
> auto-start=true src.vfsrc ! queue ! video/x-raw, format=YUY2, width=160,
> height=90, framerate=30/1 ! fakesink src.vidsrc ! queue !
> video/x-h264,width=1920, height=1080,framerate=30/1 ! h264parse ! mp4mux !
> filesink location=noframe_issue.mp4
>
> noframe_issue.mp4 download link:
>
> https://drive.google.com/a/ungulate.net/file/d/0B2VWNnMtikioYmVOY1FieUxvOEU/view?usp=sharing
>
> Bad capture -
>
> gst-launch-1.0 -v -e v4l2src device=/dev/video0 ! video/x-h264,
> width=1920, height=1080, framerate=30/1 ! queue ! h264parse ! mp4mux !
> filesink location=frame_issue.mp4
>
> frame_issue.mp4 download link:
>
> https://drive.google.com/a/ungulate.net/file/d/0B2VWNnMtikioUjJLVThydkcxcHM/view?usp=sharing
>
> Steps I've done to attempt to eliminate the problem -
>
> Downloaded and built the latest media_build, tried three different
> Logitech C920 cameras, plugged them into a Windows box to verify that the
> cameras are functional and to update any firmware that the camera may have
> (I don't believe it does), compiled the latest v4l2 and uvcvideo code, and
> tried on different hardware including the Raspberry Pi B+ and the Raspberry
> Pi2 B, as well as Ubuntu 14.04 and Ubuntu 14.10 on three different boxes.
>
> Other observations -
>
> When I run avprobe -show_packets against the two files, I noticed that in
> the mp4 produced from the v4l2src, some of the the pts and pts_time values
> appear out of order/decrease in the frame_issue.mp4 (they decrease in time
> instead of increase in time.) but appear in perfect order in the
> noframe_issue.mp4.
>
> Example:
>
> [packets.packet.20]
> codec_type=video
> stream_index=0
> pts=3074
> pts_time=1.024667
> dts=2000
> dts_time=0.666667
> duration=100
> duration_time=0.033333
> size=14056.000000
> pos=304362
> flags=_
>
> [packets.packet.21]
> codec_type=video
> stream_index=0
> pts=2661
> pts_time=0.887000
> dts=2100
> dts_time=0.700000
> duration=100
> duration_time=0.033333
> size=14313.000000
> pos=318418
> flags=_
>
> Links that may be a reference to this problem -
>
> Re: Help needed to work around incorrect DTS when capturing H264 from a
> Logitech C920
> http://permalink.gmane.org/gmane.comp.video.ffmpeg.user/55069
>
> Logitech C920 capture broken by UVC timestamp support (66847ef)
> http://sourceforge.net/p/linux-uvc/mailman/message/33164469/
> (Link above includes bisected kernel source to show where the issue was
> introduced)
>
> UVC driver produce decreasing timestamp
>
> http://permalink.gmane.org/gmane.linux.drivers.video-input-infrastructure/79344
> <http://article.gmane.org/gmane.linux.drivers.video-input-infrastructure/79344>
> (Possibly related)
>
>
> Thanks!
> Rucio
>