[linux-uvc-devel] Logitech C920 dropped/duplicated frames / decreasing timestamp problem (h.264)

[Rucio D. <sf-...@un...>]

Greetings,

I am having an issue where some of the h.264 timestamps from a v4l2 source
appear to decrease in time or are out of order roughly ever 2.5 seconds of
captured video. The result is that the video frames are duplicated or
dropped causing the appearance of paused frames or missing frames.

This issue appears to have been reported from others already, but sadly the
threads all seem to die with no solution given. At the bottom I have
included a collection of references to other posts about this issue. If
anyone has any suggestions on how to resolve this issue, I would be very
grateful. If I can provide more information to help get to the bottom of
this issue, please let me know and I will provide it.

Issue -

Captured H.264 video from the Logitech C920 camera in the H.264 format in
either VLC or gstreamer using the v4l2 source reports dropped/duplicated
frames and reports a decreasing timestamp error - sometimes reported as
'too late to be displayed' in the case of VLC. This causes the appearance
of a "pause" / frame duplication when played back live or from the captured
file. This pause in the video occurs approximately once every 2.5 seconds.

However, when video is captured using the uvch264src from gstreamer, this
problem does not occur and the video produced plays back perfectly with no
pauses or playback problems.

I have captured two mp4 videos using gstreamer 1.2 and have linked to them
below. The first was captured using the uvch264src source and the second
using the v4l2src. Although the uvch264src is sometimes unreliable, it
produces the most consistent and error-free video I can acquire from the
Logitech C920 camera. The second video however when played back using vlc
-v produces four "picture is too late to be displayed (missing ### ms)"
errors.

Good capture -

gst-launch-1.0 -vv -e uvch264src device=/dev/video0 name=src
auto-start=true src.vfsrc ! queue ! video/x-raw, format=YUY2, width=160,
height=90, framerate=30/1 ! fakesink src.vidsrc ! queue !
video/x-h264,width=1920, height=1080,framerate=30/1 ! h264parse ! mp4mux !
filesink location=noframe_issue.mp4

noframe_issue.mp4 download link:
https://drive.google.com/a/ungulate.net/file/d/0B2VWNnMtikioYmVOY1FieUxvOEU/view?usp=sharing

Bad capture -

gst-launch-1.0 -v -e v4l2src device=/dev/video0 ! video/x-h264, width=1920,
height=1080, framerate=30/1 ! queue ! h264parse ! mp4mux ! filesink
location=frame_issue.mp4

frame_issue.mp4 download link:
https://drive.google.com/a/ungulate.net/file/d/0B2VWNnMtikioUjJLVThydkcxcHM/view?usp=sharing

Steps I've done to attempt to eliminate the problem -

Downloaded and built the latest media_build, tried three different Logitech
C920 cameras, plugged them into a Windows box to verify that the cameras
are functional and to update any firmware that the camera may have (I don't
believe it does), compiled the latest v4l2 and uvcvideo code, and tried on
different hardware including the Raspberry Pi B+ and the Raspberry Pi2 B,
as well as Ubuntu 14.04 and Ubuntu 14.10 on three different boxes.

Other observations -

When I run avprobe -show_packets against the two files, I noticed that in
the mp4 produced from the v4l2src, some of the the pts and pts_time values
appear out of order/decrease in the frame_issue.mp4 (they decrease in time
instead of increase in time.) but appear in perfect order in the
noframe_issue.mp4.

Example:

[packets.packet.20]
codec_type=video
stream_index=0
pts=3074
pts_time=1.024667
dts=2000
dts_time=0.666667
duration=100
duration_time=0.033333
size=14056.000000
pos=304362
flags=_

[packets.packet.21]
codec_type=video
stream_index=0
pts=2661
pts_time=0.887000
dts=2100
dts_time=0.700000
duration=100
duration_time=0.033333
size=14313.000000
pos=318418
flags=_

Links that may be a reference to this problem -

Re: Help needed to work around incorrect DTS when capturing H264 from a
Logitech C920
http://permalink.gmane.org/gmane.comp.video.ffmpeg.user/55069

Logitech C920 capture broken by UVC timestamp support (66847ef)
http://sourceforge.net/p/linux-uvc/mailman/message/33164469/
(Link above includes bisected kernel source to show where the issue was
introduced)

UVC driver produce decreasing timestamp
http://permalink.gmane.org/gmane.linux.drivers.video-input-infrastructure/79344
<http://article.gmane.org/gmane.linux.drivers.video-input-infrastructure/79344>
(Possibly related)


Thanks!
Rucio