libquicktime-devel — Discussion about development

Re: [Libquicktime-devel] FFmpeg plugin resync DNW for out-of-order B-frames after full sync keyframes

[Derek C. <der...@mi...>]

Okay, I'll keep in mind that this should also support MP4.

> The code for calculating the pts does this right for both cases because the first ctts value is
> alaways zero for quicktime and max_decoding_delay for mp4. I don't see how the PTS can get
> negative here.

In the beginning of this thread, as with the "quicktime_is_bframe()
breaking for wobbly GOP pattern" thread, I describe an XDCAM quicktime
file with initial non-zero ctts value... The qtdump output looks like:

...
     composition time to sample (ctts)
      version 0
      flags 0
      total_entries 46437
       count 1 duration 2
       count 2 duration -1
       count 1 duration 2
...

Hence the PTS gets negative!  I'll try to come up with a solution that
can shift the PTS scale to that doesn't contain negative values
without compromising MP4 support?

-- 


Derek Chow / C++ Developer

der...@mi... / +44(0)7825373230

Office +44 (0)207 884 2530
21 Broadwall, London, SE1 9PL, United Kingdom
http://www.mirriad.com

Re: [Libquicktime-devel] FFmpeg plugin resync DNW for out-of-order B-frames after full sync keyframes

[Burkhard P. <pl...@ig...>]

Hi,

Am 29.07.2014 um 17:57 schrieb Derek Chow:
>> By the way I've also checked in a new possible variant of the resync() function
>> in the ffmpeg decoder.
>
> Okay, sorry for the delay, but here some fixes I've got for the new
> resync code (apply in order as listed).
>
> [no_negative_pts.diff]
> I decided to remove the logic that ensures the first decode sample
> also PTS 0 (so it could be greater than 0 due to ctts offset), for the
> following reasons:
>   - it's more intuitive to pass 0 instead of some negative value to
> lqt_seek_video(), to seek to the beginning of the video;
>   - according to lqt_seek_video()'s doxygen comments, it doesn't expect
> negative time parameters (though the logic seems okay otherwise);
>   - lqt_packet_index_get_next_display_frame doesn't expect there to be
> negative PTS.

-      e.pts += ctts->table[ctts_index].sample_duration -
-        ctts->table[0].sample_duration;
+      e.pts += ctts->table[ctts_index].sample_duration;

This is necessary because the ctts values are defined differently by quicktime and mp4.

In quicktime, the first PTS is equal to the first DTS, which implies that for some frames
the ctts values are negative, resulting in DTS > PTS (violating causality). Not a big deal if
you assume that PTS and DTS are on different scales, which are shifted by the maximum decoding
delay.

In mp4, DTS is always < PTS.

The code for calculating the pts does this right for both cases because the first ctts value is
alaways zero for quicktime and max_decoding_delay for mp4. I don't see how the PTS can get
negative here.

Didn't look at the other patches yet.

Burkhard

Re: [Libquicktime-devel] Multiple video samples per chunk

[Burkhard P. <pl...@ig...>]

Hi,

better late than never....

Am 30.07.2014 um 14:22 schrieb Phil Barrett:
> Anyone done any work on supporting writing multiple video samples per
> chunk?

No.

> The code at the moment always writes one sample per chunk, and in
> at least one place relies on this because it uses chunk_num to mean
> sample_num.

The question is if it would make any sense to support multiple video samples
per chunk.

I have a vague idea tom support the packetindex also for writing, i.e. generate
the packet index during the encode() calls and generate the Quicktime- or AVI-Structures
just in the end when the file is closed. But right now I'm unlikely to have time and
motivation to implement it since it wouldn't bring an immediate advantage, except a
simpler code.

Burkhard

Re: [Libquicktime-devel] FFmpeg plugin resync DNW for out-of-order B-frames after full sync keyframes

[Derek C. <der...@mi...>]

Just realised I didn't attach those updated patches - sorry!

On 29 July 2014 17:19, Derek Chow <der...@mi...> wrote:
> Ahh here's a more up to-date version of the latter 2 patches:
>
> [fix_long_gop_resync_next_display_frame.diff]
>  - fixing a comment and error clause for
> lqt_packet_index_get_next_display_frame().  -- Note that it only
> handles keyframes, assuming they will not have negative ctts.
>
> [fix_resync_bframe_skipping.diff]
>  - updated patch to avoid hunk offset/fuzz - otherwise nothing changed here.
>
> On 29 July 2014 16:57, Derek Chow <der...@mi...> wrote:
>>> By the way I've also checked in a new possible variant of the resync() function
>>> in the ffmpeg decoder.
>>
>> Okay, sorry for the delay, but here some fixes I've got for the new
>> resync code (apply in order as listed).
>>
>> [no_negative_pts.diff]
>> I decided to remove the logic that ensures the first decode sample
>> also PTS 0 (so it could be greater than 0 due to ctts offset), for the
>> following reasons:
>>  - it's more intuitive to pass 0 instead of some negative value to
>> lqt_seek_video(), to seek to the beginning of the video;
>>  - according to lqt_seek_video()'s doxygen comments, it doesn't expect
>> negative time parameters (though the logic seems okay otherwise);
>>  - lqt_packet_index_get_next_display_frame doesn't expect there to be
>> negative PTS.
>>
>>
>> [fix_long_gop_resync_next_display_frame.diff]
>> I noticed that quicktime_set_video_position() and
>> quicktime_init_maps() was missing a check for next_display_frame
>> values when the keyframe it leaves on actually has a 'ctts' offset.
>> Hence I added a lqt_packet_index_get_resync_display_frame() method to
>> help.
>>
>>
>> And finally...
>> [fix_resync_bframe_skipping.diff]
>> Possible fix for the issue I've mentioned in this thread for resync -
>> lemme know what you think:
>>  - Rather than skip the entire first group of B-frames (at risk of
>> skipping reference B-frames, or those that were part of the output we
>> want to keep), skip any B-frame provided that:
>>    - they aren't reference frames;
>>    - they are truly 'obsolete' by the definition that their PTS is
>> less than the desired timestamp.
>> Also includes:
>>  - Fix for crash when trying to call avcodec_flush_buffers() when
>> there isn't a valid structure for it (possible when calling seek
>> before any decoding)/
>>  - Adjusted signature of lqt_packet_index_get_next_display_frame() and
>> hence renamed it to lqt_packet_index_increment_display_frame() for
>> more code-reuse.
>>  - Fix to increment 'next_display_frame' when skipping B-frames so
>> that the while() loop terminates accurately.  E.g. Seeking to PTS 2
>> when samples starting from the keyframe go 2, 0, 1... will feed the
>> first sample, then skip the 2nd and 3rd before terminating the loop.
>>
>>
>> I've come across a horrible file with duplicate PTS, are we supposed
>> to feed both samples and expect only one frame for the PTS to come
>> out?  See snippet of qtdump output below:
>> ...
>>>      composition time to sample (ctts)
>>>       version 0
>>>       flags 0
>>>       total_entries 82615
>>>        count 1 duration 0
>>>        count 1 duration 1000
>>>        count 2 duration 0
>>>        count 1 duration 1000
>> ...
>>>      Packet index (generated)
>>>       num_entries 84530
>>>        pos 1799498 pts 0 dts 0 dur 1000 sz 14031 FT I stsd_id 0 key ref
>>>        pos 1813529 pts 2000 dts 1000 dur 1000 sz 31711 FT P stsd_id 0 ref
>>>        pos 1845240 pts 2000 dts 2000 dur 1000 sz 42 FT P stsd_id 0 ref
>>>        pos 1845282 pts 3000 dts 3000 dur 1000 sz 34804 FT P stsd_id 0 ref
>>
>>
>>
>> Cheers
>
>
>
> --
>
>
> Derek Chow / C++ Developer
>
> der...@mi... / +44(0)7825373230
>
> Office +44 (0)207 884 2530
> 21 Broadwall, London, SE1 9PL, United Kingdom
> http://www.mirriad.com



-- 


Derek Chow / C++ Developer

der...@mi... / +44(0)7825373230

Office +44 (0)207 884 2530
21 Broadwall, London, SE1 9PL, United Kingdom
http://www.mirriad.com

[Libquicktime-devel] Multiple video samples per chunk

[Phil B. <ph...@fi...>]

Anyone done any work on supporting writing multiple video samples per  
chunk? The code at the moment always writes one sample per chunk, and in  
at least one place relies on this because it uses chunk_num to mean  
sample_num.

Phil
-- 
Phil Barrett
FilmLight Ltd

Re: [Libquicktime-devel] FFmpeg plugin resync DNW for out-of-order B-frames after full sync keyframes

[Derek C. <der...@mi...>]

> By the way I've also checked in a new possible variant of the resync() function
> in the ffmpeg decoder.

Okay, sorry for the delay, but here some fixes I've got for the new
resync code (apply in order as listed).

[no_negative_pts.diff]
I decided to remove the logic that ensures the first decode sample
also PTS 0 (so it could be greater than 0 due to ctts offset), for the
following reasons:
 - it's more intuitive to pass 0 instead of some negative value to
lqt_seek_video(), to seek to the beginning of the video;
 - according to lqt_seek_video()'s doxygen comments, it doesn't expect
negative time parameters (though the logic seems okay otherwise);
 - lqt_packet_index_get_next_display_frame doesn't expect there to be
negative PTS.


[fix_long_gop_resync_next_display_frame.diff]
I noticed that quicktime_set_video_position() and
quicktime_init_maps() was missing a check for next_display_frame
values when the keyframe it leaves on actually has a 'ctts' offset.
Hence I added a lqt_packet_index_get_resync_display_frame() method to
help.


And finally...
[fix_resync_bframe_skipping.diff]
Possible fix for the issue I've mentioned in this thread for resync -
lemme know what you think:
 - Rather than skip the entire first group of B-frames (at risk of
skipping reference B-frames, or those that were part of the output we
want to keep), skip any B-frame provided that:
   - they aren't reference frames;
   - they are truly 'obsolete' by the definition that their PTS is
less than the desired timestamp.
Also includes:
 - Fix for crash when trying to call avcodec_flush_buffers() when
there isn't a valid structure for it (possible when calling seek
before any decoding)/
 - Adjusted signature of lqt_packet_index_get_next_display_frame() and
hence renamed it to lqt_packet_index_increment_display_frame() for
more code-reuse.
 - Fix to increment 'next_display_frame' when skipping B-frames so
that the while() loop terminates accurately.  E.g. Seeking to PTS 2
when samples starting from the keyframe go 2, 0, 1... will feed the
first sample, then skip the 2nd and 3rd before terminating the loop.


I've come across a horrible file with duplicate PTS, are we supposed
to feed both samples and expect only one frame for the PTS to come
out?  See snippet of qtdump output below:
...
>      composition time to sample (ctts)
>       version 0
>       flags 0
>       total_entries 82615
>        count 1 duration 0
>        count 1 duration 1000
>        count 2 duration 0
>        count 1 duration 1000
...
>      Packet index (generated)
>       num_entries 84530
>        pos 1799498 pts 0 dts 0 dur 1000 sz 14031 FT I stsd_id 0 key ref
>        pos 1813529 pts 2000 dts 1000 dur 1000 sz 31711 FT P stsd_id 0 ref
>        pos 1845240 pts 2000 dts 2000 dur 1000 sz 42 FT P stsd_id 0 ref
>        pos 1845282 pts 3000 dts 3000 dur 1000 sz 34804 FT P stsd_id 0 ref



Cheers

Re: [Libquicktime-devel] FFmpeg plugin resync DNW for out-of-order B-frames after full sync keyframes

[Derek C. <der...@mi...>]

Ahh here's a more up to-date version of the latter 2 patches:

[fix_long_gop_resync_next_display_frame.diff]
 - fixing a comment and error clause for
lqt_packet_index_get_next_display_frame().  -- Note that it only
handles keyframes, assuming they will not have negative ctts.

[fix_resync_bframe_skipping.diff]
 - updated patch to avoid hunk offset/fuzz - otherwise nothing changed here.

On 29 July 2014 16:57, Derek Chow <der...@mi...> wrote:
>> By the way I've also checked in a new possible variant of the resync() function
>> in the ffmpeg decoder.
>
> Okay, sorry for the delay, but here some fixes I've got for the new
> resync code (apply in order as listed).
>
> [no_negative_pts.diff]
> I decided to remove the logic that ensures the first decode sample
> also PTS 0 (so it could be greater than 0 due to ctts offset), for the
> following reasons:
>  - it's more intuitive to pass 0 instead of some negative value to
> lqt_seek_video(), to seek to the beginning of the video;
>  - according to lqt_seek_video()'s doxygen comments, it doesn't expect
> negative time parameters (though the logic seems okay otherwise);
>  - lqt_packet_index_get_next_display_frame doesn't expect there to be
> negative PTS.
>
>
> [fix_long_gop_resync_next_display_frame.diff]
> I noticed that quicktime_set_video_position() and
> quicktime_init_maps() was missing a check for next_display_frame
> values when the keyframe it leaves on actually has a 'ctts' offset.
> Hence I added a lqt_packet_index_get_resync_display_frame() method to
> help.
>
>
> And finally...
> [fix_resync_bframe_skipping.diff]
> Possible fix for the issue I've mentioned in this thread for resync -
> lemme know what you think:
>  - Rather than skip the entire first group of B-frames (at risk of
> skipping reference B-frames, or those that were part of the output we
> want to keep), skip any B-frame provided that:
>    - they aren't reference frames;
>    - they are truly 'obsolete' by the definition that their PTS is
> less than the desired timestamp.
> Also includes:
>  - Fix for crash when trying to call avcodec_flush_buffers() when
> there isn't a valid structure for it (possible when calling seek
> before any decoding)/
>  - Adjusted signature of lqt_packet_index_get_next_display_frame() and
> hence renamed it to lqt_packet_index_increment_display_frame() for
> more code-reuse.
>  - Fix to increment 'next_display_frame' when skipping B-frames so
> that the while() loop terminates accurately.  E.g. Seeking to PTS 2
> when samples starting from the keyframe go 2, 0, 1... will feed the
> first sample, then skip the 2nd and 3rd before terminating the loop.
>
>
> I've come across a horrible file with duplicate PTS, are we supposed
> to feed both samples and expect only one frame for the PTS to come
> out?  See snippet of qtdump output below:
> ...
>>      composition time to sample (ctts)
>>       version 0
>>       flags 0
>>       total_entries 82615
>>        count 1 duration 0
>>        count 1 duration 1000
>>        count 2 duration 0
>>        count 1 duration 1000
> ...
>>      Packet index (generated)
>>       num_entries 84530
>>        pos 1799498 pts 0 dts 0 dur 1000 sz 14031 FT I stsd_id 0 key ref
>>        pos 1813529 pts 2000 dts 1000 dur 1000 sz 31711 FT P stsd_id 0 ref
>>        pos 1845240 pts 2000 dts 2000 dur 1000 sz 42 FT P stsd_id 0 ref
>>        pos 1845282 pts 3000 dts 3000 dur 1000 sz 34804 FT P stsd_id 0 ref
>
>
>
> Cheers



-- 


Derek Chow / C++ Developer

der...@mi... / +44(0)7825373230

Office +44 (0)207 884 2530
21 Broadwall, London, SE1 9PL, United Kingdom
http://www.mirriad.com

Re: [Libquicktime-devel] Delay determination of decoder colormodel

[Erik J. <eri...@gm...>]

On Tue, May 6, 2014 at 5:36 PM, Burkhard Plaum
<pl...@ig...> wrote:
>> Different issue: Could we have a max_stsd_index (or equivalent) in
>> lqt_packet_index_t too?
>
> That would be the same as stsd->total_entries, right?

Yes, it's the same.

> I won't use or support multiple sample descriptions per track. But
> I wonder that if one would use it, we'd need to export the stsd atom for the
> public API anyway, or not?

To add partial support for multiple stsd entries I've hacked, wrapped
and exposed a version of quicktime_stsd_get_user_atom() that takes a
parameter for the stsd entry I'm looking for. But clearly, that is
insufficient for e.g. resolution changes. Exposing the entire stsd
atom would allow support for multiple sample descriptions per track
without having to modify the library, so would be a good way forwards
for my needs.

Thanks
// Erik

Re: [Libquicktime-devel] Delay determination of decoder colormodel

[Burkhard P. <pl...@ig...>]

Hi,

Am 06.05.2014 10:26, schrieb Erik Johansson:
> Hi Burkhard,
>
>> I implemented it a bit differently:
>> A private function lqt_ensure_stream_cmodel_decode() is called from
>> all functions, which use the colormodel. Can you check if that works for you?
>
> That is a lot cleaner, thanks. Works ok in my test, which also served
> as a reminder that my local build of the library is starting to lag
> quite a bit behind the current CVS. Thanks for your efforts here!

Good.

> Different issue: Could we have a max_stsd_index (or equivalent) in
> lqt_packet_index_t too?

That would be the same as stsd->total_entries, right?

I won't use or support multiple sample descriptions per track. But
I wonder that if one would use it, we'd need to export the stsd atom for the
public API anyway, or not?

Burkhard

Re: [Libquicktime-devel] Delay determination of decoder colormodel

[Erik J. <eri...@gm...>]

Hi Burkhard,

> I implemented it a bit differently:
> A private function lqt_ensure_stream_cmodel_decode() is called from
> all functions, which use the colormodel. Can you check if that works for you?

That is a lot cleaner, thanks. Works ok in my test, which also served
as a reminder that my local build of the library is starting to lag
quite a bit behind the current CVS. Thanks for your efforts here!

Different issue: Could we have a max_stsd_index (or equivalent) in
lqt_packet_index_t too?

Thanks
// Erik

Re: [Libquicktime-devel] Delay determination of decoder colormodel

[Burkhard P. <pl...@ig...>]

Hi,

Am 29.04.2014 15:55, schrieb Erik Johansson:
> I realise I had omitted the initialisation of the stream_cmodel
> variable in my patch. My apologies - attaching an updated patch.

I implemented it a bit differently:
A private function lqt_ensure_stream_cmodel_decode() is called from
all functions, which use the colormodel. Can you check if that works for you?

Burkhard

[Libquicktime-devel] New audio decoding API

[Burkhard P. <pl...@ig...>]

Hi,

while upgrading the codecs for the new demuxing API, I also
did some major updates for the Audio decoding:

- We now support audio decoders with non-interleaved output. This
   makes the ffmpeg mp3 and ac3 decoders work again.

- Audio decoders have a much simplified method decode_audio_packet(),
   which takes the next packet/chunk from the file and decodes it.
   All other stuff, like buffering, seeking etc. is done by the core.
   This makes the codecs much simpler.

- The audio related public API functions are unchanged and should work
   like before

Now I'm thinking if the packet index could also be used for writing
files: Just write packets for into the file and build a packet index.
Before closing the file, we can generate all the crazy quicktime tables.

Burkhard

Re: [Libquicktime-devel] Delay determination of decoder colormodel

[Erik J. <eri...@gm...>]

I realise I had omitted the initialisation of the stream_cmodel
variable in my patch. My apologies - attaching an updated patch.

// Erik

On Tue, Apr 29, 2014 at 12:28 PM, Erik Johansson
<eri...@gm...> wrote:
> Hi,
>
> As mentioned in another thread on the list, quicktime_open() currently
> uses libquicktime plugins to decode a frame to determine the decoder
> colormodel (stream_cmodel). This occasionally causes error messages
> for files that the ffmpeg plugin tries, but fails, to decode. If the
> essence is decoded outside of libquicktime, this step is unnecessary.
> The attached patch delays the determination of the decoder colormodel
> until it is actually needed.
>
> Kind regards,
> Erik



-- 
Erik Johansson
Barkvägen 10A
SE-541 64 Skövde, Sweden
Mobile: +46 (0)73 707 94 94
LinkedIn | YouTube | Google+ | Facebook

[Libquicktime-devel] Delay determination of decoder colormodel

[Erik J. <eri...@gm...>]

Hi,

As mentioned in another thread on the list, quicktime_open() currently
uses libquicktime plugins to decode a frame to determine the decoder
colormodel (stream_cmodel). This occasionally causes error messages
for files that the ffmpeg plugin tries, but fails, to decode. If the
essence is decoded outside of libquicktime, this step is unnecessary.
The attached patch delays the determination of the decoder colormodel
until it is actually needed.

Kind regards,
Erik

Re: [Libquicktime-devel] FFmpeg plugin resync DNW for out-of-order B-frames after full sync keyframes

[Burkhard P. <pl...@ig...>]

Hi,

Am 24.04.2014 18:21, schrieb Derek Chow:
> Apologies, the problem still happens with my latest
> resync_bframes_after_full_sync_frames3.patch - I want to change the
> line:
>
> int skip_initial_bframes = sync_disp_frame <= target_display_frame;
>
> to
>
> int skip_initial_bframes = sync_disp_frame < target_display_frame;
>
> but will need to investigate a suspicious side-effect I'm having with
> my test file...

By the way I've also checked in a new possible variant of the resync() function
in the ffmpeg decoder.

There are 2 new variables in the core:

- vtrack->track->idx_pos:     Index position of the next packet to be read
- vtrack->next_display_frame: Index position of the next frame to be displayed

next_display_frame is updated by lqt_packet_index_get_next_display_frame() in the core,
which simply looks +/- 32 packets relative to the last frame and takes the one with the closest
PTS. It's the most brain-dead variant but it should work for all kinds of weird reordering
schemes as long as there are less than 32 consecutive B-Frames.
In the ffmpeg plugin, this makes the decoding_delay variable unnecessary.

Seeking now works like:

quicktime_set_video_position() seeks to a frame in *decoding* order, it sets both
vtrack->next_display_frame and vtrack->trak->idx_pos to the keyframe before the target
frame, and vtrack->timestamp to the target PTS.

The resync() function in the ffmpeg plugin reads packets until we reach the target PTS.
Obsolete B-frames and non-reference B-Frames are simply skipped, i.e. they aren't even sent
to libavcodec. I started testing this a bit with H.264 streams and found no obvious problems
so far.

For seeking to a frame in *display* order, use lqt_seek_video(), which takes a PTS as argument
and thus also works for variable framerate streams. It looks in the index for the target frame
and calls quicktime_set_video_position().

Burkhard

Re: [Libquicktime-devel] FFmpeg plugin resync DNW for out-of-order B-frames after full sync keyframes

[Derek C. <der...@mi...>]

Apologies, the problem still happens with my latest
resync_bframes_after_full_sync_frames3.patch - I want to change the
line:

int skip_initial_bframes = sync_disp_frame <= target_display_frame;

to

int skip_initial_bframes = sync_disp_frame < target_display_frame;

but will need to investigate a suspicious side-effect I'm having with
my test file...


On 24 April 2014 10:57, Derek Chow <der...@mi...> wrote:
> I've revised and tidied up the initial patch to fix bad logic where it
> only skips one B-frame when it should be able to skip more than one
> until the next non-B-frame.
>
> I think this thread got lost amidst Burkhard's packet index work but
> it is still a valid problem regardless of how we 'retrieve the
> previous ref sample', 'check the PTS of a sample', 'check if sample is
> B-frame'.  Basically, if you have a packet sequence like:
>
>        pos 1155584 pts 0 dts 0 dur 100 sz 56956 FT I key ref
>        pos 1212540 pts -200 dts 100 dur 100 sz 1095 FT B
>        pos 1213635 pts -100 dts 200 dur 100 sz 35253 FT B
>
> and try to resync to the 3rd frame (PTS -100), the next frame to be
> decoded will end up being the 2nd frame (PTS -200).
>
> Once we are happy with the packet index generation, we can go ahead
> with simplifying the *bitstream*() function code to use it.  -- Side
> note, I've just come across a file that generates "? key" packet index
> entries so for now I've had to use the current demux code - I'll debug
> and update on the "New Demultiplexing mode" thread later.
>
> On 5 March 2014 09:38, Derek Chow <der...@mi...> wrote:
>> (sorry I've switch email client and forgot to send as plain text -
>> original message below):
>>
>> Example with XDCAM file:
>>
>> (IBBP... bitstream with 'ctts' 2, -1, -1, 2 ...).
>> After seeking (or decoding) to frame 2, use
>> quicktime_set_video_position() to then seek backwards to frame 1.
>> get_bitstream_sync_frame() returns 0.
>> decoded_frames_to_discard becomes < 0 despite "// Should not be < 0
>> unless the stream is broken."
>>
>> Please see possible fix attached.
>>
>> On 4 March 2014 10:10, Derek Chow <der...@mi...> wrote:
>>> Example with XDCAM file:
>>>
>>> (IBBP... bitstream with 'ctts' 2, -1, -1, 2 ...).
>>> After seeking (or decoding) to frame 2, use quicktime_set_video_position()
>>> to then seek backwards to frame 1.
>>> get_bitstream_sync_frame() returns 0.
>>> decoded_frames_to_discard becomes < 0 despite "// Should not be < 0 unless
>>> the stream is broken."
>>>
>>> Please see possible fix attached.
>>>
>>>
>>> Derek Chow / C++ Developer
>>>
>>> der...@mi... / +44(0)7825373230
>>>
>>> Office +44 (0)207 884 2530
>>> 21 Broadwall, London, SE1 9PL, United Kingdom
>>> http://www.mirriad.com
>>
>>
>>
>> --
>>
>>
>> Derek Chow / C++ Developer
>>
>> der...@mi... / +44(0)7825373230
>>
>> Office +44 (0)207 884 2530
>> 21 Broadwall, London, SE1 9PL, United Kingdom
>> http://www.mirriad.com
>
>
>
> --
>
>
> Derek Chow / C++ Developer
>
> der...@mi... / +44(0)7825373230
>
> Office +44 (0)207 884 2530
> 21 Broadwall, London, SE1 9PL, United Kingdom
> http://www.mirriad.com



-- 


Derek Chow / C++ Developer

der...@mi... / +44(0)7825373230

Office +44 (0)207 884 2530
21 Broadwall, London, SE1 9PL, United Kingdom
http://www.mirriad.com

Re: [Libquicktime-devel] FFmpeg plugin resync DNW for out-of-order B-frames after full sync keyframes

[Derek C. <der...@mi...>]

I've revised and tidied up the initial patch to fix bad logic where it
only skips one B-frame when it should be able to skip more than one
until the next non-B-frame.

I think this thread got lost amidst Burkhard's packet index work but
it is still a valid problem regardless of how we 'retrieve the
previous ref sample', 'check the PTS of a sample', 'check if sample is
B-frame'.  Basically, if you have a packet sequence like:

       pos 1155584 pts 0 dts 0 dur 100 sz 56956 FT I key ref
       pos 1212540 pts -200 dts 100 dur 100 sz 1095 FT B
       pos 1213635 pts -100 dts 200 dur 100 sz 35253 FT B

and try to resync to the 3rd frame (PTS -100), the next frame to be
decoded will end up being the 2nd frame (PTS -200).

Once we are happy with the packet index generation, we can go ahead
with simplifying the *bitstream*() function code to use it.  -- Side
note, I've just come across a file that generates "? key" packet index
entries so for now I've had to use the current demux code - I'll debug
and update on the "New Demultiplexing mode" thread later.

On 5 March 2014 09:38, Derek Chow <der...@mi...> wrote:
> (sorry I've switch email client and forgot to send as plain text -
> original message below):
>
> Example with XDCAM file:
>
> (IBBP... bitstream with 'ctts' 2, -1, -1, 2 ...).
> After seeking (or decoding) to frame 2, use
> quicktime_set_video_position() to then seek backwards to frame 1.
> get_bitstream_sync_frame() returns 0.
> decoded_frames_to_discard becomes < 0 despite "// Should not be < 0
> unless the stream is broken."
>
> Please see possible fix attached.
>
> On 4 March 2014 10:10, Derek Chow <der...@mi...> wrote:
>> Example with XDCAM file:
>>
>> (IBBP... bitstream with 'ctts' 2, -1, -1, 2 ...).
>> After seeking (or decoding) to frame 2, use quicktime_set_video_position()
>> to then seek backwards to frame 1.
>> get_bitstream_sync_frame() returns 0.
>> decoded_frames_to_discard becomes < 0 despite "// Should not be < 0 unless
>> the stream is broken."
>>
>> Please see possible fix attached.
>>
>>
>> Derek Chow / C++ Developer
>>
>> der...@mi... / +44(0)7825373230
>>
>> Office +44 (0)207 884 2530
>> 21 Broadwall, London, SE1 9PL, United Kingdom
>> http://www.mirriad.com
>
>
>
> --
>
>
> Derek Chow / C++ Developer
>
> der...@mi... / +44(0)7825373230
>
> Office +44 (0)207 884 2530
> 21 Broadwall, London, SE1 9PL, United Kingdom
> http://www.mirriad.com



-- 


Derek Chow / C++ Developer

der...@mi... / +44(0)7825373230

Office +44 (0)207 884 2530
21 Broadwall, London, SE1 9PL, United Kingdom
http://www.mirriad.com

Re: [Libquicktime-devel] New Demultiplexing mode

[Derek C. <der...@mi...>]

>
> Thanks a lot, I have it now. From what I see, it looks quite reasonable.

Yeah the flags seem to be coming out properly - it will fix and obsolete
the buggy 'bframe_detector' code being used in ffmpeg at the moment.


On 17 April 2014 16:29, Burkhard Plaum <pl...@ig...> wrote:

> Hi,
>
> Am 14.04.2014 17:54, schrieb Derek Chow:
> > Oops, apologies!
> >
> > Uploaded newer dump files from a latest CVS build to WeTransfer:
> >
> > Files (12.5 MB total)
> > wobbly_gop_qtdumps.tar.gz
> > Available until
> > 21 April, 2014
> > Download link
> > http://we.tl/NPGyseGIPT
>
> Thanks a lot, I have it now. From what I see, it looks quite reasonable.
>
> I continue to hack on this. Yesterday I converted lqt_read_video_packet()
> to use the packet index. I remultiplexed a file a and the result played
> back perfectly.
>
> I noticed, that there is lots of code to change, so I'll need some time
> for it.
> But I guess the result will be smaller, cleaner and easier to maintain, so
> it's
> definitely worth it.
>
> Burkhard
>
>
>
> ------------------------------------------------------------------------------
> Learn Graph Databases - Download FREE O'Reilly Book
> "Graph Databases" is the definitive new guide to graph databases and their
> applications. Written by three acclaimed leaders in the field,
> this first edition is now available. Download your free book today!
> http://p.sf.net/sfu/NeoTech
> _______________________________________________
> Libquicktime-devel mailing list
> Lib...@li...
> https://lists.sourceforge.net/lists/listinfo/libquicktime-devel
>



-- 



Derek Chow / C++ Developer

der...@mi... / +44(0)7825373230

Office +44 (0)207 884 2530
21 Broadwall, London, SE1 9PL, United Kingdom
http://www.mirriad.com

<http://www.twitter.com/mirriad>   <http://www.linkedin.com/company/mirriad>
   <http://www.facebook.com/MirriAd>

Re: [Libquicktime-devel] New Demultiplexing mode

[Burkhard P. <pl...@ig...>]

Hi,

Am 14.04.2014 17:54, schrieb Derek Chow:
> Oops, apologies!
>
> Uploaded newer dump files from a latest CVS build to WeTransfer:
>
> Files (12.5 MB total)
> wobbly_gop_qtdumps.tar.gz
> Available until
> 21 April, 2014
> Download link
> http://we.tl/NPGyseGIPT

Thanks a lot, I have it now. From what I see, it looks quite reasonable.

I continue to hack on this. Yesterday I converted lqt_read_video_packet()
to use the packet index. I remultiplexed a file a and the result played
back perfectly.

I noticed, that there is lots of code to change, so I'll need some time for it.
But I guess the result will be smaller, cleaner and easier to maintain, so it's
definitely worth it.

Burkhard

Re: [Libquicktime-devel] New Demultiplexing mode

[Derek C. <der...@mi...>]

Oops, apologies!

Uploaded newer dump files from a latest CVS build to WeTransfer:

Files (12.5 MB total)
wobbly_gop_qtdumps.tar.gz
Available until
21 April, 2014
Download link
http://we.tl/NPGyseGIPT

If you need the URL for longer I can put it on Dropbox.

On 11 April 2014 07:45, Burkhard Plaum <pl...@ig...> wrote:
> Hi,
>
> Am 10.04.2014 11:32, schrieb Derek Chow:
>> forgot to point out - one of them has a stps atom ;)
>
> Did you upgrade to libquicktime the latest CVS before? Because I
> don't see the packet index.
>
> Also, can you please upload large dumps somewhere on a webspace or
> sharehoster? 6 MB is a bit too large for a mailing list.
>
> Burkhard
>
>
> ------------------------------------------------------------------------------
> Put Bad Developers to Shame
> Dominate Development with Jenkins Continuous Integration
> Continuously Automate Build, Test & Deployment
> Start a new project now. Try Jenkins in the cloud.
> http://p.sf.net/sfu/13600_Cloudbees
> _______________________________________________
> Libquicktime-devel mailing list
> Lib...@li...
> https://lists.sourceforge.net/lists/listinfo/libquicktime-devel



-- 


Derek Chow / C++ Developer

der...@mi... / +44(0)7825373230

Office +44 (0)207 884 2530
21 Broadwall, London, SE1 9PL, United Kingdom
http://www.mirriad.com

Re: [Libquicktime-devel] New Demultiplexing mode

[Burkhard P. <pl...@ig...>]

Erik Johansson wrote
> Hi,
>
>> Of course we can think of exporting the index information somehow in the
>> public API.
>
> That would be nice.
>
>>> While on the topic of indexes... I've lately come across a few files
>>> from different sources that makes use of multiple entries in the stsd
>>> sample descriptions. For these files, it is also necessary to keep
>>> track of the 'description' entry in the stsc table. I guess that will
>>> require one more filed in the lqt_packet_index_entry_t struct,
>>
>> That can be added.
>>
>>> among
>>> other changes.
>>
>> And these would be a LOT. In the worst case it means to change *all*
>> stream
>> parameters (image size, codec, colormodel....) in the middle of the
>> file.
>
> Indeed. Would it make sense to add a 'raw' reading mode to the
> library? For most files, we use libquicktime to demux and extract
> essence, which is then decoded using various SDKs.

We already have lqt_read_audio_packet() and lqt_read_video_packet(),
but these are only supported for codecs which are well defined also
outside of quicktime files.

On the other hand, if the packet index is available in the public API,
the raw "reading" simply means fseek() and fread(), which can be done
without the help of the library.

> Two loose thoughts
> / observations;
> - The required changes to support multiple stsd entries would be more
> limited if reading in raw mode

Exactly. The only thing we need to do is to export all stsd entries.

> - Currently quicktime_open() tries to decode essence using the plugins
> to determine colormodels, which is unnecessary for the raw reading
> scenario (and sometimes produces error messages).

That could be avoided by obtaining the stream colormodel only when the user
asks for it (i.e. triggered by a call to lqt_get_decoder_colormodel()).

Burkhard

Re: [Libquicktime-devel] New Demultiplexing mode

[Erik J. <eri...@gm...>]

Hi,

> Of course we can think of exporting the index information somehow in the
> public API.

That would be nice.

>> While on the topic of indexes... I've lately come across a few files
>> from different sources that makes use of multiple entries in the stsd
>> sample descriptions. For these files, it is also necessary to keep
>> track of the 'description' entry in the stsc table. I guess that will
>> require one more filed in the lqt_packet_index_entry_t struct,
>
> That can be added.
>
>> among
>> other changes.
>
> And these would be a LOT. In the worst case it means to change *all* stream
> parameters (image size, codec, colormodel....) in the middle of the file.

Indeed. Would it make sense to add a 'raw' reading mode to the
library? For most files, we use libquicktime to demux and extract
essence, which is then decoded using various SDKs. Two loose thoughts
/ observations;
- The required changes to support multiple stsd entries would be more
limited if reading in raw mode
- Currently quicktime_open() tries to decode essence using the plugins
to determine colormodels, which is unnecessary for the raw reading
scenario (and sometimes produces error messages).

// Erik

-- 
Erik Johansson

Re: [Libquicktime-devel] New Demultiplexing mode

[Burkhard P. <pl...@ig...>]

Hi,

Am 10.04.2014 11:32, schrieb Derek Chow:
> forgot to point out - one of them has a stps atom ;)

Did you upgrade to libquicktime the latest CVS before? Because I
don't see the packet index.

Also, can you please upload large dumps somewhere on a webspace or
sharehoster? 6 MB is a bit too large for a mailing list.

Burkhard

Re: [Libquicktime-devel] New Demultiplexing mode

[Derek C. <der...@mi...>]

forgot to point out - one of them has a stps atom ;)

On 10 April 2014 10:30, Derek Chow <der...@mi...> wrote:
> Sorry for the wait, attached are some qtdump outputs for movies with
> unusual GOP patterns.  Hope they help.
>
> On 10 April 2014 09:44, Burkhard Plaum <pl...@ig...> wrote:
>> Hi,
>>
>> Am 09.04.2014 21:46, schrieb Erik Johansson:
>>> Hi Burkhard,
>>>
>>>> The solution is to unpack the indices (stts, ctts, stco, stsc,
>>>> stss and stps) and make one single index table from them.
>>>
>>> Incidentally, I was writing code to create such an index outside of
>>> libquicktime just the other day. Nice to have support for it in the
>>> library! :-)
>>
>> Of course we can think of exporting the index information somehow in the
>> public API.
>>
>>>> In the meantime you can help a bit by calling qtdump on your files (ideally
>>>> with really weird GOP patterns) and check if the generated index makes sense.
>>>
>>> I've given it a spin across a selection of the more convoluted files I
>>> have at hand. It holds up just fine so far.
>>
>> Good. I also tested it with H.264 streams with B-Pyramids. As far as I see,
>> the reference B-Frames are correctly detected also.
>>
>> What I don't have at hand are files with stps atoms (partial sync sample).
>>
>>> While on the topic of indexes... I've lately come across a few files
>>> from different sources that makes use of multiple entries in the stsd
>>> sample descriptions. For these files, it is also necessary to keep
>>> track of the 'description' entry in the stsc table. I guess that will
>>> require one more filed in the lqt_packet_index_entry_t struct,
>>
>> That can be added.
>>
>>> among
>>> other changes.
>>
>> And these would be a LOT. In the worst case it means to change *all* stream
>> parameters (image size, codec, colormodel....) in the middle of the file.
>>
>>> Kind regards,
>>> // Erik
>>
>> Burkhard
>>
>> ------------------------------------------------------------------------------
>> Put Bad Developers to Shame
>> Dominate Development with Jenkins Continuous Integration
>> Continuously Automate Build, Test & Deployment
>> Start a new project now. Try Jenkins in the cloud.
>> http://p.sf.net/sfu/13600_Cloudbees
>> _______________________________________________
>> Libquicktime-devel mailing list
>> Lib...@li...
>> https://lists.sourceforge.net/lists/listinfo/libquicktime-devel
>
>
>
> --
>
>
> Derek Chow / C++ Developer
>
> der...@mi... / +44(0)7825373230
>
> Office +44 (0)207 884 2530
> 21 Broadwall, London, SE1 9PL, United Kingdom
> http://www.mirriad.com



-- 


Derek Chow / C++ Developer

der...@mi... / +44(0)7825373230

Office +44 (0)207 884 2530
21 Broadwall, London, SE1 9PL, United Kingdom
http://www.mirriad.com

Re: [Libquicktime-devel] New Demultiplexing mode

[Burkhard P. <pl...@ig...>]

Hi,

Am 09.04.2014 21:46, schrieb Erik Johansson:
> Hi Burkhard,
>
>> The solution is to unpack the indices (stts, ctts, stco, stsc,
>> stss and stps) and make one single index table from them.
>
> Incidentally, I was writing code to create such an index outside of
> libquicktime just the other day. Nice to have support for it in the
> library! :-)

Of course we can think of exporting the index information somehow in the
public API.

>> In the meantime you can help a bit by calling qtdump on your files (ideally
>> with really weird GOP patterns) and check if the generated index makes sense.
>
> I've given it a spin across a selection of the more convoluted files I
> have at hand. It holds up just fine so far.

Good. I also tested it with H.264 streams with B-Pyramids. As far as I see,
the reference B-Frames are correctly detected also.

What I don't have at hand are files with stps atoms (partial sync sample).

> While on the topic of indexes... I've lately come across a few files
> from different sources that makes use of multiple entries in the stsd
> sample descriptions. For these files, it is also necessary to keep
> track of the 'description' entry in the stsc table. I guess that will
> require one more filed in the lqt_packet_index_entry_t struct,

That can be added.

> among
> other changes.

And these would be a LOT. In the worst case it means to change *all* stream
parameters (image size, codec, colormodel....) in the middle of the file.

> Kind regards,
> // Erik

Burkhard