From: Burkhard P. <pl...@ig...> - 2014-04-07 07:15:14
|
Hi, I thought a bit how to clean up the current mess in the ffmpeg video decoder and I think I came up with a universal solution. The problem is the index tables in quicktime files: They have the advantage that they have implicit compression, which makes them quite small for simple files (e.g. constant framerate, I-frame only). The disadvantage is, that if we use them while demultiplexing, the code becomes like it is now :( The solution is to unpack the indices (stts, ctts, stco, stsc, stss and stps) and make one single index table from them. A separate routine generates the same index from the idx1 or indx chunks of AVI files. The entry for one video frame looks like (qtprivate.h): typedef struct { int64_t position; // Position from the file start int64_t size; // Size int64_t pts; // Presentation time stamp int64_t dts; // Decoding time stamp int64_t duration; // Duration of this packet int flags; // Flags (like in lqt_packet_t) } lqt_packet_index_entry_t; Using qtdump you'll see something like: Packet index (generated) num_entries 35188 pos 374851 pts 0 dts 0 dur 2002 sz 14272 FT I key ref pos 389123 pts 2002 dts 2002 dur 2002 sz 12415 FT P ref pos 401538 pts 4004 dts 4004 dur 2002 sz 18332 FT P ref pos 419870 pts 6006 dts 6006 dur 2002 sz 1336 FT P ref pos 421206 pts 8008 dts 8008 dur 2002 sz 16041 FT P ref pos 437247 pts 10010 dts 10010 dur 2002 sz 20196 FT P ref pos 457443 pts 12012 dts 12012 dur 2002 sz 312 FT P ref .... This reduces "demultiplexing" basically to calls to fseek()/fread() and taking the packet flags from the index. In the current CVS version the packet index is already generated for video streams, but not used yet for demultiplexing. In the meantime you can help a bit by calling qtdump on your files (ideally with really weird GOP patterns) and check if the generated index makes sense. Burkhard |
From: Erik J. <eri...@gm...> - 2014-04-09 19:47:13
|
Hi Burkhard, > The solution is to unpack the indices (stts, ctts, stco, stsc, > stss and stps) and make one single index table from them. Incidentally, I was writing code to create such an index outside of libquicktime just the other day. Nice to have support for it in the library! :-) > In the meantime you can help a bit by calling qtdump on your files (ideally > with really weird GOP patterns) and check if the generated index makes sense. I've given it a spin across a selection of the more convoluted files I have at hand. It holds up just fine so far. While on the topic of indexes... I've lately come across a few files from different sources that makes use of multiple entries in the stsd sample descriptions. For these files, it is also necessary to keep track of the 'description' entry in the stsc table. I guess that will require one more filed in the lqt_packet_index_entry_t struct, among other changes. Kind regards, // Erik |
From: Burkhard P. <pl...@ig...> - 2014-04-10 08:44:14
|
Hi, Am 09.04.2014 21:46, schrieb Erik Johansson: > Hi Burkhard, > >> The solution is to unpack the indices (stts, ctts, stco, stsc, >> stss and stps) and make one single index table from them. > > Incidentally, I was writing code to create such an index outside of > libquicktime just the other day. Nice to have support for it in the > library! :-) Of course we can think of exporting the index information somehow in the public API. >> In the meantime you can help a bit by calling qtdump on your files (ideally >> with really weird GOP patterns) and check if the generated index makes sense. > > I've given it a spin across a selection of the more convoluted files I > have at hand. It holds up just fine so far. Good. I also tested it with H.264 streams with B-Pyramids. As far as I see, the reference B-Frames are correctly detected also. What I don't have at hand are files with stps atoms (partial sync sample). > While on the topic of indexes... I've lately come across a few files > from different sources that makes use of multiple entries in the stsd > sample descriptions. For these files, it is also necessary to keep > track of the 'description' entry in the stsc table. I guess that will > require one more filed in the lqt_packet_index_entry_t struct, That can be added. > among > other changes. And these would be a LOT. In the worst case it means to change *all* stream parameters (image size, codec, colormodel....) in the middle of the file. > Kind regards, > // Erik Burkhard |
From: Derek C. <der...@mi...> - 2014-04-10 10:01:09
|
forgot to point out - one of them has a stps atom ;) On 10 April 2014 10:30, Derek Chow <der...@mi...> wrote: > Sorry for the wait, attached are some qtdump outputs for movies with > unusual GOP patterns. Hope they help. > > On 10 April 2014 09:44, Burkhard Plaum <pl...@ig...> wrote: >> Hi, >> >> Am 09.04.2014 21:46, schrieb Erik Johansson: >>> Hi Burkhard, >>> >>>> The solution is to unpack the indices (stts, ctts, stco, stsc, >>>> stss and stps) and make one single index table from them. >>> >>> Incidentally, I was writing code to create such an index outside of >>> libquicktime just the other day. Nice to have support for it in the >>> library! :-) >> >> Of course we can think of exporting the index information somehow in the >> public API. >> >>>> In the meantime you can help a bit by calling qtdump on your files (ideally >>>> with really weird GOP patterns) and check if the generated index makes sense. >>> >>> I've given it a spin across a selection of the more convoluted files I >>> have at hand. It holds up just fine so far. >> >> Good. I also tested it with H.264 streams with B-Pyramids. As far as I see, >> the reference B-Frames are correctly detected also. >> >> What I don't have at hand are files with stps atoms (partial sync sample). >> >>> While on the topic of indexes... I've lately come across a few files >>> from different sources that makes use of multiple entries in the stsd >>> sample descriptions. For these files, it is also necessary to keep >>> track of the 'description' entry in the stsc table. I guess that will >>> require one more filed in the lqt_packet_index_entry_t struct, >> >> That can be added. >> >>> among >>> other changes. >> >> And these would be a LOT. In the worst case it means to change *all* stream >> parameters (image size, codec, colormodel....) in the middle of the file. >> >>> Kind regards, >>> // Erik >> >> Burkhard >> >> ------------------------------------------------------------------------------ >> Put Bad Developers to Shame >> Dominate Development with Jenkins Continuous Integration >> Continuously Automate Build, Test & Deployment >> Start a new project now. Try Jenkins in the cloud. >> http://p.sf.net/sfu/13600_Cloudbees >> _______________________________________________ >> Libquicktime-devel mailing list >> Lib...@li... >> https://lists.sourceforge.net/lists/listinfo/libquicktime-devel > > > > -- > > > Derek Chow / C++ Developer > > der...@mi... / +44(0)7825373230 > > Office +44 (0)207 884 2530 > 21 Broadwall, London, SE1 9PL, United Kingdom > http://www.mirriad.com -- Derek Chow / C++ Developer der...@mi... / +44(0)7825373230 Office +44 (0)207 884 2530 21 Broadwall, London, SE1 9PL, United Kingdom http://www.mirriad.com |
From: Burkhard P. <pl...@ig...> - 2014-04-11 06:45:16
|
Hi, Am 10.04.2014 11:32, schrieb Derek Chow: > forgot to point out - one of them has a stps atom ;) Did you upgrade to libquicktime the latest CVS before? Because I don't see the packet index. Also, can you please upload large dumps somewhere on a webspace or sharehoster? 6 MB is a bit too large for a mailing list. Burkhard |
From: Derek C. <der...@mi...> - 2014-04-14 16:26:13
|
Oops, apologies! Uploaded newer dump files from a latest CVS build to WeTransfer: Files (12.5 MB total) wobbly_gop_qtdumps.tar.gz Available until 21 April, 2014 Download link http://we.tl/NPGyseGIPT If you need the URL for longer I can put it on Dropbox. On 11 April 2014 07:45, Burkhard Plaum <pl...@ig...> wrote: > Hi, > > Am 10.04.2014 11:32, schrieb Derek Chow: >> forgot to point out - one of them has a stps atom ;) > > Did you upgrade to libquicktime the latest CVS before? Because I > don't see the packet index. > > Also, can you please upload large dumps somewhere on a webspace or > sharehoster? 6 MB is a bit too large for a mailing list. > > Burkhard > > > ------------------------------------------------------------------------------ > Put Bad Developers to Shame > Dominate Development with Jenkins Continuous Integration > Continuously Automate Build, Test & Deployment > Start a new project now. Try Jenkins in the cloud. > http://p.sf.net/sfu/13600_Cloudbees > _______________________________________________ > Libquicktime-devel mailing list > Lib...@li... > https://lists.sourceforge.net/lists/listinfo/libquicktime-devel -- Derek Chow / C++ Developer der...@mi... / +44(0)7825373230 Office +44 (0)207 884 2530 21 Broadwall, London, SE1 9PL, United Kingdom http://www.mirriad.com |
From: Erik J. <eri...@gm...> - 2014-04-11 07:28:15
|
Hi, > Of course we can think of exporting the index information somehow in the > public API. That would be nice. >> While on the topic of indexes... I've lately come across a few files >> from different sources that makes use of multiple entries in the stsd >> sample descriptions. For these files, it is also necessary to keep >> track of the 'description' entry in the stsc table. I guess that will >> require one more filed in the lqt_packet_index_entry_t struct, > > That can be added. > >> among >> other changes. > > And these would be a LOT. In the worst case it means to change *all* stream > parameters (image size, codec, colormodel....) in the middle of the file. Indeed. Would it make sense to add a 'raw' reading mode to the library? For most files, we use libquicktime to demux and extract essence, which is then decoded using various SDKs. Two loose thoughts / observations; - The required changes to support multiple stsd entries would be more limited if reading in raw mode - Currently quicktime_open() tries to decode essence using the plugins to determine colormodels, which is unnecessary for the raw reading scenario (and sometimes produces error messages). // Erik -- Erik Johansson |
From: Burkhard P. <pl...@ig...> - 2014-04-11 07:54:29
|
Erik Johansson wrote > Hi, > >> Of course we can think of exporting the index information somehow in the >> public API. > > That would be nice. > >>> While on the topic of indexes... I've lately come across a few files >>> from different sources that makes use of multiple entries in the stsd >>> sample descriptions. For these files, it is also necessary to keep >>> track of the 'description' entry in the stsc table. I guess that will >>> require one more filed in the lqt_packet_index_entry_t struct, >> >> That can be added. >> >>> among >>> other changes. >> >> And these would be a LOT. In the worst case it means to change *all* >> stream >> parameters (image size, codec, colormodel....) in the middle of the >> file. > > Indeed. Would it make sense to add a 'raw' reading mode to the > library? For most files, we use libquicktime to demux and extract > essence, which is then decoded using various SDKs. We already have lqt_read_audio_packet() and lqt_read_video_packet(), but these are only supported for codecs which are well defined also outside of quicktime files. On the other hand, if the packet index is available in the public API, the raw "reading" simply means fseek() and fread(), which can be done without the help of the library. > Two loose thoughts > / observations; > - The required changes to support multiple stsd entries would be more > limited if reading in raw mode Exactly. The only thing we need to do is to export all stsd entries. > - Currently quicktime_open() tries to decode essence using the plugins > to determine colormodels, which is unnecessary for the raw reading > scenario (and sometimes produces error messages). That could be avoided by obtaining the stream colormodel only when the user asks for it (i.e. triggered by a call to lqt_get_decoder_colormodel()). Burkhard |
From: Burkhard P. <pl...@ig...> - 2014-04-17 15:29:55
|
Hi, Am 14.04.2014 17:54, schrieb Derek Chow: > Oops, apologies! > > Uploaded newer dump files from a latest CVS build to WeTransfer: > > Files (12.5 MB total) > wobbly_gop_qtdumps.tar.gz > Available until > 21 April, 2014 > Download link > http://we.tl/NPGyseGIPT Thanks a lot, I have it now. From what I see, it looks quite reasonable. I continue to hack on this. Yesterday I converted lqt_read_video_packet() to use the packet index. I remultiplexed a file a and the result played back perfectly. I noticed, that there is lots of code to change, so I'll need some time for it. But I guess the result will be smaller, cleaner and easier to maintain, so it's definitely worth it. Burkhard |
From: Derek C. <der...@mi...> - 2014-04-17 15:56:44
|
> > Thanks a lot, I have it now. From what I see, it looks quite reasonable. Yeah the flags seem to be coming out properly - it will fix and obsolete the buggy 'bframe_detector' code being used in ffmpeg at the moment. On 17 April 2014 16:29, Burkhard Plaum <pl...@ig...> wrote: > Hi, > > Am 14.04.2014 17:54, schrieb Derek Chow: > > Oops, apologies! > > > > Uploaded newer dump files from a latest CVS build to WeTransfer: > > > > Files (12.5 MB total) > > wobbly_gop_qtdumps.tar.gz > > Available until > > 21 April, 2014 > > Download link > > http://we.tl/NPGyseGIPT > > Thanks a lot, I have it now. From what I see, it looks quite reasonable. > > I continue to hack on this. Yesterday I converted lqt_read_video_packet() > to use the packet index. I remultiplexed a file a and the result played > back perfectly. > > I noticed, that there is lots of code to change, so I'll need some time > for it. > But I guess the result will be smaller, cleaner and easier to maintain, so > it's > definitely worth it. > > Burkhard > > > > ------------------------------------------------------------------------------ > Learn Graph Databases - Download FREE O'Reilly Book > "Graph Databases" is the definitive new guide to graph databases and their > applications. Written by three acclaimed leaders in the field, > this first edition is now available. Download your free book today! > http://p.sf.net/sfu/NeoTech > _______________________________________________ > Libquicktime-devel mailing list > Lib...@li... > https://lists.sourceforge.net/lists/listinfo/libquicktime-devel > -- Derek Chow / C++ Developer der...@mi... / +44(0)7825373230 Office +44 (0)207 884 2530 21 Broadwall, London, SE1 9PL, United Kingdom http://www.mirriad.com <http://www.twitter.com/mirriad> <http://www.linkedin.com/company/mirriad> <http://www.facebook.com/MirriAd> |