From: Wim T. <wim...@ch...> - 2002-11-19 18:21:10
|
Hi, > Since the ffmpeg plugin isn't ready yet ... > > in gst_mpeg2dec_convert_sink, i see that the time->byte offset > conversion is done with the byte_rate. This assumes a CBR. However, > most streams are VBR. At least, i only care about VBR. i haven't > found any open-source encoders which aren't VBR (which makes it hard > to create crystal clear VCDs). yes, seeking is currently only done using the bitrate (CBR) > How hard would it be to extract the time->byte offset mapping into an > index file and use that (like cinelerra & mpeg3toc)? It seems like a > matter of: > > 1. Adding another sink pad (and mode?) to mpeg2dec which outputs the > index at maximum parse speed .. then pipe the index to fdsink. > > 2. Add an "index-file" property to mpeg2dec. This file could be > mmap'd in whole .. and that is it. > The idea is to hand a GstTimeCache object (object is already included in the core) to the plugin. The Timecache object could be prefilled with previously save data (XML format?), in which case the plugin would assume that it contains byte->time mappings of a previous indexing round. If the timecache object is empty (or partially filled) the plugin would update if as it receives data. The procedure to fill the cache would go like: a) connect filesrc->mpeg2parse/dec (don't connect any srcpads) b) hand an empty timecache object to mpeg2parse/dec c) run the pipeline to EOS d) read/use/save the timecache The procedure to hand an index to a plugin would be like: a) create/fill timecache from previously saved data b) create playback pipeline c) hand index to plugin (mpeg2dec) d) run/seek etc.. If no timecache is handed to mpeg2dec, it would simply not do any indexing. The design for the timecache is however not done yet. If you want to give it a go, this is what I think the timecahce API should (at least) contain: - API to to mapping from byte<->timeoffset - API to get individual index records. - API to get info about a record (is it a keyframe (I frame,... ), ..) - API to load/save timecache (XML?, customizable?) nice to have: - mapping to other (abritrary) formats (samples, frames, ...) - metadata (who created the index, what is the corresponding URI of the master media file, timestamp, ...) - API to specifiy the type of records to index (only keyframes, every N seconds, max size of index, ringbuffer, ...) - certainty of index (for indexes created after a seek, ...) - merging of index entries if certainty is known (playback reaches previously seeked and indexed position from region with higher certainty) - indexing of time<->metadata (or other arbitrary properties) > It sounds easy enough that i'd be willing to code something up. > > Is this design the gstreamer way of doing things? What would be the > _perfect_ way to support this type of index file option? Can we just > hack it temporarily? I would not mind your hacking on GstTimecache to implement the bare minimum for your requirements, I'm sure it'll be time well spent. A standard Timecache potentially allows apps like an NLE to work more efficiently too. > Let's assume that all frames are I-frames. What the best byte-offset > for a given frame? Where in mpeg2dec is the "start of frame" > identified? Is this the correct byte-offset to use for seeking? libmpeg2 CVS allows you to get notification when an I frame is starting, I'm not sure how many bytes of the frame it has already consumed by then or if you can even query that. For the plain indexing round, I would just count the number of bytes handed to mpeg2dec, guestimate the start code of the picture and store that time<->offset pair in the cache. I'm also thinking that the indexing should actually happen both in the mpeg demuxer and the mpeg video decoder. The video decoder would map frames (I frames) to PTS timestamps, the mpeg demuxer would index byte offsets to the PTS values of the different streams, it would probably also index SCR timestamps to offsets (need to set timecaches on pads? need to store the pad id in the cache entries too?..). Just a few issues I'm thinking of here... The seeking event on the mpeg video decoder would then first figure out what the nearest I frame is, it would convert the timestamp (or frame number) to the PTS. It would then do a timeseek on its sinkpad with the PTS value. Mpegdemux would get the seek on the PTS, it would map this to the byteoffset of the videopacket with that PTS and would then forward the byteseek to filesrc. Wim > Everything else is tested & ready. i just need accurate seek for VBR. |
From: Joshua N P. <vi...@po...> - 2002-11-20 12:17:28
|
On Tue, Nov 19, 2002 at 07:24:44PM +0100, Wim Taymans wrote: > > 1. Adding another sink pad (and mode?) to mpeg2dec which outputs the > > index at maximum parse speed .. then pipe the index to fdsink. > > > > 2. Add an "index-file" property to mpeg2dec. This file could be > > mmap'd in whole .. and that is it. i gonna read your whole reply, but before i forget, thomasvs suggested that i can put the whole time->byte mapping in my app (not in gstreamer). i just need to make mpeg2dec run in batch mode to extract the mapping to an index file. Then i can easily manage things within my app until the TimeCache stuff is mature enough. Gotta go now .. i'll comment on the other stuff soon. -- Victory to the Divine Mother!! after all, http://sahajayoga.org http://why-compete.org |
From: Thomas V. S. <th...@ur...> - 2002-11-20 12:52:54
|
Hi, > On Tue, Nov 19, 2002 at 07:24:44PM +0100, Wim Taymans wrote: > > > 1. Adding another sink pad (and mode?) to mpeg2dec which outputs the > > > index at maximum parse speed .. then pipe the index to fdsink. > > > > > > 2. Add an "index-file" property to mpeg2dec. This file could be > > > mmap'd in whole .. and that is it. > > i gonna read your whole reply, but before i forget, thomasvs suggested > that i can put the whole time->byte mapping in my app (not in gstreamer). Yeah, but I only suggested that because I got the feeling you weren't into the idea of writing/fixing TimeCache. If you were sitting next to me I'd tie you to a chair and make you do the right thing ;) But yeah, if you want the quick hack, doing it in the app will be easiest for your purposes. From what I read from wtay's reply, it seems I got the other mechanics as we discussed on IRC about right. So, if you have any interest at all in taking Wim's suggestions to code, please do, it would benefit us all. Thomas -- The Dave/Dina Project : future TV today ! - http://davedina.apestaart.org/ <-*- thomas (dot) apestaart (dot) org -*-> P.S. You rock my world <-*- thomas (at) apestaart (dot) org -*-> URGent, the best radio on the Internet - 24/7 ! - http://urgent.rug.ac.be/ |
From: Joshua N P. <vi...@po...> - 2002-11-20 15:21:35
|
On Tue, Nov 19, 2002 at 07:24:44PM +0100, Wim Taymans wrote: > > How hard would it be to extract the time->byte offset mapping into an > > index file and use that (like cinelerra & mpeg3toc)? It seems like a > > matter of: > > > > 1. Adding another sink pad (and mode?) to mpeg2dec which outputs the > > index at maximum parse speed .. then pipe the index to fdsink. > > > > 2. Add an "index-file" property to mpeg2dec. This file could be > > mmap'd in whole .. and that is it. > > The idea is to hand a GstTimeCache object (object is already included in > the core) > to the plugin. The Timecache object could be prefilled with previously > save data (XML format?), XML seems a little bulky for (byte,time) tuples. i guess it depends what you are optimizing for. > in which case the plugin would assume that it > contains byte->time mappings of a previous indexing round. > > If the timecache object is empty (or partially filled) the plugin would > update if as it receives data. Yah, i'm sure some apps will work like that. However, my app needs to index the whole media stream before playback. > The procedure to fill the cache would go like: > > a) connect filesrc->mpeg2parse/dec (don't connect any srcpads) > b) hand an empty timecache object to mpeg2parse/dec > c) run the pipeline to EOS > d) read/use/save the timecache Yah, i will try to implement that ASAP. > The procedure to hand an index to a plugin would be like: > > a) create/fill timecache from previously saved data > b) create playback pipeline > c) hand index to plugin (mpeg2dec) > d) run/seek etc.. > > If no timecache is handed to mpeg2dec, it would simply not do any > indexing. Yah, i don't need incremental indexing. > The design for the timecache is however not done yet. If you want to > give it a go, What i will try to do is provide a sample implementation of the batch mode indexing. > this is what I think the timecahce API should (at least) > contain: > > - API to to mapping from byte<->timeoffset > - API to get individual index records. > - API to get info about a record (is it a keyframe (I frame,... ), ..) > - API to load/save timecache (XML?, customizable?) > > nice to have: > > - mapping to other (abritrary) formats (samples, frames, ...) > - metadata (who created the index, what is the corresponding URI > of the master media file, timestamp, ...) > - API to specifiy the type of records to index (only keyframes, every > N seconds, max size of index, ringbuffer, ...) > - certainty of index (for indexes created after a seek, ...) > - merging of index entries if certainty is known (playback reaches > previously seeked and indexed position from region with higher > certainty) > - indexing of time<->metadata (or other arbitrary properties) Yah, whatever. Let apps drive the API design. > > Let's assume that all frames are I-frames. What the best byte-offset > > for a given frame? Where in mpeg2dec is the "start of frame" > > identified? Is this the correct byte-offset to use for seeking? > > libmpeg2 CVS allows you to get notification when an I frame is starting, > > I'm not sure how many bytes of the frame it has already consumed by then > or if you can even query that. For the plain indexing round, I would > just count the number of bytes handed to mpeg2dec, guestimate the start > code of the picture and store that time<->offset pair in the cache. Yah, that will work well enough given that the current seeking just throws random byte offsets into mpeg2dec. > I'm also thinking that the indexing should actually happen both in the > mpeg demuxer and the mpeg video decoder. The video decoder would map > ... > of the videopacket with that PTS and would then forward the byteseek to > filesrc. That's too much work for me. i need a quick solution! -- Victory to the Divine Mother!! after all, http://sahajayoga.org http://why-compete.org |
From: Joshua N P. <vi...@po...> - 2002-11-22 18:57:28
Attachments:
mkindex.c
|
On Wed, Nov 20, 2002 at 08:51:15PM +0530, Joshua N Pritikin wrote: > > a) connect filesrc->mpeg2parse/dec (don't connect any srcpads) > > b) hand an empty timecache object to mpeg2parse/dec > > c) run the pipeline to EOS > > d) read/use/save the timecache > > Yah, i will try to implement that ASAP. Here's my first attempt. There are some problems. * At least on my system, gnomevfssrc leaks memory (fast) while filesrc doesn't. Maybe i'm a few versions behind. Can someone compare the memory profile of these two? gst-launch gnomevfssrc location=/some-film.mpg ! mpegdemux gst-launch filesrc location=/some-film.mpg ! mpegdemux This probably explains the libgstplay leak i was seeing. i'll confirm later. * Should i be using mpegdemux or mpegparse? * How do i get the correct clock? What gst_bin_get_clock returns is a clock for the elapse time of the indexer. In other words, if i scan the whole film in 54 seconds then the clock finished at 54 seconds. Useless. What i need is the film's idea of the elapse time. The clock should finish at 1 hour (or the duration of the film) even if i can scan the whole film in 1 minute. i see some suspicious code in mpegparse: static GstClock* gst_mpeg_parse_get_clock (GstElement *element) { //GstMPEGParse *parse = GST_MPEG_PARSE (element); //return parse->provided_clock; return NULL; } And it seems like the only way to call gst_mpeg_parse_get_time is via the provided_clock. Is the SCR what i want? What about correcting for discontinuities? How about just counting the video frames and making the frame-rate and frame count available as a property or query? -- Victory to the Divine Mother!! after all, http://sahajayoga.org http://why-compete.org |
From: Joshua N P. <vi...@po...> - 2002-11-23 07:30:46
Attachments:
mkindex.c
|
On Sat, Nov 23, 2002 at 12:27:17AM +0530, Joshua N Pritikin wrote: > * How do i get the correct clock? What gst_bin_get_clock returns > is a clock for the elapse time of the indexer. OK, now i am using "filesrc ! mpegdemux ! mpeg2dec" and it seems to work. AlienSong seems OK, at least. New code is attached. The indexing seems a little too slow, especially for a DVD vob file. Is there more that can be disabled in mpeg2dec when the src pad isn't connected? i am not familiar enough with the code to try skipping the image decoding step. i'm getting a SEGV after indexing 10 minutes of an mpeg1 film. What more diagnostics can i provide? (gdb) where #0 gst_mpeg_parse_parse_packhead (mpeg_parse=0x80da0e0, buffer=0x37ccd80) at gstmpegparse.c:316 #1 0x40835459 in gst_mpeg_demux_parse_packhead (mpeg_parse=0x80da0e0, buffer=0x8085e48) at gstmpegdemux.c:277 #2 0x40834509 in gst_mpeg_parse_loop (element=0x80da0e0) at gstmpegparse.c:380 #3 0x408152c9 in gst_basic_scheduler_loopfunc_wrapper (argc=0, argv=0x80da0e0) at gstbasicscheduler.c:279 #4 0x4081999c in cothread_stub () at cothreads.c:444 #5 0x40819d48 in cothread_switch (thread=0xbfe40000) at cothreads.c:650 #6 0x408186bd in gst_basic_scheduler_iterate (sched=0x80bd528) at gstbasicscheduler.c:1342 #7 0x4005075f in gst_scheduler_iterate (sched=0x80bd528) at gstscheduler.c:645 (gdb) list 311 scr |= (scr2 & 0xfe000000) >> 25; 312 313 mpeg_parse->bytes_since_scr = 0; 314 315 buf += 5; 316 new_rate = (GUINT32_FROM_BE ((*(guint32 *) buf)) & 0x7ffffe00) >> 9; 317 } 318 319 scr_adj = scr + mpeg_parse->adjust; 320 (gdb) p buf $2 = (guint8 *) 0x416d6ff8 "!\r�\233�\200\e\221" <Address 0x416d7000 out of bounds> i guess i'll try updating and recompiling everything. -- Victory to the Divine Mother!! after all, http://sahajayoga.org http://why-compete.org |
From: Joshua N P. <vi...@po...> - 2002-11-23 07:35:59
|
On Sat, Nov 23, 2002 at 01:00:21PM +0530, Joshua N Pritikin wrote: > i'm getting a SEGV after indexing 10 minutes of an mpeg1 film. > What more diagnostics can i provide? > > (gdb) where > #0 gst_mpeg_parse_parse_packhead (mpeg_parse=0x80da0e0, buffer=0x37ccd80) > at gstmpegparse.c:316 > #1 0x40835459 in gst_mpeg_demux_parse_packhead (mpeg_parse=0x80da0e0, buffer=0x8085e48) > at gstmpegdemux.c:277 > #2 0x40834509 in gst_mpeg_parse_loop (element=0x80da0e0) at gstmpegparse.c:380 > #3 0x408152c9 in gst_basic_scheduler_loopfunc_wrapper (argc=0, argv=0x80da0e0) > at gstbasicscheduler.c:279 > #4 0x4081999c in cothread_stub () at cothreads.c:444 > #5 0x40819d48 in cothread_switch (thread=0xbfe40000) at cothreads.c:650 > #6 0x408186bd in gst_basic_scheduler_iterate (sched=0x80bd528) at gstbasicscheduler.c:1342 > #7 0x4005075f in gst_scheduler_iterate (sched=0x80bd528) at gstscheduler.c:645 This crash is not happening consistently. On the most recent trial, the whole film got indexed without an error. Sometimes it crashes, sometimes not. i'll try valgrind, etc. -- Victory to the Divine Mother!! after all, http://sahajayoga.org http://why-compete.org |
From: Joshua N P. <vi...@po...> - 2002-11-23 07:45:04
|
On Sat, Nov 23, 2002 at 01:05:49PM +0530, Joshua N Pritikin wrote: > On Sat, Nov 23, 2002 at 01:00:21PM +0530, Joshua N Pritikin wrote: > > i'm getting a SEGV after indexing 10 minutes of an mpeg1 film. > > What more diagnostics can i provide? > > This crash is not happening consistently. On the most recent trial, the whole > film got indexed without an error. Sometimes it crashes, sometimes not. > i'll try valgrind, etc. The crash seems to be due to libhoard <http://www.cs.utexas.edu/users/emery/hoard/> After i stopped linking with hoard then the SEGVs disappeared. Cool! -- Victory to the Divine Mother!! after all, http://sahajayoga.org http://why-compete.org |
From: Xavier B. <xav...@fr...> - 2002-11-23 10:58:43
|
Le sam 23/11/2002 =C3 08:30, Joshua N Pritikin a =C3=A9crit=C2 : > The indexing seems a little too slow, especially for a DVD vob file. How slow ? |
From: Joshua N P. <vi...@po...> - 2002-11-23 11:06:27
|
On Sat, Nov 23, 2002 at 12:00:06PM +0100, Xavier Bestel wrote: > Le sam 23/11/2002 à 08:30, Joshua N Pritikin a écrit : > > The indexing seems a little too slow, especially for a DVD vob file. > > How slow ? On my 800Mhz Transmeta laptop, it's about one hour for one hour of film. VCD format is about five times faster. i'm sure there is more stuff to turn off in mpeg2dec. At a minibmum, we don't need to decompress the frames. This is just a proof of concept implementation. -- Victory to the Divine Mother!! after all, http://sahajayoga.org http://why-compete.org |
From: Xavier B. <xav...@fr...> - 2002-11-23 11:25:25
|
Le sam 23/11/2002 =C3 12:06, Joshua N Pritikin a =C3=A9crit=C2 : > On Sat, Nov 23, 2002 at 12:00:06PM +0100, Xavier Bestel wrote: > > Le sam 23/11/2002 =C3 08:30, Joshua N Pritikin a =C3=A9crit=C2 : > > > The indexing seems a little too slow, especially for a DVD vob file. > >=20 > > How slow ? >=20 > On my 800Mhz Transmeta laptop, it's about one hour for one hour of film. > VCD format is about five times faster. I've seen video editing software do it in nearly realtime (you "load" the mpeg and the thumbnails start appearing) on a P3 1GHz. Except when the mpeg is on a networked drive: then you can see it accesses a lot of data in the file before displaying its thumbnails. That was on a 30mn SVCD mpeg IIRC. > i'm sure there is more stuff to turn off in mpeg2dec. At a minibmum, > we don't need to decompress the frames. This is just a proof of > concept implementation. |
From: Joshua N P. <vi...@po...> - 2002-11-23 11:32:31
|
On Sat, Nov 23, 2002 at 12:26:41PM +0100, Xavier Bestel wrote: > Le sam 23/11/2002 à 12:06, Joshua N Pritikin a écrit : > > On my 800Mhz Transmeta laptop, it's about one hour for one hour of film. > > VCD format is about five times faster. > > I've seen video editing software do it in nearly realtime (you "load" > the mpeg and the thumbnails start appearing) on a P3 1GHz. Except when > the mpeg is on a networked drive: then you can see it accesses a lot of > data in the file before displaying its thumbnails. > That was on a 30mn SVCD mpeg IIRC. Yah, i'm sure we can do that too. It just takes a lot more infrastructure. -- Victory to the Divine Mother!! after all, http://sahajayoga.org http://why-compete.org |
From: Joshua N P. <vi...@po...> - 2002-11-27 10:09:58
|
On Tue, Nov 19, 2002 at 07:24:44PM +0100, Wim Taymans wrote: > For the plain indexing round, I would just count the number of bytes > handed to mpeg2dec, guestimate the start code of the picture and store > that time<->offset pair in the cache. What i am finding is that my external time->byte offset index is OK for VCD but not working when the bitrate is reduced down to 130kbps (1/4 size video, 64kbps audio). The stream is too compact to locate frames accurately using only byte offsets. Instead of 5-10 frame accuracy, i'm getting less than 30 frame accuracy. i speculate that i have no alternative other that using the GstTimeCache (with 2 level indexing). At least i'm a lot more familiar with the gstreamer mpeg internals now. > The idea is to hand a GstTimeCache object (object is already included in > the core) to the plugin. Looking at current CVS, it seems strange that the entries are stored in a GList (GstTimeCacheGroup). For a per-I frame index, this is going to kill performance. i would expect a bsearch'able and mmap'able array with numbers in big-endian byte order. Mmap is great if we get a chance to map it read-only. For updates, i expect that we are mostly appending records to the end of the array. (i don't think we need to optimize the data structures for indexing during reverse playback.) > The seeking event on the mpeg video decoder would then first figure out > what the nearest I frame is, it would convert the timestamp (or frame number) > to the PTS. It would then do a timeseek on its sinkpad with the PTS > value. Mpegdemux would get the seek on the PTS, it would map this to the > byteoffset of the videopacket with that PTS and would then forward the > byteseek to filesrc. OK, but there are still some steps remaining to achieve frame-accurate seek with only an I-frame index: * Seek to the nearest I-frame * Decode the I-frame, but don't display it * Decode, but don't display the intermediate P & B frames * Once the desired frame is found then resume normal playback Correct? Is it true that P frames depend on the previous P frame? Or only the previous I frames? > - API to to mapping from byte<->timeoffset > - API to get individual index records. > - API to get info about a record (is it a keyframe (I frame,... ), ..) > - API to load/save timecache (XML?, customizable?) This looks pretty easy except for load/save. What i suggest is to save each timecache as a directory instead of as a single file. This way we can put big-endian binary data in individual files along side an XML file with all the metadata. If this sounds strange then we can also support a tar.gz format transparently. > - certainty of index (for indexes created after a seek, ...) > - merging of index entries if certainty is known (playback reaches > previously seeked and indexed position from region with higher > certainty) This sounds like a mess. Do we really need so many different levels of uncertainty? How about two certainty levels: anchored and unanchored? For example, if you seek to the middle of an mpeg and start indexing then you can create an I-frame -> PTS index except that you don't really know the exact I-frame number. So this would be a relative index. As soon as another index overlaps which is anchored (has an exact I-frame counter) then the indicies can be matched on the PTS and merged. > I'm also thinking that the indexing should actually happen both in the > mpeg demuxer and the mpeg video decoder. The video decoder would map > frames (I frames) to PTS timestamps, the mpeg demuxer would index > byte offsets to the PTS values of the different streams, it would > probably also index SCR timestamps to offsets Yah, that sounds great. How can i help? The load/save code seems like a fairly independent project, but the timecache data structures need to be finalized. -- Victory to the Divine Mother!! after all, http://sahajayoga.org http://why-compete.org |
From: Xavier B. <xav...@fr...> - 2002-11-27 10:52:39
|
Le mer 27/11/2002 =C3 11:09, Joshua N Pritikin a =C3=A9crit=C2 : > On Tue, Nov 19, 2002 at 07:24:44PM +0100, Wim Taymans wrote: > > The seeking event on the mpeg video decoder would then first figure o= ut > > what the nearest I frame is, it would convert the timestamp (or frame= number) > > to the PTS. It would then do a timeseek on its sinkpad with the PTS > > value. Mpegdemux would get the seek on the PTS, it would map this to = the > > byteoffset of the videopacket with that PTS and would then forward th= e > > byteseek to filesrc. >=20 > OK, but there are still some steps remaining to achieve frame-accurate > seek with only an I-frame index: >=20 > * Seek to the nearest I-frame > * Decode the I-frame, but don't display it > * Decode, but don't display the intermediate P & B frames > * Once the desired frame is found then resume normal playback IMHO all this should only be done by the app. The app would just ask "what's the I frame just before that point" and seek there, then it would decode until it reaches the desired time point. This would enable some apps to do something useful with the decoded frames (even if they are not displayed), e.g. cache them. > Correct? Is it true that P frames depend on the previous P frame? Or = only the > previous I frames? >=20 > > - API to to mapping from byte<->timeoffset > > - API to get individual index records. > > - API to get info about a record (is it a keyframe (I frame,... ), ..= ) > > - API to load/save timecache (XML?, customizable?) >=20 > This looks pretty easy except for load/save. What i suggest is to save > each timecache as a directory instead of as a single file. This way > we can put big-endian binary data in individual files along side > an XML file with all the metadata. If this sounds strange then we > can also support a tar.gz format transparently. Maybe the load/save ops should be done (or doable) by the app. Some apps will want to save some other metadata about the mpeg, and want to avoid to have a gazillion metafiles. Nautilus could integrate the timecache in its metadata. Plus, some apps don't need the certainty info (e.g. a NLE will always start indexing from beginning, so its indexes will be certain). What would be great is a decoder/demuxer-independant API. Being able to say "seek to the decodable frame just before this timestamp" on any type of file (mpeg, avi/qt with anything inside, etc.) would help a lot. Xav - just rambling |
From: Joshua N P. <vi...@po...> - 2002-11-27 11:27:32
|
On Wed, Nov 27, 2002 at 11:53:59AM +0100, Xavier Bestel wrote: > Le mer 27/11/2002 à 11:09, Joshua N Pritikin a écrit : > > OK, but there are still some steps remaining to achieve frame-accurate > > seek with only an I-frame index: > > > > * Seek to the nearest I-frame > > * Decode the I-frame, but don't display it > > * Decode, but don't display the intermediate P & B frames > > * Once the desired frame is found then resume normal playback > > IMHO all this should only be done by the app. The app would just ask > "what's the I frame just before that point" and seek there, then it > would decode until it reaches the desired time point. This would enable > some apps to do something useful with the decoded frames (even if they > are not displayed), e.g. cache them. Yah, that's fine too. However, we still need to provide an API to accomplish it. Currently, i don't think there is a way to tell xvideosink to skip the next 3 frames or osssink to drop the next 3/25 seconds. (Correct me if i am mistaken.) Something like a semi-flush event? > > This looks pretty easy except for load/save. What i suggest is to save > > each timecache as a directory instead of as a single file. This way > > we can put big-endian binary data in individual files along side > > an XML file with all the metadata. If this sounds strange then we > > can also support a tar.gz format transparently. > > Maybe the load/save ops should be done (or doable) by the app. Dunno. Personally, i'd want to make the timecache an opaque data structure. > Some apps > will want to save some other metadata about the mpeg, and want to avoid > to have a gazillion metafiles. Nautilus could integrate the timecache in > its metadata. What use cases do you have in mind? Nautilus can store a tar of XML & binary data in its metadata-cache just as easily as any other format, no? > Plus, some apps don't need the certainty info (e.g. a NLE > will always start indexing from beginning, so its indexes will be > certain). The certainty flags consume an insignificant amount of space. i don't see any reason to omit them, even if they remain unused. Another idea which is probably obvious: When we convert fuzzy or uncertain data into certain data, maybe we can avoid changing all the values. For example, if all timestamps are changing by a constant amount then we can add a timestamp_base field to the TimeCacheGroup and only update this one value. > What would be great is a decoder/demuxer-independant API. Being able to > say "seek to the decodable frame just before this timestamp" on any type > of file (mpeg, avi/qt with anything inside, etc.) would help a lot. Yah, obviously. Perhaps the terminology in GstTimeCache should be changed. Instead of (location, timestamp) tuples, use arbitrary names like (value1, value2). -- Victory to the Divine Mother!! after all, http://sahajayoga.org http://why-compete.org |
From: Xavier B. <xav...@fr...> - 2002-11-27 12:14:12
|
Le mer 27/11/2002 =C3 12:27, Joshua N Pritikin a =C3=A9crit=C2 : > On Wed, Nov 27, 2002 at 11:53:59AM +0100, Xavier Bestel wrote: > > Le mer 27/11/2002 =C3 11:09, Joshua N Pritikin a =C3=A9crit=C2 : > > > OK, but there are still some steps remaining to achieve frame-accur= ate > > > seek with only an I-frame index: > > >=20 > > > * Seek to the nearest I-frame > > > * Decode the I-frame, but don't display it > > > * Decode, but don't display the intermediate P & B frames > > > * Once the desired frame is found then resume normal playback > >=20 > > IMHO all this should only be done by the app. The app would just ask > > "what's the I frame just before that point" and seek there, then it > > would decode until it reaches the desired time point. This would enab= le > > some apps to do something useful with the decoded frames (even if th= ey > > are not displayed), e.g. cache them. >=20 > Yah, that's fine too. However, we still need to provide an API > to accomplish it. Currently, i don't think there is a way to tell > xvideosink to skip the next 3 frames or osssink to drop the next > 3/25 seconds. (Correct me if i am mistaken.) Something like a > semi-flush event? Just disconnect it for 3 frames ? > > Some apps > > will want to save some other metadata about the mpeg, and want to avo= id > > to have a gazillion metafiles. Nautilus could integrate the timecache= in > > its metadata. >=20 > What use cases do you have in mind? Nautilus can store a tar of > XML & binary data in its metadata-cache just as easily as any other > format, no? I was thinking about an NLE who could save other informations (corrected ratio, etc.) with the timecache. But the more I think about it, the more I feel the "right" solution is input-plugin dependant: saving would vary very much depending on where the data comes from. A streaming client or a file plugin won't do the same thing. The gnome-vfs plugin should take advantage of the nautilus meta-data. > > Plus, some apps don't need the certainty info (e.g. a NLE > > will always start indexing from beginning, so its indexes will be > > certain). >=20 > The certainty flags consume an insignificant amount of space. i don't > see any reason to omit them, even if they remain unused. >=20 > Another idea which is probably obvious: When we convert fuzzy or uncer= tain > data into certain data, maybe we can avoid changing all the values. > For example, if all timestamps are changing by a constant amount > then we can add a timestamp_base field to the TimeCacheGroup and > only update this one value. Mmh .. this seems unnecessary complex. I don't think doing a bunch of additions to convert the timestamps from relative to absolute will have any mesurable cost. Xav |
From: Joshua N P. <vi...@po...> - 2002-11-27 12:25:39
|
On Wed, Nov 27, 2002 at 01:15:36PM +0100, Xavier Bestel wrote: > Le mer 27/11/2002 à 12:27, Joshua N Pritikin a écrit : > > What use cases do you have in mind? Nautilus can store a tar of > > XML & binary data in its metadata-cache just as easily as any other > > format, no? > > I was thinking about an NLE who could save other informations (corrected > ratio, etc.) with the timecache. > But the more I think about it, the more I feel the "right" solution is > input-plugin dependant: saving would vary very much depending on where > the data comes from. A streaming client or a file plugin won't do the > same thing. The gnome-vfs plugin should take advantage of the nautilus > meta-data. For what it's worth, my app implements a per-film XML spec: <?xml version="1.0"?> <!DOCTYPE film> <film> <media> <path>/local/aleader/nau-scene1.mpg</path> <toc>/local/aleader/nau-scene1.mpg.toc</toc> <duration>208.52</duration> <timeadj>643.6</timeadj> </media> </film> So that's how i solved it. -- Victory to the Divine Mother!! after all, http://sahajayoga.org http://why-compete.org |
From: Joshua N P. <vi...@po...> - 2002-11-27 12:19:00
|
On Wed, Nov 27, 2002 at 01:15:36PM +0100, Xavier Bestel wrote: > Le mer 27/11/2002 à 12:27, Joshua N Pritikin a écrit : > > Yah, that's fine too. However, we still need to provide an API > > to accomplish it. Currently, i don't think there is a way to tell > > xvideosink to skip the next 3 frames or osssink to drop the next > > 3/25 seconds. (Correct me if i am mistaken.) Something like a > > semi-flush event? > > Just disconnect it for 3 frames ? But how do you know when to reconnect it? Currently, i get frame notifications only via the frame-displayed signal emitted by xvideosink. > > Another idea which is probably obvious: When we convert fuzzy or uncertain > > data into certain data, maybe we can avoid changing all the values. > > For example, if all timestamps are changing by a constant amount > > then we can add a timestamp_base field to the TimeCacheGroup and > > only update this one value. > > Mmh .. this seems unnecessary complex. I don't think doing a bunch of > additions to convert the timestamps from relative to absolute will have > any measurable cost. OK, dump that idea. ;-) -- Victory to the Divine Mother!! after all, http://sahajayoga.org http://why-compete.org |
From: Xavier B. <xav...@fr...> - 2002-11-27 12:51:43
|
Le mer 27/11/2002 =C3 13:18, Joshua N Pritikin a =C3=A9crit=C2 : > On Wed, Nov 27, 2002 at 01:15:36PM +0100, Xavier Bestel wrote: > > Le mer 27/11/2002 =C3 12:27, Joshua N Pritikin a =C3=A9crit=C2 : > > > Yah, that's fine too. However, we still need to provide an API > > > to accomplish it. Currently, i don't think there is a way to tell > > > xvideosink to skip the next 3 frames or osssink to drop the next > > > 3/25 seconds. (Correct me if i am mistaken.) Something like a > > > semi-flush event? > >=20 > > Just disconnect it for 3 frames ? >=20 > But how do you know when to reconnect it? Currently, i get frame > notifications only via the frame-displayed signal emitted by > xvideosink. You connect it to a fakesink, and you have a notification signal for each buffer, with a timestamp. The only problem I see with this approach is that you will miss 1 frame. Xav |