From: Amitha P. <ami...@us...> - 2008-12-10 06:44:03
|
When a vidl2 video is opened, one thing it does is that it will count the number of frames by advancing through the whole video: is_->num_frames_ = 0; while(advance()) ++is_->num_frames_; seek_frame(0); I think this is a really bad idea, and we should remove it. This implementation means that before doing anything with, say, a 2hr video, we have to essentially read all of it. We can try to estimate the number of frames from the duration field of the stream, but the documentation should make it clear that this is just an estimate, and could be wrong if the video is not quite kosher. The documentation should also make clear that the safest way to visit every frame in the video is vid.open(...); while( vid.advance() ) { // process } Thoughts? Amitha. |
From: Matthew L. <mat...@gm...> - 2008-12-10 13:49:52
|
Amitha, First a clarification, this frame counting issue relates to FFMPEG video streams. open() and num_frames() are virtual functions that are implemented by each istream. Some streams have efficient ways of counting frames (it's trivial for the image list), other are live streams and provide no count. Frame counting with FFMPEG has always been an issue. While we could provide an estimated frame count, I've found that every time I need a frame count I've needed an exact frame count. I agree that the frame counting should not be in the open() function. That was a quick fix to get things working. It should probably be done the first time num_frames() is called, and then cached. However, In this case you have to be careful to do some addition seeking because the frame pointer might not be at zero when num_frames() is called. For the record, I have used this on some large videos, and it wasn't as bad as I expected. Yes, you have to read through the whole file, but since only advance() is called there is no decoding and it goes about as fast as your disk can provide the data. I've been meaning to get back writing the vidl2 book chapter and updating the documentation, but I'm not sure If I'll have much time for that until after I finish my thesis. Matt On Dec 10, 2008, at 1:43 AM, Amitha Perera wrote: > When a vidl2 video is opened, one thing it does is that it will count > the number of frames by advancing through the whole video: > > is_->num_frames_ = 0; > while(advance()) > ++is_->num_frames_; > seek_frame(0); > > I think this is a really bad idea, and we should remove it. This > implementation means that before doing anything with, say, a 2hr > video, > we have to essentially read all of it. > > We can try to estimate the number of frames from the duration field of > the stream, but the documentation should make it clear that this is > just > an estimate, and could be wrong if the video is not quite kosher. > > The documentation should also make clear that the safest way to visit > every frame in the video is > vid.open(...); > while( vid.advance() ) { > // process > } > > Thoughts? > > Amitha. > > ------------------------------------------------------------------------------ > SF.Net email is Sponsored by MIX09, March 18-20, 2009 in Las Vegas, > Nevada. > The future of the web can't happen without you. Join us at MIX09 to > help > pave the way to the Next Web now. Learn more and register at > http://ad.doubleclick.net/clk;208669438;13503038;i?http://2009.visitmix.com/ > _______________________________________________ > Vxl-maintainers mailing list > Vxl...@li... > https://lists.sourceforge.net/lists/listinfo/vxl-maintainers |
From: Amitha P. <ami...@us...> - 2008-12-10 14:42:07
|
Matthew Leotta wrote: [re-ordered] > First a clarification, this frame counting issue relates to FFMPEG > video streams. Sorry, yes, I should've been clear about that. > For the record, I > have used this on some large videos, and it wasn't as bad as I > expected. Yes, you have to read through the whole file, but since only > advance() is called there is no decoding and it goes about as fast as > your disk can provide the data. Depending on where the video file is, disk speed, etc, this could be quite long! Sometimes, we run experiments with all the video files residing on network disks. For the experiments, the videos don't need to be read faster than the encoding rate (5 Mbit/s, say), which most network storage solutions can handle. In reality, it's slower than this, because experimental code doesn't always process a frame in 30 ms. With the loop, however, the experiment cannot begin until the entire video is read. This is especially an issue in the parameter tuning and debugging stages, where one often looks at the results of the first few seconds, adjusts parameters, and repeats. Moreover, tools like extract_video_frame extract_video_frame -i video.avi -f 15432 frame.png now take a long time to run, when it should be nearly instantaneous (for a video that supports seeking). And if you try to build a "real" pipeline using the vidl2 reader, a sudden startup delay becomes very noticable, and hard to justify. > Frame counting with FFMPEG has always been an issue. While we could > provide an estimated frame count, I've found that every time I need a > frame count I've needed an exact frame count. I'll observe that duration information with ffmpeg is actually okay *provided* the video file is actually good. The problem is that many videos are not correctly encoded, and often have an incorrect or invalid duration header. This is especially true of videos one gets from outside sources. Given this, I don't agree that the call to num_frames() should read the entire video. In many cases, duration()*frame_rate() will give you the correct number. I'll suggest a work around: "count_frames_on_open" could be a parameter in ffmpeg_istream_param. You can determine for your application whether you are willing to pay that cost or not. And the default should be "off". Amitha. |
From: Matthew L. <mat...@gm...> - 2008-12-10 17:38:39
|
On Dec 10, 2008, at 9:42 AM, Amitha Perera wrote: > > Depending on where the video file is, disk speed, etc, this could be > quite long! Yes, I agree that this is a problem that needs to be fixed. I didn't mean to imply that because I haven't run into problems that no one else would. I only meant to say that there are many cases when you need to count frames and a 2 hr video doesn't necessarily mean 2 hours of frame counting (though it certainly could in the circumstances that you describe). > In many cases, duration()*frame_rate() will give you the > correct number. From my experience using FFMPEG with the old vidl, I rarely remember duration()*frame_rate() giving the exact correct result. This was years ago, it is possible that FFMPEG has improved in this regard. However, I remember finding that some "bad" videos returned garbage from the calculation, while others estimated a value within a few frames of the correct result. It's probably acceptable in many cases if you loose a few frames in a long video, but it could be must worse if you are expecting some number of frames and they don't all exist. > I'll suggest a work around: "count_frames_on_open" could be a > parameter > in ffmpeg_istream_param. You can determine for your application > whether > you are willing to pay that cost or not. And the default should be > "off". This sounds reasonable, but maybe it's not necessary. My intention was that the num_frames() function should always be trusted to provide an accurate count of the number of frames in any finite video, regardless of the istream class used. It is often that case that a program doesn't need to know the number of frames, so num_frames() is not called. This is why I was thinking the counting should happen the first time num_frames() is called. If you really need to know how many frames there are then you can call num_frames() and be prepared to wait for the correct answer. Otherwise, don't call it. There should probably also be separate function called duration() on the istream that quickly gives an estimate of duration time. This would be useful for showing a time slider in a GUI where the exact number of frames isn't as important. What do you think? Matt > > > Amitha. > > ------------------------------------------------------------------------------ > SF.Net email is Sponsored by MIX09, March 18-20, 2009 in Las Vegas, > Nevada. > The future of the web can't happen without you. Join us at MIX09 to > help > pave the way to the Next Web now. Learn more and register at > http://ad.doubleclick.net/clk;208669438;13503038;i?http://2009.visitmix.com/ > _______________________________________________ > Vxl-maintainers mailing list > Vxl...@li... > https://lists.sourceforge.net/lists/listinfo/vxl-maintainers |
From: C. V. <cri...@gm...> - 2008-12-10 18:29:50
|
I think the number of frames shouldn't be counted on the constructor of vidl2_ffmpeg_istream, because it's not everyone who needs that information every time. A lazy calculation seems better to me. My program now takes a lot more time to open a video, when I use the latest VXL source code, due to this loop. But I actually need the number of frames, because I'm using a graphical slider to indicate the progression of the video (like YouTube's), and I need the total number of frames to do that. By the way, how do other programs, like mplayer, do this? As soon as I start mplayer, I can seek the video forward, and the program shows a progress bar of the current position of the video. So I suppose mplayer knows how many frames the video has, so it can show me a progress bar of its current position. If mplayer uses ffmpeg, how does it calculate the number of frames so fast? -- Crístian Deives dos Santos Viana [aka CD1] Google Talk: cri...@gm... |
From: Amitha P. <ami...@us...> - 2008-12-10 19:46:40
|
Matthew Leotta wrote: > This sounds reasonable, but maybe it's not necessary. My intention was > that the num_frames() function should always be trusted to provide an > accurate count of the number of frames in any finite video, regardless > of the istream class used. [...] > There should probably also be separate function called duration() on the > istream that quickly gives an estimate of duration time. This would work for me. Of course, this would need to be clearly spelled out in the documentation. Crístian, to your point (in another email): I think tools like mplayer rely on the duration, which is stored in the header. As Matt points out, it is often not accurate (at least, not frame accurate). However, it is often accurate enough for the purposes of a gui slider, progress bar, etc. Also note that the concept of a frame, and frame accurate seeking, is important for computer vision. However, it is often not that important in many video players and such. Amitha. |
From: Amitha P. <ami...@us...> - 2008-12-11 04:00:06
|
I have implemented a lazy version of num_frames() that will only scan the entire video on the first call. It will not scan the video on open(). On (currently undocumented) difference is that it will only try to compute the exact frame count if the first call to num_frames() occurs before the reader reads any data. This way, after the scan, the stream can be reset to the beginning, and we don't have to worry about frame accurate seeking. If you read some data, and then call num_frames() for the first time, you will get -1, indicating that the number of frames is not known. Since there aren't any extensive tests, I'll rely on the community for feedback on whether this fix breaks anything. Amitha. |
From: Matthew L. <mat...@gm...> - 2008-12-11 13:57:12
|
Amitha, Thanks for fixing this. I think this is the best solution for now. When we have frame accurate seeking implemented we can make this lazy evaluation valid at any time. Matt On Dec 10, 2008, at 8:54 PM, Amitha Perera wrote: > I have implemented a lazy version of num_frames() that will only scan > the entire video on the first call. It will not scan the video on > open(). > > On (currently undocumented) difference is that it will only try to > compute the exact frame count if the first call to num_frames() occurs > before the reader reads any data. This way, after the scan, the > stream > can be reset to the beginning, and we don't have to worry about frame > accurate seeking. If you read some data, and then call num_frames() > for > the first time, you will get -1, indicating that the number of > frames is > not known. > > Since there aren't any extensive tests, I'll rely on the community for > feedback on whether this fix breaks anything. > > Amitha. > > ------------------------------------------------------------------------------ > SF.Net email is Sponsored by MIX09, March 18-20, 2009 in Las Vegas, > Nevada. > The future of the web can't happen without you. Join us at MIX09 to > help > pave the way to the Next Web now. Learn more and register at > http://ad.doubleclick.net/clk;208669438;13503038;i?http://2009.visitmix.com/ > _______________________________________________ > Vxl-maintainers mailing list > Vxl...@li... > https://lists.sourceforge.net/lists/listinfo/vxl-maintainers |