2008/9/4 Matt Leotta <matt.leotta@...>:
> On Tue, Sep 2, 2008 at 10:03 PM, Brendan McCane
> <brendan.mccane@...> wrote:
>> That should be easy for vidl2_v4l_istream. Since these are always live
>> streams AFAIK, returning -1 is the right thing to do. Can anyone think
>> of any situation where this isn't true?
> Well in that case it should be all set. I was under the impression
> that v4l did more than just live streams, but I've never really used
>> There's another issue with the v4l stuff - v4l is now obsolete and has
>> been for some time. It has been replaced by v4l2. Unfortunately, I
>> haven't kept up with the developments. I don't think it would take
>> much to convert over to v4l2, but it's something I just don't have the
>> time for at the moment. AFAIK v4l still works though.
> I've got an e-mail from someone claiming to have already written, but
> not checked in, a v4l2_istream. I'll forward that message along after
> this. If I get that code from him should it replace v4l_istream or is
> there a reason to keep both versions around?
v4l is obsoleted in the 2.5.x and 2.6.x kernels, but is still useful
for the 2.4.x kernels. So it depends if we want to support 2.4.x
kernels. I couldn't find any information on the whether 2.4.x is still
in widespread use, but it's probably just easier to make a
v4l2_istream just in case someone wants to use 2.4.
>>> Second, I think we need to clear up the API regarding advance(),
>>> current_frame(), and read_frame(). The way it is now, advance() moves
>>> the stream pointer one frame ahead (usually without fully decoding the
>>> next frame), current_frame() decodes the current frame (if necessary)
>>> and returns the frame, and read_frame() calls advance() followed by
>>> current_frame(). It was also originally assumed that each istream
>>> could be opened in a state with the current frame pointing to an
>>> invalid frame before the first frame of the video. The reason for
>>> this was that a live stream could be opened without capturing any
>>> images until the first call of advance() (or read_frame()). The
>>> problem with opening in an invalid state is that when piping an
>>> istream into an ostream the image size and other properties often need
>>> to be known before opening the ostream. This usually requires
>>> accessing the first frame of the istream for probing anyway. The
>>> probing process can lead to dropping of the first frame because
>>> read_frame() calls advance() again before calling current_frame(). So
>>> the questions I have are:
>>> 1) Should each istream always open in a valid state so that a frame
>>> can be accessed with current_frame() and probed for properties? In
>>> the case of live streams we can always discard the initial frame by
>>> calling advance() right before the capture loop. However, this does
>>> require that capture devices be ready to capture when the stream is
>>> 2) Should the redundant read_frame() function be removed? It may make
>>> iterating through streams easier for beginners, but it doesn't add any
>>> new functionality.
>>> 3) If read_frame() stays and each stream opens on the first frame,
>>> should read_frame() return the current frame and then advance instead
>>> of the reverse. Think i++ instead of ++i. This way you can open an
>>> istream and then immediately call read_frame() without dropping the
>>> first frame.
>>> There are still other lingering issues like synchronous vs
>>> asynchronous capture, but I think I'll leave that for another time.
>> Unfortunately, asynchronous capture has significant implications for
>> all the questions above. For reading stuff off disk, it probably
>> doesn't matter much, most of the following is flavoured by wanting to
>> capture stuff from a camera in real time.
> Not necessarily. Depending on your needs, you can still do
> asynchronous capture without worrying about the order of calling
> advance() and current_frame(). The dc1394_istream captures live video
> asynchronously using a separate thread that continually captures
> frames into a ring buffer. When I "open" the stream I allocate a
> buffer of say 10 frames (usually just 2 or 3 is even enough). It
> spawns off a thread that continually captures images into the buffer
> by overwriting the oldest image. Asynchronously, the advance()
> function iterates through the same ring buffer but has a check to make
> sure the data is ready to prevent the vidl2 loop from outpacing the
> capture loop.
> This gives me asynchronous capture. It is fast, nonblocking, and
> rarely drops a frame. The disadvantage is that each capture is not
> triggered by a call to advance() so you can't precisely control the
> timing of the image capture. You tell the capture thread to collect
> at a certain frame rate and it does that by itself.
> Unfortunately, all of this functionality is provided within libdc1394.
> I'm sure we could duplicate it in vidl2 if there was general
> interest. Although it does require multithreading which we don't have
> a standard way of doing in vxl yet.
Well, that would be an excellent solution. But you're right, it
probably should wait until the multithreading issue is sorted out.
>> For question 1), generally having the stream open in a valid state is
>> a good idea as it is likely to lead to fewer errors in user code based
>> on sequence dependencies. Rather than an API that says: first do this,
>> then do that, then that etc, it's generally better to have an API that
>> packages up all the sequence dependencies into a convenient form.
>> Unfortunately with things like capturing from a camera, this can cause
>> significant IO blocking problems. With asynchronous IO, you'd like to
>> be able to say "start getting the camera ready here asynchronously, so
>> I can start using it there immediately without waiting). If you're
>> ignoring asynch IO for the moment, then I'd say yes.
> I was hoping that in the case of live streams we could capture a
> single test image when opening. The camera can then go back into a
> waiting mode until the capture loop starts. The test image would be
> used to probe properties of the frames to come and will be discarded
> if the capture loop starts with advance() before the first
OK then - I'd be happy with that.
>> Q 2). No, read_frame shouldn't be removed, again because of sequential
>> dependencies. Almost all programs will want to do get current/capture
>> next, so might as well make it easy for them.
> Yeah, that was my original intent, but I recall there being some
> discussion a while back that it was redundant and not needed.
>> Q3). Again it depends heavily on asynchronous capture. With asynch, it
>> should definitely be "return current; get next" since getting the next
>> one can be done asynchronously and won't be blocking unless another
>> read_frame is called before it's had a chance to capture. But if it's
>> not asynchronous, then it makes more sense to do "get next; return
>> current", since you probably want to get the latest frame.
> The only advantage of "get next; return current" is that subsequent
> calls to current_frame() will return the same frame returned by the
> last read_frame() call. But this also leads to dropping the first
> frame, even in the non-asynchronous case.
>> It's starting to feel like we need a bigger API to handle asynch and
>> synch, or at least two read_frames (one asynch, one synch)? I was also
>> thinking that having an iterator interface would be syntactically neat
>> and familiar, but again asynchronous IO really makes a mess of that
>> (e.g. (*i++) would be an asynchronous read_frame, whereas (*++i) would
>> be a synchronous read_frame).
> Actually, I feel that this might be more of an argument for a smaller
> API (no read_frames functions). I am more in favor of no read_frames
> function than I am of having multiple read_frames functions. The idea
> of read_frames was to provided a simple function to use in a loop
> without really needing to know what what going on under the hood.
> It's not simple anymore when you have multiple versions.
> Instead of
> while(frame = istream.read_frame())
> it's not too much harder to use
> for(bool valid = true; valid && frame = istream.current_frame();
> valid = istream.advance())
> or for asynchronous capture
> while(frame = istream.current_frame() && istream.advance())
> These make the order of operations explicit. If they are still too
> complicated, maybe your idea of frame iterators could be used
> _instead_ of the read_frame() function. That would make a more
> compact and familiar way of iterating frames. A good C++ programmer
> should know the difference between ++i and i++, so they can be used
> appropriately to get the desired result.
Well, I would prefer the iterator syntax to the wordier current_frame,
advance syntax, and as you say, it's a nicer solution to multiple
read_frames, but I'm not keen enough to do anything about it myself
right now:-). I don't think it will be too big an issue if read_frame