From: Wim T. <wi...@fl...> - 2006-02-23 13:36:01
|
On Thu, 2006-02-23 at 14:22 +0530, Sameer Naik wrote: > Hi Everybody, > I was working on how avsync happens in gstreamer. > Now, avsync is provided by the GstBaseSink Class... > through my study of the basesink class i found out that avsync can only > happen if a new segment is sent, otherwise ill get the message saying > "Buffer Received without new segment: Cannot Sync to Clock" Hi, Synchronisation in the sinks is performed using the following information: - base_time of the element - clock time - segment.start - buffer timestamp The stream time in a sink is calculated as: clock_time - base_time When setting a pipeline from READY to PLAYING or after a flushing seek, the clock is sampled and that value is set on _all_ elements as the base_time. This makes sure that clock_time - base_time yields the same value starting from 0 for _all_ elements. This also means that stream_time is reset to 0 when first starting a pipeline or after doing a flushing seek (that's why you either need to send the flushing seek to the pipeline so it can reset the stream_time or you reset the stream time yourself). When a buffer arrives in the sink, first the clipping check is done: segment.start <= buffer timestamp < segment.stop If the buffer timestamp is outside of the last received segment it is dropped. The segment therefore defines the valid range of buffer timestamps, this is usefull when doing accurate seeking (see later). The equation buffer_timestamp - segment.start therefore yields a value between 0 and the duration of the segment. If no segment arrived in the sink, a segment.start value of 0 is assumed, which requires that buffer timestamps start from 0 do do proper synchronisation. Then the equation: (buffer_timestamp - segment.start) / segment.abs_rate + base_time is used to compare the presentation time of the buffer against the clock. Also the segment.rate is used to rescale the timestamps and speed up/slow down playback. A typical demuxer will produce buffers on the source pads with timestamps that specify when they are to be displayed relative to eachother. The segment.start and stop values define the valid timestamp range and the 0 point of the timestamps. Many tricks can be done with this system, like retimestamping by adjusting the segment values (see gnonlin) or accurate seeking. Suppose you have a demuxer and you do a seek to, let's say, time 11s. Suppose there is a keyframe at time 10s the demuxer can send a segment.start of 11s and can start sending the first keyframe buffer at second 10. The decoder then decodes the keyframe but does not forward the buffer (as it is outside of the segment range). When a buffer at 11s is decoded, the decoder outputs that as the first buffer inside the segment (the decoder can also output 10s, in which case the sink will drop the buffer, although that only works for playback). Since the 0 point of the buffer timestamps is defined by the segment.start value, it is possible to output buffers from a demuxer that don't start with a 0 timestamps. If this happens, the sink has to receive the newsegment before it can properly sync on the buffer (and that's why it gives this warning in the absence of a segment since it cannot know if that was the intention...). > > so heres by question: > say my demultiplexer sends a new segment with the start time as the > base_scr (in case of transport stream demux) and the end time as > -1(infinity) ...and then starts sending video frames with a valid > timestamp, after being decoded these frames will be sent to xvimagesink > (which inherits the basesink)... > in the sync'ing functionality of the basesink i found out that the start > time and the end time of the received frame are extracted via the > get_times function call and then gst_segment_clip is done to find out > whether the frame shud be displayed or dropped... > so i what i would like to know is when the segment clip is done are > boundaries of the segment clipped too... > for example.. > if the segment sent has starttime of 0 and end time of -1, > next say a frame is sent with timestamp of 3...and say the frame has to > be displayed and not dropped, so will the segment boundary be changed > from the segment boundary is only changed with a newsegment event or with a flush (which resets all segment info), never with a buffer. > [ 0 to -1 ] to > [ 3 to -1 ] > > what im confused on is that :say a frame with timestamp 3 is received > and displayed and say the next frame with timestamp 2 arrives (not sure > whether this could happen :), or case where the display time of the > nextframe frame has passed )... how will it be dropped... (if the > segment itself is clipped then,.. it makes sense to me, if not then i > wonder how it happens..) Out of order buffers are a mostly a decoder error and should be fixed, there is no way a videosink can meaningfully display them without buffering some random N frames. That said, the audiosink can handle out of order buffers since it maintains a queue in a ringbuffer, but that is just an implementation detail. The frame should be dropped because it arrived too late in the sink but that is currently not done. The frames will just be shown in the wrong order. Alternatively since sinks also keep track of the last received timestamp, they could drop any buffers before that, which is not done either (but would be a trivial and acceptable patch for 0.10). Hope that helps, Wim > > _________ > > next thing is that in the basesink theres this case in the switch > statement > Case GST_CLOCK_EARLY: > which prints "Buffer to late, rendering anyway" > ...what is meant by buffer to late.... > is it that the display time has passed, but its rendering the frame > or is it that the display time has not yet come but it is rendering... The return value is a bit weird, from the perspective of the clock, the current timestamp is before it's current time (EARLY), which means that the time has already passed in the clock. Currently the sink still renders buffers that arived too late in the sink as fast as possible but that's going to change eventually. > > hope u guys understood my questions :) > > plz reply > thanks in advance > > take care > sameer > > > > ------------------------------------------------------- > This SF.Net email is sponsored by xPML, a groundbreaking scripting language > that extends applications into web and mobile media. Attend the live webcast > and join the prime developer group breaking into this new coding territory! > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642 > _______________________________________________ > gstreamer-devel mailing list > gst...@li... > https://lists.sourceforge.net/lists/listinfo/gstreamer-devel -- Wim Taymans <wi...@fl...> |