From: <wt...@ke...> - 2008-04-04 13:43:37
|
CVS Root: /cvs/gstreamer Module: gstreamer Changes by: wtay Date: Fri Apr 04 2008 13:43:40 UTC Log message: * docs/design/draft-latency.txt: Fix typo. * docs/design/part-buffering.txt: Update design docs with more buffering ideas. Modified files: . : ChangeLog docs/design : draft-latency.txt part-buffering.txt Links: http://freedesktop.org/cgi-bin/viewcvs.cgi/gstreamer/gstreamer/ChangeLog.diff?r1=1.3746&r2=1.3747 http://freedesktop.org/cgi-bin/viewcvs.cgi/gstreamer/gstreamer/docs/design/draft-latency.txt.diff?r1=1.7&r2=1.8 http://freedesktop.org/cgi-bin/viewcvs.cgi/gstreamer/gstreamer/docs/design/part-buffering.txt.diff?r1=1.5&r2=1.6 ====Begin Diffs==== Index: ChangeLog =================================================================== RCS file: /cvs/gstreamer/gstreamer/ChangeLog,v retrieving revision 1.3746 retrieving revision 1.3747 diff -u -d -r1.3746 -r1.3747 --- ChangeLog 3 Apr 2008 21:15:07 -0000 1.3746 +++ ChangeLog 4 Apr 2008 13:43:23 -0000 1.3747 @@ -1,3 +1,11 @@ +2008-04-04 Wim Taymans <wim...@co...> + + * docs/design/draft-latency.txt: + Fix typo. + * docs/design/part-buffering.txt: + Update design docs with more buffering ideas. 2008-04-03 Tim-Philipp Müller <tim at centricular dot net> * configure.ac: Index: draft-latency.txt RCS file: /cvs/gstreamer/gstreamer/docs/design/draft-latency.txt,v retrieving revision 1.7 retrieving revision 1.8 diff -u -d -r1.7 -r1.8 --- draft-latency.txt 24 Sep 2007 11:22:24 -0000 1.7 +++ draft-latency.txt 4 Apr 2008 13:43:26 -0000 1.8 @@ -293,7 +293,7 @@ When a playing sink is flushed by a FLUSH_START event, an ASYNC_START message is posted by the element. As part of the message, the fact that the element got -flushes is included. The element also goes to a pending PAUSED state and has to +flushed is included. The element also goes to a pending PAUSED state and has to be set to the PLAYING state again later. The ASYNC_START message is kept by the parent bin. When the element prerolls, Index: part-buffering.txt RCS file: /cvs/gstreamer/gstreamer/docs/design/part-buffering.txt,v retrieving revision 1.5 retrieving revision 1.6 diff -u -d -r1.5 -r1.6 --- part-buffering.txt 6 Nov 2007 10:21:01 -0000 1.5 +++ part-buffering.txt 4 Apr 2008 13:43:26 -0000 1.6 @@ -6,26 +6,109 @@ The purpose of buffering is to accumulate enough data in a pipeline so that playback can occur smoothly and without interruptions. It is typically done -when reading from a (slow) and non-live network source. +when reading from a (slow) and non-live network source but can also be used for +live sources. +We want to be able to implement the following features: + - buffering up to a specifc amount of data, in memory, before starting playback + so that network fluctuations are minimized. + - download of the network file to a local disk with fast seeking in the + downloaded data. This is similar to the quicktime/youtube players. + - caching of semi-live streams to a local, on disk, ringbuffer with seeking in + the cached area. This is similar to tivo-like timeshifting. + - progress report about the buffering operations + - easy (backward compatible) application notification of buffering + - the possibility for the application to do more complex buffering Some use cases: - +---------+ +--------+ +-------+ - | httpsrc | | buffer | | demux | - | src - sink src - sink .... + * Stream buffering: + +---------+ +---------+ +-------+ + | httpsrc | | buffer | | demux | + | src - sink src - sink .... In this case we are reading from a slow network source into a buffer element - (such as queue). + (such as queue2). The buffer element has a low and high watermark expressed in bytes. The buffer uses the watermarks as follows: - - The buffer element will not produce data on the src pad until the high - watermark is hit. While accumulating data in the buffer, progress is - reported by posting BUFFERING messages. - - The source will stop to produce data on the src pad when the low watermark - is hit. + - The buffer element will post BUFFERING messages until the high watermark + is hit. This instructs the application to keep the pipeline PAUSED, which + will eventually block the srcpad from pushing while data is prerolled in + the sinks. + - When the high watermark is hit, a BUFFERING message with 100% will be + posted, which instructs the application to continue playback. + - When during playback, the low watermark is hit, the queue will start posting + BUFFERING messages again, making the application PAUSE the pipeline again + until the high watermark is hit again. + - during playback, the queue level will fluctuate between the high and the + low watermark as a way to compensate for network irregularities. + This buffering method is usable when the demuxer operates in push mode. + Seeking in the stream requires the seek to happen in the network source. + It is mostly desirable when the total duration of the file is not know, such + as in live streaming or when efficient seeking is not possible/required. + * Incremental download + +---------+ +----|----+ +-------+ + V + file + + In this case, we know the server is streaming a fixed length file to the + client. The application can choose to download the file on disk. The buffer + element will provide a push or pull based srcpad to the demuxer to navigate in + the downloaded file. + This mode is only suitable when the client can determine the length of the + file on the server. + In this case, buffering messages will be emited as usual when the requested + range is not within the downloaded area + buffersize. The buffering message + will also contain an indication that incremental download is being performed. + This flag can be used to let the application control the buffering in a more + intelligent way, using the BUFFERING query, for example. + The application can use the BUFFERING query to get the estimated download time + and match this time to the current/remaining playback time to control when + playback should start to have a non-interupted playback experience. + * Timeshifting + file-ringbuffer + In this mode, a fixed size ringbuffer is kept to download the server content. + This allows for seeking in the buffered data. Depending on the size of the + buffer one can seek further back in time. + This mode is suitable for all live streams. + As with the incremental download mode, buffering messages are emited along + with an indication that timeshifting download is in progress. + * Live buffering + In live pipelines we usually introduce some latency between the capture and + the playback elements. This latency can be introduced by a queue (such as a + jitterbuffer) or by other means (in the audiosink). + Buffering messages can be emited in those live pipelines as well and serve as + an indication to the user of the latency buffering. The application usually + does not react to these buffering messages with a state change. Messages @@ -37,6 +120,46 @@ buffering is in progress. The BUFFERING message should be intercepted and acted upon by the application. +The message contains at least one field that is sufficient for basic +functionality: + "buffer-percent", G_TYPE_INT, between 0 and 100 +Several more clever ways of dealing with the buffering messages can be used when +in incremental or timeshifting download mode. For this purpose additional fields +are added to the buffering message: + "buffering-mode", GST_TYPE_BUFFERING_MODE, + enum { "none", "stream", "download", "timeshift", "live" } + - gives the buffering mode in use. See above for an explanation of the + different modes of buffering. + "avg-in-rate", G_TYPE_INT + - gives the average input buffering speed in bytes/second. -1 is unknown. + This is the average number of bytes per second that is received on the + buffering element input (sink) pads. If is a measurement of the network + speed in most cases. + "avg-out-rate", G_TYPE_INT + - gives the average consumption speed in bytes/second. -1 is unknown. + This is the average number of bytes per second that is consumed by the + downstream element of the buffering element. + "buffering-left", G_TYPE_INT + - gives the estimated time that bufferring will take in milliseconds. + -1 unknown. + This is measured based on the avg-in-rate and the filled level of the + queue. The application can use this hint to update the gui while buffering. + "estimated-time", G_TYPE_INT + - gives the estimated download time in milliseconds. -1 unknown. + When the size of the downloaded file is known, this value will contain + the latest estimate of the remaining download time. This value is usualy + only filled for the "download" buffering mode. Application @@ -51,29 +174,72 @@ with 100 percent value is received, which might only happen after the pipeline prerolled. +An exception is made for live pipelines. The application may not change +the state of a live pipeline when a buffering message is received. Usually these +buffering messages contain the "buffering-mode" = "live". +The buffering message can also instruct the application to switch to a periodical +BUFFERING query instead to more precisely control the buffering process. The +application can, for example, choose to not act on the BUFFERING message with +100 percent fill level to resume playback but instead use the estimated download +time to resume playback to get uninterrupted playback. Buffering Query --------------- -It is possible to query the amount of buffering performed in the pipeline, which -is defined as the amount of data made available at the source. This amount is -expressed in some GstFormat and is usually compared to the duration or position -of the media stream in the same GstFormat. +In addition to the BUFFERING messages posted by the buffering elements we want +to be able to query the same information from the application. We also want to +be able to present the user with information about the downloaded range in the +file so that the GUI can react on it. -The buffering query should return the following information: +In addition to all the fields present in the buffering message, the BUFFERING +query contains the following field, which indicate the available downloaded +range in a specific format: - - format - - position - - duration + "format", GST_TYPE_FORMAT + - the format of the "start" and "stop" values below + + "start", G_TYPE_INT64, -1 unknown + - the start position of the available data -The format is of lesser importance, the ratio of position versus duration can be -used to calculate the percentage of available media. It should also be possible -for the application to calculate the expected time when the complete file will -be buffered. + "stop", G_TYPE_INT64, -1 unknown + - the stop position of the available data +For the "download" and "timeshift" buffering-modes, the start and stop positions +specify the ranges where efficient seeking in the downloaded media is possible. +Seeking outside of these ranges might be slow or not at all possible. +For the "stream" and "live" mode the start and stop values describe the oldest +and newest item (expressed in "format") in the buffer. -Incremental download --------------------- +Defaults +-------- +Some defaults for common elements: +A GstBaseSrc with random access replies to the BUFFERING query with: + "buffer-percent" = 100 + "buffering-mode" = "none" + "avg-in-rate" = -1 + "avg-out-rate" = -1 + "buffering-left" = 0 + "estimated-time" = 0 + "format" = GST_FORMAT_BYTES + "start" = 0 + "stop" = the total filesize +A GstBaseSrc in push mode replies to the BUFFERING query with: + "format" = a valid GST_TYPE_FORMAT + "start" = current position + "stop" = current position |