[xine-devel] Skipping, dropping, replacing... on purpose!

[Shawn W. <sh...@wi...>]

I'm interested in adding some features to Xine to allow on-the-fly editing of 
DVDs according to an "edit script".  My main goal is to permit the creation 
of edit scripts that can simulate the "airline versions" of movies, so that 
people who like to watch movies but find some of the content offensive can 
still enjoy them.  After thinking about it a bit, it occurs to me that there 
are other uses as well, for example an appropriate script could be used to 
create a montage of scenes from a particular movie.  A friend of mine wants 
to be able to "invert" the script, so he can just watch all of the offensive 
parts ;-)

To do this well, I think the following editing features would suffice:

Audio editing:

1.	Silence.  The audio stream could be muted or even dropped entirely for 
short periods of time.
2.	Bleeps.  The classic way of covering up offensive words.
3.	Replacement.  The script file could provide a snippet of audio to be 
played instead of the audio stream from the DVD.

Video editing:

1.	Freeze frame.  In many cases, I think a short offensive scene could be 
removed by simply freezing the image for a moment, allowing the audio to 
continue.
2.	Blocking.  Sections of the image could be overwritten with blocks of solid 
color or the "giant pixel" blurring technique that seems to be favored by TV 
news.
3.	Skipping.  In other cases, it's probably necessary to jump forward seconds 
or minutes in the movie.

Subtitle editing:

1.	Replacement.  Simple enough.

So, I'm looking for any insights into what would be the best way to achieve 
these effects within the xine framework.  Ideally, it would be nice to 
centralize the changes in one place.

Here is where/how I think the effects can be implemented.  Note that this is 
my first foray into video/audio players, so there are undoubtedly things that 
I don't understand (hence this post).

It appears to me that there is a possiblity that audio dropping/replacement 
and video freezing can be done in the demuxer, by either dropping or 
replacing the appropriate packets, based on offset.  It would be nice to use 
PTS to determine where to begin and end the edits, but after running xine in 
a debugger it looks like PTS hasn't been determined yet at demux time?  
buf->PTS always contains the same value, anyway...

However, a potential problem occurs to me:  Packets may not provide 
sufficiently fine-grained control.  I would expect, for example that a 2KB 
packet of audio data would contain around a tenth of a second of audio (I'm 
assuming 48Khz 16-bit samples (not sure where I got those numbers from) and a 
4 to 1 compression ratio), which seems good enough but my numbers are suspect 
and only experience will show whether .1s accuracy is good enough.

It seems that the other option is to move the audio and video editing out 
closer to the audio and video output plugins.  This is kind of elegant, 
actually, in that it may be possible to insert an "edit plugin" between, for 
example, the real audio output plugin and the rest of xine.  This edit plugin 
would have raw, decoded audio bits which it could munge at will.  Same for 
video.  With video frames, this would seem to be the ideal location for 
blurring or overwriting.

Finally, I think skipping sections of a movie really has to be done in the 
demuxer or in the input plugin.  I fiddled around with making short jumps 
forward by simply bumping the metronome forward, but that doesn't work well 
unless the jumps are very short (<1 second or so), and probably wouldn't work 
well if many jumps were required because it would exhaust the buffered input.

For larger jumps, say a minute or two (or 15 or 30, if you're making a 
montage), we obviously don't want to bother reading and discarding all of the 
skipped data, so we need to lseek forward to the proper location, which would 
be done in the demuxer or the input plugin.  I'm thinking, again, that an 
"edit plugin" between the demuxer and the input plugin could take care of 
this.

So, to summarize, I'm thinking of a design that would add another thread 
which would be responsible for reading and parsing the edit script and would 
use fifos to send edit commands to three "shims" that would be inserted next 
to the video, audio and input plugins.

Comments?  I haven't put much time into this yet, so be kind :)

Shawn.