From: Ronald B. <rb...@ro...> - 2003-06-27 13:30:27
|
Howdy, On Fri, 2003-06-27 at 01:47, Guenter Bartsch wrote: > so, basically i think it would be interesting to see if it is possible > to agree on a common standard for free multimedia plugins, especially > for > > - a common input abstraction layer > - demuxers > - codecs > > and maybe also > > - audio/video output > > among some media player projects. I'd be in favour of some of these. There's some buts here... * if we take ffmpeg's demuxer (application) interface as an example, we see that it is severly limited. I don't see any way to specify subtitle streams, nor can I get private streams. I'm limited to video/audio, which isn't a good thing. Of course this is fixable, but in the best case, it'd require some sort of objectification, and as far as I understand, you guys aren't really in favour of this. There's three ways to do this: * c++ * g_* stuffies * c struct with casts Basically, (2) is (3) with some nicities of separation of classes and instances around it. Anyway, most people will dislike both (1) and (2) simply because it has dependencies (c++/glib). (3) looks evil and is quite some work to implement, but would be worth a try. What you probably want is - taking the demuxer as an example - to have a bytestream object which exports codecstreams (parent object) which can be a video/audio/subtitle/teletext/private/any stream (a child of the codecstream parent). Fortunately, for codecs, the idea is much simpler, but the same problem applies to properties: each codec must be able to generate *any* property with *any* value type. This is my main problem with ffmpeg currently (I do want to mention that ffmpeg totally rocks, but just like anything, it isn't perfect. ;) ), it's just one static struct. This is nice'n'simple, but also severely limiting. This is why using ffmpeg's current form of identifying codecs/streams/etc. wouldn't be a good idea to use as the basis of such a generic codec API, imo. GStreamer isn't perfect either. It depends on glib - I can understand that people won't like that. Same goes for other interfaces. It's basically fairly complex for a thing like a codec interface. It's fairly generalized towards basically anything, which makes it less suited as example for specific purposes. Basically, if I want a codec API or a demuxer API, I probably want these to be specifically suited for that purpose. For GStreamer, both APIs would be the same. ;). What I'm saying is that if we want to make a good codec API, some ideas of GStreamer (extendibility etc.) might be worth considering, but in general, GStreamer's API an sich might be a bit too general. I'm not sure what others think of this. The good thing is that you're not constrained by a limited set of properties, streams, types of streams or anything; everything is extendible without having to modify any of the GStreamer core. As for Xine/Mplayer, I'm sorry to say that I don't have enough experience with their code to give a strong opinion on it. I did look at the code a few times, but don't know the codebase well enough. I'll try to make some general comments, though, regarding the codec API of both. I've had a quick look at the mplayer demuxer code (simply because I took that as an example for the ffmpeg one too), and noticed that it has entries for audio, video and subtitles in the demuxer_t struct. That leads to the same complaint as for ffmpeg - it's pretty good, but it is somewhat limiting. Another problem (well, take these as comments, not complaints) is that the actual filestream data (which should be private in the demuxer, not adaptable by the application) is integrated in the demuxer_t struct, too. This isn't a problem, but it's not a good idea to make this part of the codec/demuxer API - it should only contain things that the application actually cares about. The rest should be kept private. As for Xine, I read through it quickly but apparently, I don't get it. ;). xine-lib/src/demuxers/demux.h doesn't mention anything apart from video. I guess I'm missing something. ;). > is there some documentation available on gstreamer's plugin api? sounds > very interesting to me, but what i found on their website was pretty > incomplete and hat lots of broken links in it. gstreamer is definitely > worth looking into, though. Well, there used to be some, but I can't find it anywhere, that's pretty much a bad thing. In our CVS, there's a gst-template module. In the directory gst-plugin/src/*, you'll find an example plugin. That is probably a good start. Direct WWW link: http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/gstreamer/gst-template/gst-plugin/src/ At the bottom, plugin_desc is the structure that loads the plugin. The _class_init(), _init() and _get_type() functions are all gobject core stuff. _get_property() and _set_property() (for properties) are all glib/gobject property functions. _chain() is the actual datastream callback for I/O. At the top, the pad templates define I/O points for this plugin, and a caps (none given here) defines the type of the stream. A GstCaps is basically what we use as way of identifying data types. Some more info on this is in CVS, module gstreamer: docs/random/mimetypes. www link: http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/gstreamer/gstreamer/docs/random/mimetypes. We're currenty basically done documenting all this, but it's not all implemented perfectly yet. > let's see, what he says :) one problem with gstreamer imho is their many > g'isms, not sure what the current state there is. i think a common api > should not be dependant on stuff like glib or gobject (though i love > glib i don't think it's a good idea to force everyone into using it). As said above, I tend to agree that it's not a good dependency for a core thing such as a codec API. What I'd love to see is the codec API not being an actual lib, but just a protocol, like X. Each application can then write it's own implementation code for this, and the codec API can, if wanted, provide a header file describing all structs (if any) etc. used in the protocol. Tying ourselves to one and the same lib likely won't work, even now we're already reproducing so much of the same code... Do people agree with this? Ok, now back to the actual question, what would it look like. A protocol describes structs or data fields, documents what properties mean what, etc. If we want a common demuxer API, I'd very much prefer if people would use the parent-child object stuffing I described in the beginning of this email. Each bytestream can export codec streams, which can be anything, including but not limited to audio, video, subtitle, teletext, private data or whatever. Types define the actual type of the codec stream, and by means of a child class, all specifics are filled in for the particular stream. For audio, this is rate, channels, maybe (in case of raw audio) the bitsize (8, 16) or container type (uint8_t, int16_t, maybe expansions of these for future reference), this can also be float data, btw! We just need to document what property name means what, and then simply make a name/value pair system that describes all these. For video, this would be width, height, ... For subtitles, this would be nothing at first, I guess. The parent object describes things like timestamps, duration, the actual data+size. Codecs: much easier. One input, one output. Properties are the same as the properties for the codec stream in the demuxer in case of the encoded data. For the decoded data, the same applies, but we'll probably want to provide some more properties for 'raw' data. Anyway, this just needs specifying. The mimetypes document above is what we currently use, other comments are welcome, of course. Concerns from my side, for as far as my experience goes with codecs in GStreamer: do we want to separate between codec and bytestream in case where these are (almost) the same, such as ogg/vorbis, mp3, etc? If so, what are the exact tasks of the parser (e.g. defining properties required by the codec, maybe metadata) and the codec (decoding), and what do we do if these interfere (e.g. I'm being told that not ogg, but vorbis contains the metadata of a stream!). And just to state clearly: our final goal is to propose a standardized API or interface of how codecs, muxer/demuxer libraries etc. should look to be usable by our applications. It is not to define how a bytestream should look. ;). Just so I (and you) know what we're actually talking about. Enough typing for now, I'd better get back to actual work here. :-o. Comments are very much appreciated. ;). Thanks for reading, Ronald -- Ronald Bultje <rb...@ro...> |