From: Benjamin O. <in...@pu...> - 2004-01-19 15:21:12
|
Hi guys, I recently tried to write a vorbis decoder and vorbis uses float internally, so I thought I'd use float caps. However float seems to subscribe to the one-pad-per-channel mentality that just doesn't work with GStreamer. For several reasons: - having multiple connections between elements easily leads to deadlocks - even if it works it's slow because of all that cothread switching - it's extremely hard to code with REQUEST and SOMETIMES pads (I reduced the float2int code from 530 to 180 lines while adding stuff) - Nowhere in GStreamer are multiple connections used to represent one "real" connection. Try convincing autopluggers that they might need to use multiple connections to get stereo output for vorbis. And as far as I can see in our current elements, noone needs multiple pads. I just saw that float2int and alsasink are a hell of a lot more complicated then they need to be just because of float. So my questions are: - Why was that design decision made and is there any reason to keep using it? - If not and we designed a new float format, how should data in it be represented? What are the requirements? - Who is using float audio atm? Benjamin |
From: Steve B. <st...@st...> - 2004-01-19 19:39:58
|
On Tue, 2004-01-20 at 04:21, Benjamin Otte wrote: > And as far as I can see in our current elements, noone needs multiple > pads. I just saw that float2int and alsasink are a hell of a lot more > complicated then they need to be just because of float. > So my questions are: > - Why was that design decision made and is there any reason to keep using > it? > - If not and we designed a new float format, how should data in it be > represented? What are the requirements? > - Who is using float audio atm? The mono float format is used for pro-audio. If you want to see the technical justifications for the format then read through the LAD archives (sorry I can't be more specific right now). (http://www.linuxdj.com/audio/lad/archive.php3) Because we wanted to encourage developers to stick to this convention the float format was only given one channel and int2float always deinterleaves. Until now there were no practical uses for consumer audio using floats - it seems that now there is. I think the best thing might be to make the number of channels unlimited for float format, and change audioconvert to handle the int/float conversion. This would leave float2int/int2float untouched to do their specialised functions. cheers -- Steve Baker <st...@st...> |
From: Andy W. <wi...@po...> - 2004-01-22 14:07:37
|
Hi Benjamin, This is a discussion we've been through many times. I'll document the full extent of the problem sometime later, but I need to make it home now and so this will be necessarily short. On Mon, 19 Jan 2004, Benjamin Otte wrote: > However float seems to subscribe to the one-pad-per-channel mentality that > just doesn't work with GStreamer. For several reasons: > - having multiple connections between elements easily leads to deadlocks Not if (1) Elements adhere to the frames-per-buffer caps property (2) An element does not change the data flow rate, as measured in frames/sec, when in a multipad pipeline (impossible unless you set things up that way yourself) > - even if it works it's slow because of all that cothread switching I use opt. I know you think it's broken, but it's quite fast for my work and I haven't seen a problem yet. Ignore that scheduler, if you will, but it's being used. It's also more friendly to the stack, which is important for guile. > - it's extremely hard to code with REQUEST and SOMETIMES pads (I reduced > the float2int code from 530 to 180 lines while adding stuff) I coded all of that. It was tough, but there were design constraints there that perhaps you don't share. > - Nowhere in GStreamer are multiple connections used to represent one > "real" connection. Try convincing autopluggers that they might need to use > multiple connections to get stereo output for vorbis. Float data is a restricted realm that should not be autoplugged. Mono float is for DSP and analysis only, not for autoplugged media pipelines. > And as far as I can see in our current elements, noone needs multiple > pads. DSP is a lot easier with them. That's why it's that way. Check the l-a-d archives if you're unconvinced. And even if you're unconvinced and you want to play with vorbis or whatever, don't mess with this setup. My synthesizer depends on this behavior. > I just saw that float2int and alsasink are a hell of a lot more > complicated then they need to be just because of float. True. > So my questions are: > - Why was that design decision made and is there any reason to keep using > it? Search l-a-d. And yes, any kind of pro audio app (mine is one) will need this functionality. Also check the design of LADSPA, both the standard and our plugins. > - If not and we designed a new float format, how should data in it be > represented? What are the requirements? Perhaps this is the best way forward. But don't get rid of the old framework. That would give us 2 caps formats for what really are two different kinds of data: float-for-dsp and float-for-autoplugging-media-what-what. And you can allow interleaving in that format. But please, DON'T MESS WITH THE BEHAVIOR OF ADDER OR FLOAT/INT OR LADSPA PLUGINS without a full discussion here, on the list. > - Who is using float audio atm? Me and iain, I think, prolly steveb as well. Maybe the pakt kids, I dunno. Man, I sound grumpy. I guess I am because I have to make this argument every six months or so. But there are good reasons for it. So please, if you don't do any DSP work, just leave that area of caps-space alone :-) Regards, Wingo |
From: <in...@pu...> - 2004-01-22 16:28:33
|
Thanks for your answer. Looking at this and shooting from my hip, how about this: - Since every dsp element needs mono float, it'll just set the channels property to 1. Now since this is properly interleaved float audio, we can just expand it to more channels and use it in consumer float. - If we allow the buffer-frames property to be 0 to mean undefined, dsp elements could advertise [1 - MAX] in there templates if they want it fixed or [0, MAX] if they don't care or even 0 if they require unfixed. As a bonus the core would even automatically fixate to undefined behaviour. - If we split int2float and float2int each into two plugins, one doing channel splitting/merging, the other doing the actual conversion, we could use float2int and int2float with autoplugging. That would also solve the current problems for the people that have been reporting them with int2float ! float2int not working. Another option would be to make multichannel float be not interleaved but in succession in the same buffer. This would still be exactly the same for 1 channel, would allow simpler 1-to-n channel conversions but int2float would be harder. And the third option would be to do 1-channel-per-buffer and push one channel after another in multiple buffers. This would still be the same for 1-channel float but would make multichannel elements a bit harder. However it would get rid of some memcpys when splitting/merging. And yet another thing that moves us closer to conversion hell would be to invent another float format. Hm, what do you think? Benjamin PS: I'm not going to touch your code. In the worst case I'll do the actual float to int conversion inside vorbisdec myself (just like vorbisfile does currently). But since this problem came up for the equalizer, too I thought it'd be better to just simplify the float approach. Quoting Andy Wingo <wi...@po...>: > Hi Benjamin, > > This is a discussion we've been through many times. I'll document the > full extent of the problem sometime later, but I need to make it home > now and so this will be necessarily short. > > On Mon, 19 Jan 2004, Benjamin Otte wrote: > > > However float seems to subscribe to the one-pad-per-channel mentality that > > just doesn't work with GStreamer. For several reasons: > > - having multiple connections between elements easily leads to deadlocks > > Not if > (1) Elements adhere to the frames-per-buffer caps property > (2) An element does not change the data flow rate, as measured in > frames/sec, when in a multipad pipeline (impossible unless you set > things up that way yourself) > > > - even if it works it's slow because of all that cothread switching > > I use opt. I know you think it's broken, but it's quite fast for my work > and I haven't seen a problem yet. Ignore that scheduler, if you will, > but it's being used. It's also more friendly to the stack, which is > important for guile. > > > - it's extremely hard to code with REQUEST and SOMETIMES pads (I reduced > > the float2int code from 530 to 180 lines while adding stuff) > > I coded all of that. It was tough, but there were design constraints > there that perhaps you don't share. > > > - Nowhere in GStreamer are multiple connections used to represent one > > "real" connection. Try convincing autopluggers that they might need to use > > multiple connections to get stereo output for vorbis. > > Float data is a restricted realm that should not be autoplugged. Mono > float is for DSP and analysis only, not for autoplugged media pipelines. > > > And as far as I can see in our current elements, noone needs multiple > > pads. > > DSP is a lot easier with them. That's why it's that way. Check the l-a-d > archives if you're unconvinced. And even if you're unconvinced and you > want to play with vorbis or whatever, don't mess with this setup. My > synthesizer depends on this behavior. > > > I just saw that float2int and alsasink are a hell of a lot more > > complicated then they need to be just because of float. > > True. > > > So my questions are: > > - Why was that design decision made and is there any reason to keep using > > it? > > Search l-a-d. And yes, any kind of pro audio app (mine is one) will need > this functionality. Also check the design of LADSPA, both the standard > and our plugins. > > > - If not and we designed a new float format, how should data in it be > > represented? What are the requirements? > > Perhaps this is the best way forward. But don't get rid of the old > framework. That would give us 2 caps formats for what really are two > different kinds of data: float-for-dsp and > float-for-autoplugging-media-what-what. And you can allow interleaving > in that format. But please, DON'T MESS WITH THE BEHAVIOR OF ADDER OR > FLOAT/INT OR LADSPA PLUGINS without a full discussion here, on the list. > > > - Who is using float audio atm? > > Me and iain, I think, prolly steveb as well. Maybe the pakt kids, I > dunno. > > Man, I sound grumpy. I guess I am because I have to make this argument > every six months or so. But there are good reasons for it. So please, if > you don't do any DSP work, just leave that area of caps-space alone :-) > > Regards, > > Wingo > > > ------------------------------------------------------- > The SF.Net email is sponsored by EclipseCon 2004 > Premiere Conference on Open Tools Development and Integration > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > http://www.eclipsecon.org/osdn > _______________________________________________ > gstreamer-devel mailing list > gst...@li... > https://lists.sourceforge.net/lists/listinfo/gstreamer-devel > |
From: Andy W. <wi...@po...> - 2004-01-30 14:10:05
|
Hi Benjamin, First off, thanks for not returning my grumpiness ;) Sorry about that. We can now return to our regularly-scheduled wingo. I was probing my mind and wondering why I didn't want a didn't want channels=1 or whatever. I guess the complexity of the old caps[0] made it a PITA. Also, I feel in "pro-audio"[1] space we should encourage uniformity. I have a pro-audio document that addresses this. Furthermore, the whole caps system is an inefficiency wrt state transitions when you know you're in a realm of restricted variables. That might be paranoia, though. One more point is that buffer-frames is a key part of ensuring the efficiency and operability of a graph. It allows graphs that aren't linear not have to worry about bytestream-type hacks (which are prone to lockups in circular pipes) and just chain their buffers with efficient function calls. It is the thing that makes multi-pad elements bearable to write. Nevertheless, I could prolly be convinced to introduce more caps into the float domain, but we'd have to look at some other options first. [0] The new caps, from what I've coded with it, looks wonderful. I feel much more comfortable predicting the behavior of with the new caps. It feels more Correct. Nice job, Dave :) [1] I don't like this phrase, but I'll use it because of existing practice. Same goes for "consumer". Webs we weave! On Thu, 2004-01-22 at 18:28, in...@pu... wrote: > - If we split int2float and float2int each into two plugins, one doing channel > splitting/merging, the other doing the actual conversion, we could use > float2int and int2float with autoplugging. That would also solve the current > problems for the people that have been reporting them with int2float ! > float2int not working. int2float ! float2int should work. Why doesn't it? Sorry if I missed this one. It will drop all but the first channel though, in that parse syntax. > And yet another thing that moves us closer to conversion hell would be to > invent another float format. Consider this possibility, though. It has a number of advantages: (1) It reduces the complexity of capsnego in DSP pipelines (think many many elements) (2) It cleanly separates DSP plugins from media-processing plugins, allowing simplifications or flexibilities as desired (3) It removes the cause of these questions that seem to come up every 8 months or so The "consumer" format could even be called audio/x-raw-float, which would be more general, and the dsp format audio/x-dsp-float or something. The disadvantage would be that interoperability with the rest of GStreamer elements wouldn't be so good. You'd have to make int2dsp or float2dsp and their opposites. So, as I see it, we have to make int2float and the channel splitter into different elements anyway, for your purposes. Oneton is already done for us of course. The question is, should DSP plugins advertise x-raw-float caps or have their own caps? I'm leaning towards the second. Thoughts? Regards... -- Andy Wingo <wi...@po...> |
From: Benjamin O. <in...@pu...> - 2004-01-31 10:00:40
|
I'm not sure we need to seperate float caps, though it probably wouldn't hurt either, you can accept more than one mimetype ;) Anyway, my idea for a unified float caps is like this: mimetype: audio/x-raw-float (doh) properties: - width (int): 32 for float, 64 for double - endianness (int): BIG_ENDIAN or LITTLE_ENDIAN (we're still missing a converter for this. (off topic: mms needs those conversion functions, too as M$ sends out floats as timestamps) - channels (int): same as for int audio - rate (int): same as for int audio - buffer-frames (int): number of samples per channel in one buffer. If set to 0, the number of samples per buffer is undefined. All of the above is used like this in current GStreamer. One property I'd like to have: - layout (string): "interleaved" or "sequential", which would describe how the samples are put into the buffer on multichannel output. "interleaved" would be like today (example for stereo): | channel1,sample1 | channel2,sample1 | channel1,sample2 | ... | channel2,sampleN | "sequential" would be like this: | channel1,sample1 | channel1,sample2 | ... | channel1,sampleN | channel2,sample1 | ... | channel2,sampleN | sequential is obviously better suited for pro-audio as it is easy to split for oneton (just make subbuffers), while interleaved is easier for conversions from/to int. Currently we use interleaved everywhere. Given that format, pro-audio would set the properties like this: - channels to 1 - omit layout (it's only interesting for > 1 channel) - buffer-frames to [1 - MAX] consumer audio (like vorbisdec) use this: - channels to [1, N] - layout to what it can produce/consume (vorbis produces "sequential" for example, it's currently converted inside the plugin) - buffer-frames to 0 This obviously requires converters (floatconvert, ntoone, oneton (this one needs to work with float, too), int2float, float2int and maybe buffer-frames-adapt). Does that make sense? Or do you see issues there? Benjamin On Thu, 29 Jan 2004, Andy Wingo wrote: > Hi Benjamin, > > First off, thanks for not returning my grumpiness ;) Sorry about that. > We can now return to our regularly-scheduled wingo. > > I was probing my mind and wondering why I didn't want a didn't want > channels=1 or whatever. I guess the complexity of the old caps[0] made > it a PITA. Also, I feel in "pro-audio"[1] space we should encourage > uniformity. I have a pro-audio document that addresses this. > Furthermore, the whole caps system is an inefficiency wrt state > transitions when you know you're in a realm of restricted variables. > That might be paranoia, though. > > One more point is that buffer-frames is a key part of ensuring the > efficiency and operability of a graph. It allows graphs that aren't > linear not have to worry about bytestream-type hacks (which are prone to > lockups in circular pipes) and just chain their buffers with efficient > function calls. It is the thing that makes multi-pad elements bearable > to write. > > Nevertheless, I could prolly be convinced to introduce more caps into > the float domain, but we'd have to look at some other options first. > > [0] The new caps, from what I've coded with it, looks wonderful. I > feel much more comfortable predicting the behavior of with the > new caps. It feels more Correct. Nice job, Dave :) > [1] I don't like this phrase, but I'll use it because of existing > practice. Same goes for "consumer". Webs we weave! > > On Thu, 2004-01-22 at 18:28, in...@pu... wrote: > > - If we split int2float and float2int each into two plugins, one doing channel > > splitting/merging, the other doing the actual conversion, we could use > > float2int and int2float with autoplugging. That would also solve the current > > problems for the people that have been reporting them with int2float ! > > float2int not working. > > int2float ! float2int should work. Why doesn't it? Sorry if I missed > this one. It will drop all but the first channel though, in that parse > syntax. > > > And yet another thing that moves us closer to conversion hell would be to > > invent another float format. > > Consider this possibility, though. It has a number of advantages: > > (1) It reduces the complexity of capsnego in DSP pipelines (think many > many elements) > (2) It cleanly separates DSP plugins from media-processing plugins, > allowing simplifications or flexibilities as desired > (3) It removes the cause of these questions that seem to come up every 8 > months or so > > The "consumer" format could even be called audio/x-raw-float, which > would be more general, and the dsp format audio/x-dsp-float or > something. The disadvantage would be that interoperability with the rest > of GStreamer elements wouldn't be so good. You'd have to make int2dsp or > float2dsp and their opposites. > > So, as I see it, we have to make int2float and the channel splitter into > different elements anyway, for your purposes. Oneton is already done for > us of course. > > The question is, should DSP plugins advertise x-raw-float caps or have > their own caps? I'm leaning towards the second. Thoughts? > > Regards... > -- > Andy Wingo <wi...@po...> > > > ------------------------------------------------------- > The SF.Net email is sponsored by EclipseCon 2004 > Premiere Conference on Open Tools Development and Integration > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > http://www.eclipsecon.org/osdn > _______________________________________________ > gstreamer-devel mailing list > gst...@li... > https://lists.sourceforge.net/lists/listinfo/gstreamer-devel > |
From: Andy W. <wi...@po...> - 2004-02-05 13:19:18
|
Howdy again, On Sat, 2004-01-31 at 12:00, Benjamin Otte wrote: > I'm not sure we need to seperate float caps, though it probably wouldn't > hurt either, you can accept more than one mimetype ;) Yeah, but given that I'm the only one arguing for different caps, and that Dave says there's not a difference in efficiency between the two caps, it's just easier to unify. Your suggestions are good, except for: > - layout (string): "interleaved" or "sequential", which would describe how > the samples are put into the buffer on multichannel output. This, my friend, is a horrible idea. If we are going to have multiple channels per pad, let's not pretend like we actually have multiple pads. If we go multi-channel, we go interleaved. DSP audio plugins will almost always specify channels=1 in their caps, so it doesn't matter to them anyway. > This obviously requires converters (floatconvert, ntoone, oneton (this > one needs to work with float, too), int2float, float2int and maybe > buffer-frames-adapt). Well, ok. You have int2float and float2int (which maybe can be done with audioconvert). I'll take the hit and have oneton (which should either be called one2n or one-to-n) and ntoone be necessary for dsp pipelines (damn you ;). Then we're OK, except for buffer-frames-adapt, which I hope is never necessary, ever. > Does that make sense? Or do you see issues there? Naw, it's ok. I still want to make state changes faster, but that's a problem for another day. Cheers, -- Andy Wingo <wi...@po...> |
From: Thomas V. S. <th...@ap...> - 2004-02-05 17:00:43
|
Hi, > > This obviously requires converters (floatconvert, ntoone, oneton (this > > one needs to work with float, too), int2float, float2int and maybe > > buffer-frames-adapt). > > Well, ok. You have int2float and float2int (which maybe can be done with > audioconvert). I'll take the hit and have oneton (which should either be > called one2n or one-to-n) and ntoone be necessary for dsp pipelines > (damn you ;). Then we're OK, except for buffer-frames-adapt, which I > hope is never necessary, ever. I'm trying to grok wingo's street slang. I'm not hip to the kid's lingo these days :) Does this mean int2float will be the no-pro-audio element ? I would like to get the "conflict" between these two elements resolved, I don't want to release 0.7.5 with an element named int22float :) Thomas Dave/Dina : future TV today ! - http://www.davedina.org/ <-*- thomas (dot) apestaart (dot) org -*-> Surprise sometimes will come around I will surprise you sometime I'll come around <-*- thomas (at) apestaart (dot) org -*-> URGent, best radio on the net - 24/7 ! - http://urgent.fm/ |
From: Benjamin O. <in...@pu...> - 2004-02-06 11:40:25
|
On Thu, 5 Feb 2004, Thomas Vander Stichele wrote: > I would like to get the "conflict" between these two elements resolved, > I don't want to release 0.7.5 with an element named int22float :) > Which brings up a question: Do we guarantee the avialability of elements? I mean if 0.8 included an element named "foo", can we remove or replace it in 0.8.2? I wouldn't guarantee that. Benjamin |
From: David S. <ds...@sc...> - 2004-02-06 18:32:12
|
On Fri, Feb 06, 2004 at 12:39:17PM +0100, Benjamin Otte wrote: > On Thu, 5 Feb 2004, Thomas Vander Stichele wrote: > > > I would like to get the "conflict" between these two elements resolved, > > I don't want to release 0.7.5 with an element named int22float :) > > > Which brings up a question: Do we guarantee the avialability of elements? > > I mean if 0.8 included an element named "foo", can we remove or replace it > in 0.8.2? Given that elements may appear/disappear based on what packages are installed, we don't really guarantee the existence of elements from one minute to the next. However, we would need a Really Good Reason to change the name of an element during a stable series. dave... |
From: Andy W. <wi...@po...> - 2004-02-20 12:15:02
|
On Thu, 05 Feb 2004, Thomas Vander Stichele wrote: > Hi, > > > > This obviously requires converters (floatconvert, ntoone, oneton (this > > > one needs to work with float, too), int2float, float2int and maybe > > > buffer-frames-adapt). > > > > Well, ok. You have int2float and float2int (which maybe can be done with > > audioconvert). I'll take the hit and have oneton (which should either be > > called one2n or one-to-n) and ntoone be necessary for dsp pipelines > > (damn you ;). Then we're OK, except for buffer-frames-adapt, which I > > hope is never necessary, ever. > > I'm trying to grok wingo's street slang. I'm not hip to the kid's lingo > these days :) Sorry, I'll try to contain myself ;) Also my opinions are changing. I've been corrupted to the dark side of audioconvert, I think. So for a 'normal' pipeline, with a hypothetical vorbisfloat element, you can have: filesrc ! vorbisfloat ! audioconvert ! audiosink And for a DSP app that reads some data stored in int, you have: intsrc ! audioconvert ! one-to-n ! ... dsp ... And a DSP app writing int data to an audio sink, ... dsp ... ! n-to-one ! audioconvert ! intsink. This requires some cleverness in the (de)interleaving plugins regarding caps, and a couple more chain functions in audioconvert. Also I think this can make int2float and float2int die, as without the interleaving code they're pretty trivial. Does this sound sane? Regards, Wingo. |
From: David S. <ds...@sc...> - 2004-01-22 20:00:15
|
On Thu, Jan 22, 2004 at 05:28:29PM +0100, in...@pu... wrote: > - If we allow the buffer-frames property to be 0 to mean undefined, dsp > elements could advertise [1 - MAX] in there templates if they want it fixed or > [0, MAX] if they don't care or even 0 if they require unfixed. As a bonus the > core would even automatically fixate to undefined behaviour. You could also use buffer-frames={ "undefined", (int)[1-MAX] }. I don't necessarily recommend this, but it's possible. > - If we split int2float and float2int each into two plugins, one doing channel > splitting/merging, the other doing the actual conversion, we could use > float2int and int2float with autoplugging. That would also solve the current > problems for the people that have been reporting them with int2float ! > float2int not working. from a negotiation perspective, float2int and int2float are currently _really_ painful. They currently hold the record for most complicated negotiation code that I've written. Splitting the functionality into a channel splitter/merger element would be a good idea. dave... |
From: iain <ia...@pr...> - 2004-01-23 01:43:10
|
> On Thu, Jan 22, 2004 at 05:28:29PM +0100, in...@pu... wrote: > Splitting the functionality into > a channel splitter/merger element would be a good idea. One-to-n exists to do the spliting. I never wrote a merger because float2int did it for me iain |
From: David S. <ds...@sc...> - 2004-01-22 20:02:58
|
On Thu, Jan 22, 2004 at 03:58:51PM +0200, Andy Wingo wrote: > Hi Benjamin, > > This is a discussion we've been through many times. I'll document the > full extent of the problem sometime later, but I need to make it home > now and so this will be necessarily short. > Man, I sound grumpy. I guess I am because I have to make this argument > every six months or so. But there are good reasons for it. So please, if > you don't do any DSP work, just leave that area of caps-space alone :-) This is a discussion that will continue to come up every 6 months until it gets documented. :) gstreamer/docs/random/wingo/float-audio dave... |
From: Andy W. <wi...@po...> - 2004-01-30 14:06:56
Attachments:
pro-audio-with-gstreamer
|
On Thu, 2004-01-22 at 22:02, David Schleef wrote: > This is a discussion that will continue to come up every 6 months > until it gets documented. :) gstreamer/docs/random/wingo/pro-audio-with-gstreamer actually... -- Andy Wingo <wi...@po...> |
From: David S. <ds...@sc...> - 2004-01-30 21:17:21
|
On Thu, Jan 29, 2004 at 09:33:37PM +0200, Andy Wingo wrote: > Hi Benjamin, > > First off, thanks for not returning my grumpiness ;) Sorry about that. > We can now return to our regularly-scheduled wingo. > > I was probing my mind and wondering why I didn't want a didn't want > channels=1 or whatever. I guess the complexity of the old caps[0] made > it a PITA. Also, I feel in "pro-audio"[1] space we should encourage > uniformity. I have a pro-audio document that addresses this. > Furthermore, the whole caps system is an inefficiency wrt state > transitions when you know you're in a realm of restricted variables. > That might be paranoia, though. I've spent some time thinking recently about creating a subclass of GstElement specifically for pro-audio streams. This element would automatically handle all the caps negotiation for single-channel float audio, and could also be a hint to the scheduler that a group of GstProAudio elements can be scheduled as a lock-step group. > (1) It reduces the complexity of capsnego in DSP pipelines (think many > many elements) The complexity difference between pad template caps that have "audio/x-raw-float, channels=1, rate=[1,MAX], endianness=1234, buffer-frames=[1,MAX]" and "audio/x-raw-float, rate=[1,MAX], buffer-frames=[1,MAX]" is minimal. It would require a few extra lines here and there. The alternative will cause a tendancy to support audio/x-raw-float and audio/x-raw-dspfloat in the same element. The code required to properly support is complex and confusing -- just look at, e.g., ffcolorspace, which handles both video/x-raw-rgb and -yuv. dave... |
From: Andy W. <wi...@po...> - 2004-02-05 13:19:16
|
Hi Dave, On Fri, 2004-01-30 at 23:10, David Schleef wrote: > I've spent some time thinking recently about creating a subclass of > GstElement specifically for pro-audio streams. This element would > automatically handle all the caps negotiation for single-channel > float audio, and could also be a hint to the scheduler that a group > of GstProAudio elements can be scheduled as a lock-step group. We would need to make a general specification of what exactly occurs during state transitions and how exactly does capsnego occur and which parts exactly can be overridden. This way of thinking is admittedly new to me, but I'm thinking about the meta-object protocol in lisp or scheme object systems like CLOS or GOOPS. This kind of design can be seen in a 1992 paper by Gregor Kiczales and John Lamping, "Issues In the Design and Specification of Class Libraries". Unfortunately I don't have a URL at the moment. Such is life with intermittent internet! Imagine that we find a way to override capsnego, or make it simpler. We need to do that via a well-defined interface. So in this quest (a 0.9 quest I think) we will be writing a post-hoc specification for the GStreamer class library. But you are probably right: it's better to have a unified caps-space. More on that in the response to Benjamin. Cheers, -- Andy Wingo <wi...@po...> |