From: Andy W. <wi...@po...> - 2002-06-17 17:18:19
|
On Mon, 17 Jun 2002, Baker, Steve wrote: > I've thought a bit about how to add midi support to GStreamer. I'm just > going to do a brain dump of my ideas since I doubt I'll be coding any of > this any time soon. I've thought a little bit about this too. Lemme first give you my vision, and then comment on yours. However, before starting, I have to say that I'm not a midi expert. I kinda understand the alsa seq api, and know what midi can do as far as hardware integration, but I'm not an expert. It would be good to hear what vektor has to say about all this (you listening, billy?) Unfortunately, the full MIDI specification is only available for a fee (around USD 60), so we have to go by other references... I found http://www.midi.org/about-midi/tutorial/tutor.htm to be useful, although you've got to up the font size a bit ;) > Midi could be thought of in terms of dataflow as a sparse non-constant flow > of bytes. GStreamer works best with near-constant data flow so a midi stream > would probably have to consist mostly of filler events, sent at a constant > tick-rate. As I understand it, on-the-wire hardware midi connections run at a fixed data rate, quoting from the page above: The MIDI data stream is a unidirectional asynchronous bit stream at 31.25 Kbits/sec. with 10 bits transmitted per byte (a start bit, 8 data bits, and one stop bit). Which is to say, 3125 bytes/sec. I would assume that the rawmidi interface would already filter out the stop and start bits? dunno. How about the diagram on http://www.philrees.co.uk/#midi, I found that to be useful. Now, there's another form of MIDI (the common usage?), "Standard MIDI files". We'll talk about that in a bit. I don't know very much about the OSS MIDI interface; apparently there exists an evil /dev/sequencer interface, and maybe a better /dev/midi* one. I only know this from overhearing it from people. ALSA has a couple ways to access MIDI devices. One way is the sequencer api. There's a tutorial, http://www.suse.de/~mana/alsa090_howto.html#sect04, and some example code, http://www.suse.de/~mana/seqdemo.c -- the paradigm is 'wait on some event fd's until you get an event, then process the event'. Not very GStreamer-like. This api timestamps the events, much like Standard MIDI files. The other way to use MIDI with alsa is by the rawmidi interface. Here's the canonical reference: http://www.alsa-project.org/alsa-doc/alsa-lib/rawmidi.html#rawmidi It seems there is example code, too: http://www.alsa-project.org/alsa-doc/alsa-lib/_2test_2rawmidi_8c-example.html#example_test_rawmidi This is much more like GStreamer. I do wonder about the ability to connect to other sequencer clients, though... > I think the following 3 elements would be most useful initially: > - midifileparse > takes midi file format data on sink pad, and produces timestamped midi data > on output. A property will specify what the tick rate would be (default to > 96 ticks per beat or something). If no data exists for a given tick, it can > just send a filler event. Timestamp would be derived from the bpm property, > and the time deltas of the midi file data. This would parse so-called Standard MIDI files. This, to me, is the least interesting application of MIDI. Let's be crude. Who gives a fuck about shitty little MIDI tunes? I don't. I want to use MIDI as a connection mechanism between cool softsynths and drum machines and sequencers and the like. Anyway, Standard MIDI files are just timestamped MIDI data; they don't run at a constant bitrate, and for that reason you need this element. > - ossmidisink > could be added to the existing oss plugin dir, sends midi data to oss midi > sequencer. Makes extensive use of GstClock to only send out data when the > buffer/event timestamp says it should. Yes. Or the raw midi device, doesn't matter which. > - alsamidisink > guess what this does. don't know whether alsa's sequencer interface would be > better than its raw midi one. My money is on raw midi. Yeah. I discussed this above. > - ossmidisrc, alsamidisrc > real time midi input. This needs to be from the raw api, though... > It would be nice to be able to transform midi to audio which can be further > processed in a gstreamer pipeline. You mean use GStreamer as some kind of softsynth. Yes, this would be nice. > Timidity might be the best hope for this, Maybe. See, there are two issues here. The first is sending MIDI data to softsynths and getting audio data out. There's a very, very nice way of doing this in ALSA, and that's the sequencer api. Timidity can already register itself as a sequencer client, as can amSynth, AlsaModularSynth, SpiralSynth, etc... and these latter ones are *much* more interesting. This is the proper, imho, way of doing things. But, the other question is getting that data back for use by GStreamer. In that sense a librafied timidity would be useful, I guess... see the thing is that all of these sequencer clients probably want to output to the sound card directly, although they are configurable. In this, the musician's only hope is Jack. If the synth is jacked up, we can get its output back into gstreamer. If not, oh well, it's gone... > Once we have midi streams, we can start doing fun things like writing a > midi2dparams element which would map midi data to control the dynamic > parameters of other elements, but lets not get ahead of ourselves. Yes! Anyway, it's a long road, at this point... regards, wingo. |