Re: [Qtractor-devel] Qtractor and sysex

[Albert G. <Dr....@t-...>]

On 02/12/2012 08:54 PM, Rui Nuno Capela wrote:
> simply put, in qtractorMidiBuffer.h/cpp, the lv2 and vst midi event
> buffers are limited to 4KB in size. this was pretty much enough when all
> midi events were no more than 3bytes each.

Well, you actually have proper arrays of events there, not raw buffers. 
Of course the buffers can only hold a given number of events, but for 
the ALSA events it doesn't matter how big those events are. (Note that 
the ALSA events usually keep the sysex payload in a separate 
variable-length field in the event data structure.) It's true that the 
*VST* event data structure is pretty much limited to 4 bytes payload per 
event, but that doesn't matter as my code doesn't try to store any 
events bigger than that in the VST case. See below.

> now that sysex are in, we have trouble in the current code, which
> translates alsa midi events into lv2 and vst events and vice versa,
> which counts events instead of bytes.

I might be a sloppy coder, but not *that* sloppy. ;-) That's exactly why 
I had to increase the ALSA event buffer size and untangle the code for 
VST and LV2 sysex processing. Otherwise this stuff wouldn't have worked 
at all.

If you look at the code at qtractorMidiBuffer.cpp:469 ff, first the ALSA 
MIDI event is decoded into a local buffer of sufficient size (unsigned 
char midiData[c_iMaxMidiData]). Then, in the LV2 case, that buffer gets 
passed to LV2 using lv2_event_write(). In the VST case we check whether 
the decoded event is small enough to be passed in the VST event data 
structure, and only in that case it gets written into the next free slot 
in the VST MIDI buffer (the check is at line 492).

I don't recall from the top of my head how exactly the DSSI case is 
handled, but that's done elsewhere (in the DSSI processing function 
IIRC), and it grabs the event buffer directly from the MIDI manager. In 
any case we've got ALSA event data structures there, so we're good there 
as well.

For the reverse translations (i.e., for plugin output), IIRC 
snd_midi_event_encode() is used for that and the return value is checked 
in each case. So we should be good there as well, by virtue of the 
initialization in qtractorMidiBuffer.cpp:241, which makes sure that an 
event bigger than c_iMaxMidiData will not be encoded by ALSA. (I haven't 
tested that, though, since I don't have any plugins which will produce 
sufficiently big sysex dumps.)

So what happens now if you've got some really big sysex message on 
plugin input? If the event is encoded by ALSA, ALSA will refuse to 
encode it in the first place. But even if the event is constructed in a 
way which allows for arbitrary payloads (i.e., with 
snd_seq_ev_set_sysex(), as it is done in 
qtractorMidiBus::sendSysexList(), see qtractorMidiEngine.cpp:3905), the 
call to snd_midi_event_decode() at qtractorMidiBuffer.cpp:473 simply 
fails and that condition is properly caught and handled at line 475. 
That works the same no matter where the event actually comes from (i.e., 
from sendSysexList() or from a MIDI track).

> just try sending a bunch of sysex'es that totals 4KB or more. i'm sure
> it crashes on the floor :)

I actually gave it the old college try with 8K, 4K, 1K and 513 bytes 
messages and the messages were silently ignored, as it should be, 
whereas a 512 bytes sysex went through all right. I tried really hard, 
with both single messages and batches of several big and small messages, 
and with both bus sysexes and sysexes in a track imported from a MIDI 
file. It worked as advertized every time. Sorry to disappoint you, but 
no crashing anywhere. :)

I wasn't able to actually test the VST case, though. As I mentioned 
before, I don't have any VSTs to test it with, so someone else will have 
to do that. But it should work. (Note, however, that the VST plugins 
won't be able to receive any sysexes worth the effort, because of the 
silly event size limitation. But at least they shouldn't crash.)

Rui, if you were actually able to produce a crash due to big sysexes, 
then please send me the qtz and I'd be more than happy to fix it. But I 
can't fix a bug that I can't produce, if my careful analysis also shows 
that it shouldn't be there. So please, by all means prove me wrong if 
you can. :)

Cheers,
Albert

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikinformatik.uni-mainz.de/ag