Re: [Lmuse-developer] does MusE support multithreaded audio processing?

[Tim E. R. <ter...@ro...>]

On September 12, 2013 07:18:37 PM Florian Jung wrote:
> Hi,
> 
> I wonder whether MusE can use multiple CPU cores for Audio processing,
> like plugins or my AudioStretchers.
> 
> Audio stretching is placed in void AudioPrefetch::prefetch(bool doSeek).
> Instead of reading from the file, I'm now reading from the AudioStream,
> which reads from the file, handles MP3/codecs, and handles stretching
> internally.
> 
> I guess that we can easily make the AudioPrefetch multithreaded (just
> create multiple objects, and instead of iterating through
> song->allWavetracks(), do some kind of load balancing.)
> 
> Comments on this?
> 
> 
> 
> 
> 
> But can we support multithreading with the audio effects as well?
> 
> My thoughts on this:
> - it would be cool!
> - instead of having Audio::process1() iterate through all tracks, let it
>   communicate with his "worker threads".
> - each of them may call AudioTracks::copyData on just some of all
>   AudioTracks
> - If inter-dependencies occur, solve them using Mutexes (*)
> - Communication with the workers works like this: each worker has
>   a "new data available"-semaphore and a "i'm done"-semaphore.
>   Audio::process1 will feed the data into the workers, and then release
>   the "new data avail". This will cause the worker to start.
> - After all workers are running, Audio::process will wait for all
>   workers' semaphores (*), when all are done, continue.
> 
> You probably though "whoa, no, we may not use mutexes or semaphores
> inside the realtime thread!" (*).
> 
> Are you sure? I think this rule only applies when the Mutex is shared
> between a "realtime" thread and a "non-realtime" thread.
> However, our workers are "realtime threads" as well. So the worst thing
> that can happen is:
> 
> No parallelisation is possible, all workers except one are waiting for a
> mutex. Then we have effectively serialized the work, and we have what
> Audio::process1() is doing right now anyway (with a little added
> overhead, though).
> 
> A worker thread may not call any "slow" system calls that might block,
> nor may it do any bad worst-case-operations
> 
> 
> 
> Note that this is a plan for the future, I have plenty to do right now.
> (audiostreams :)). But i'd like to know whether this is possible, or
> whether i made a mistake in thinking.
> 
> 
> Please comment :)
> Cheers,
> flo

Yeah I've wondered how we might leverage multi-core CPU power.
I don't know a lot about doing that. (Still single-core CPU here.)

But I was thinking: 
So far we've talked about the two available mechanisms in MusE: 

1) The "do some light work within one cycle" with msgXXX functions.
2) The "do some heavy work over several cycles" by putting the audio 
 in the idle state with the special msgIdle function, which causes a silence
 glitch over those cycles.

But IIUC there is apparently yet another option to do heavy work. 
As you mention, it is using worker threads.

Some of the wizards on LAD use techniques like this.
For example they talk about their DSP code that uses FFTs, which may 
 require more time, or more samples, than one cycle.
So they talk of worker threads and so on.

This came up on the jack-devel list the other day. Look in the archive for:
"[Jack-Devel] Using the callback free API with self-created RT-threads"
The discussion began with the the non-callback Jack API (look it up).
But Stephane pointed out the poster would be better off using the 
 Jack RT thread:

"The better way would be to use the JACK RT thread and implement the 
 real code in the function given with jack_set_process_thread. (so in your 
 case in  JackDriver::threads). This function would use the  
 jack_cycle_wait/jack_cycle_signal API. Then you can create additional 
 workers threads, and add the proper synchronization mechanism between 
 the JACK RT "master" thread at the worker threads."

I will not pretend to understand fully, just vaguely :)

NOTE: MusE does not use Jack's thread API - that is, we do not use Jack
 to create our threads. But the discussion still might apply to us.

Subscribe to LAD or jack-devel and introduce yourself and and ask a few
 questions. I'm sure guys like Paul or Fons or Stephane would be glad
 to help. This 'worker-threads' topic has come up from time to time 
 before.

Tim.