Re: [Audacity-devel] Audacity recording

[Dominic M. <do...@au...>]

Joshua Haberman wrote:
> On Tue, 2003-11-04 at 14:14, Monty wrote:
> 
>>If you want some solid 'never skip' playback and record strategies, I
>>can go into detail with what I implemented in Beaverphonic.  A
>>practical implication isn't really hard, but it is very tricky.  You
>>can sum it up as:
> 
> 
> Actually, our audio I/O was rewritten about a year ago, and now uses a
> model that is what you are describing.  See src/AudioIO.cpp for more
> details.  We have three threads: the main UI thread, a thread that only
> reads and writes from disk during recording/playback, and a thread that
> only reads and writes from the soundcard.

Yep.  One of the problems now is that the disk thread and the GUI
thread do need to use mutexes, so while it's unlikely that the audio
thread itself is starved, it's still possible to drop samples because
the GUI thread is locking the disk thread out of accessing the data
structures it needs to update to keep track of where it's going.

> 
>>1) The thread reading data from the record device must do *nothing else*.
> 
> 
> This is already true: it is the thread that PortAudio creates, and it
> calls the Audacity callback that does nothing but read and write
> thread/safe ring buffers.  See audacityAudioCallback() in
> src/AudioIO.cpp.
> 
> 
>>2) It must have the highest priority on the machine. 
> 
> 
> pa_unix_oss sets the priority to SCHED_RR if possible (if you are
> running as root).  I guess it's pretty uncommon to run Audacity as root
> though.  On Windows pa_win_wmme previously was written to set a higher
> priority, but this was disabled since it was inadvertently setting the
> priority of the whole process (or something like that).
> 

I'm under the impression that the audio thread always gets set to
high priority on all platforms; it's only the PROCESS priority
that doesn't unless you're root.

>>3) It must never touch disk IO for any reason (that task is for
>>another thread).
> 
> 
> This is already true: there is a thread (we call it the audio thread,
> though it would probably be better called the disk thread) that does
> nothing but read and write to the disk, reading and writing the
> ring-buffers that the callback uses.
> 

That's mostly true, but as I said earlier, it goes through the
same data structures that the GUI uses.  This is a major weakness
right now.

>>4) It must be impossible for a lower priority thread to lock any mutex
>>at any time that prevents it from running.
> 
> 
> Our callback doesn't use any mutexes: the data is shared through
> lock-free ring-buffers.  I am not completely confident that our ring
> buffer class (see src/RingBuffer.cpp) is completely thread-safe (we
> haven't rigorously tested it), but I am fairly sure that it is possible
> to write a completely correct lock-free ring buffer.  I say this based
> largely on Paul Davis's advocacy of this approach, since he is someone
> whose ability I trust.  You can see his implementation of this idea
> here:

Just in case, I'll write up a test program for RingBuffer and stress
it outside of Audacity just to make sure there aren't any problems
there.

> http://cvs.sourceforge.net/viewcvs.py/ardour/ardour/libs/pbd/ringbuffer.cc?rev=1.2&view=auto
> 
> And a quick list message about him giving his justification for why it
> can be lock-free.  Actually, it appears he is a bit skeptical about this
> approach, I was not aware of this...
> 
> http://www.mail-archive.com/lin...@mu.../msg09618.html
> 
> 
>>Those are the basic requirements for just recording.  Mixing realtime playback into it seriously complicates things.
> 
> 
> Why?  Playback is just recording in reverse.  The disk thread reads the
> data from disk, writes it into the ring buffers which the callback
> thread then reads.
> 
> 
>>(I run Audacity on a dual Athlon 2800+.  It skips regularly during
>>both record and playback despite not even denting either processor.
>>For that reason, I don't use Audacity for either).
> 
> 
> I wonder if it's because our lock-free ring-buffer has flaws that are
> only exposed under SMP.  In any case it is something we should really
> fix!
> 
> 
>>Monty 
>>(who does realtime record, playback and low-latency effect processing
>>on linux PPC for paying audiences)
> 
> 
> If you have any comments on our implementation, I'm all ears.


I feel that improving recording performance in Audacity will come via
three routes:

1. Fix any bugs in the current implementation.  I've gotten lots of
    clean recordings using Audacity 1.1.3 and higher, but there may
    be circumstances where there are problems.

2. Clean up the interaction between the disk thread and the GUI
    thread so that the GUI thread can never slow down the disk thread.
    The GUI thread could be made more efficient in general.

3. My idea for a long time has been to add a separate dialog, which
    I've been calling "Smart Record", that pulls out all of the stops,
    by writing directly to disk from the disk thread and updating the
    GUI from a lower priority thread.  This dialog would also have a
    hundred other useful options, like timed recordings, VOX, extra
    mixer controls, and more.  The only thing that wouldn't be
    supported is full-duplex.  Closing this dialog would (optionally)
    import the file you recorded back into Audacity.

I strongly believe that #3 is the way to go to solve recording
problems.  We could probably get the main recording code for
Smart Record down to a few hundred lines of code, and we could
totally focus on optimizing that independent of the rest of
Audacity.

Note that lots of other packages give you only limited GUI functionality
while recording - quite a few "professional" programs do not display
the waveform in-place while you're recording it.  I happen to think
that Audacity's way of doing it is very intuitive for new users,
but clearly there's a reason why others choose a simpler approach
in their programs.

One last thing: there's no reason we shouldn't be as honest about
this as possible.  Audacity is NOT the best tool to use for
recording, especially recording very long files.  There are
many tricks you can use to get better performance
(http://www.audacityteam.org/wiki/index.pl?RecordingTips),
but in the end, Audacity is not up to par as a recorder yet,
though many people find it excels at editing.  We should make
it clear online that we are aware of these challenges and
hope to address them in the future.

- Dominic