Re: [Iaxclient-devel] Any clues in the IAX2/NAT bug?

[Steve K. <st...@st...>]

Steven Sokol wrote:

> One thing I have noticed, is that the audio latency under Firefly 
> seems to have much lower than the latency from iaxClient-based UAs.  I 
> would guess that Adam saved a /bunch/ of CPU cycles by talking 
> directly to the MME interface, rather than using PortAudio or some 
> other abstraction layer.  The difference is notable. (Both clients 
> were using GSM).
>
PortAudio shouldn't intrinsically add a lot of latency, but there are 
two places in the implementation that add latency:

1) I'm currently using small ringbuffers in-between portaudio and the 
library.  This was mainly to simplify the implementation, where I didn't 
need to worry about re-entrancy in the library code itself.  This 
shouldn't really add a lot of latency.

2) PortAudio's implementation itself does add a significant amount of 
latency.  You can adjust this with an environment variable, or, if you 
want, you can change where the default is set in the portaudio code.  
See this message to the list for details:

http://sourceforge.net/mailarchive/message.php?msg_id=5248539

As it says there, they chose that variable to have reliable performance 
on Windows.  Perhaps in our usage, the balance of low latency vs. never 
missing a frame is different, and we can change this.

The latency there is something like 400ms on WinNT-based systems.  The 
ringbuffers probably add only abpout 20-40ms most of the time.

> I don't know how hard it would be to go back to the native audio I/O 
> interface, but that may be one way to cut down on the audio glitches.
>
As far as glitches, I think we've found that bug, and I will post it to 
CVS as soon as we get a little more testing.  If people are impatient, 
the proposed fix can be made by commenting line 666 (ironic, eh) in 
lib/portaudio/pa_win_wmme/pa_win_wmme.c.  Obviously, doing it right 
means negating the test and removing that block, etc.  But, we're going 
to discuss it with the portaudio developers to make sure it's correct, 
and contribute it back.

    662         if( stream->past_NumOutputChannels > 0 )
    663         {
    664             if((wmmeStreamData->outputBuffers[ 
wmmeStreamData->currentOutputBuffer ].dwFl        ags & WHDR_DONE) == 0)
    665             {
    666                 break;  /* If none empty then bail and try again 
later. */
    667             }

> BTW - What are your thoughts on implementing iLBC, SpeeX, G.711 A & U, 
> and possibly the new G.726/32 that just went into Asterisk?  I know 
> you are looking to get the library clean before we add too much more.  
> Just looking for thoughts as to the difficulty.
>

Right now, we're doing testing with what we have.  Those things 
shouldn't be terribly difficult to do.   First, though, we need to make 
an abstraction for codecs, similar to how we've abstracted audio 
drivers.  Once the abstraction is made, the individual codecs shouldn't 
be difficult.  (assuming, of course, they all build cross-platform).  
That means G.711 codecs are a no-brainer; SpeeX should work easy.  iLBC 
_probably_ is portable, but I haven't checked.  I'm not familiar with 
the other codecs you've mentioned, but if the abstraction is there, it 
should be possible to add them.

Just to ensure people that development here has an exciting future; the 
next _big_ thing we want to add is video support.   The hardest part 
there seems to be a portable video capture system, and a portable way to 
display video.  Unfortunately there's no "portvideo" project out there 
that I could find.    I'm not actually sure if I'll end up putting the 
audio and/or video capture/display stuff into the library proper, 
because it might make it more difficult to integrate with applications.

If people have thoughts or ideas on that, it would be great to discuss them.

-SteveK