[Iaxclient-devel] Re: The trials and tribulations of the IAX library

[Steve K. <st...@St...>]

Long letter!  I'll try and comment on things inline, as much as
possible.  

I'm also CC'ing Mark and the iaxclient list on this, maybe someone else
has thoughts, about his jitter-buffer mechanism in particular.

On Wed, Feb 26, 2003 at 09:03:47AM -0700, Shawn Lawrence wrote:
[...]
> The library is well on its way to taking shape.  Tomorrow, I will be
> starting to construct the simple client to start debugging the coding that I
> have put together up to this point.  There is still work to be done on the
> library to get it completed properly, but it is a starting point.

That's good.  If you'd like any comments, I'd be happy to look over your
work in progress if you'd send it over. 

> The part that seems to be the greatest difficulty is deciding on how much
> common code we want to keep in the library.  To give you an example, the
> recommended way for socket communications to be done using the Win32 Winsock
> libraries is to use asynchronous I/O and/or overlapped I/O functions.  For
> compatibility and efficiency on the Win32 platform, the best thing to do
> would be to write Win32 functions to make proper use of the Winsock library.
> However, now we have two different sets of socket code to maintain and keep
> consistent with the rest of the code.  I have code put together the convert
> the Win32 socket handling to be fully Winsock compatible, but I don't want
> to incorporate it until we decide whether we want to go down this road or
> not.

OK, so the main difference, besides slightly different calling
conventions, is that it's recommended to use async io on Windows, as
opposed to the unix convention for stuff like this, which would be to
use non-blocking i/o and select(), or blocking i/o in i/o threads.

With this in mind, we can look at the major i/o methods we could use,
and determine what's supported where.   For now,  I'd say that if we can
find a solution that works reasonably well across our intended
platforms, we should go with that; later, if we want to use a different
i/o mechanism for a particular platform, we could do that if we needed
to.

I think that a decision to make this platform-specific would probably be
best made once we have something working, and can profile it and see if
the compromise is a problem or if it's immaterial.

> The second thing that is bothering me a bit is the jitter buffer event
> scheduling mechanism.  The first question that came to my mind was "Why is
> there a need to schedule all these events as they come in or go out?"  Then,
> I started to look deeper into the scheduling mechanism and started to
> realize this was their method of dealing with the jitter in the IAX
> transmissions.  At this point, I'm taking a deeper look into the scheduling
> mechanism to try and track down the problem you came across in Win95 and
> determine where the incompatibility is.  I believe the compatibility issues
> are coming from the way they are doing the time calculations on the Windows
> platform.  Win9x tends to handle certain API calls different that
> WinNT/2k/XP (even though Microsoft says this is never the case).  I am going
> to load the winiphone software you worked with before onto a Win98 box
> tomorrow and try to pin-point the exact location where the code was failing
> (I do my main development on Win 2k/XP machines).  Hopefully, I will be able
> to devise a simple solution and make minor changes to the scheduling code
> for now.  Otherwise, I will have to develop a different method for providing
> a jitter buffer that's either specific to Win32 or will work efficiently on
> both platforms (which will definitely affect our timelines).

I see.  Well the whole jitter-buffering mechanism is definately
something we'll probably be working on and tuning for a long time, and
hopefully if things are modular enough, it would be something we could
best work on once we have working test platforms for this.

> Lastly, I have gone through the code for the virt1800 library and the libiax
> implementation that was done as part of the IAXPhone project to get a better
> idea of how the existing Win32 IAX clients have been put together.  I
> definitely see some good things in the IAXPhone project, but also noticed
> that they have overridden some of the socket handling in the libiax library
> with their own socket functions.  They have developed their own form of
> asynchronous I/O control to deal with the socket traffic as it comes it.
> The code does look good, but these functions they have created are to
> emulate *nix based functions and may not necessarily be the most
> accepted/efficient way to do it on Windows.
> 
> So conceptually, there are several issues that need to be addressed here:
> 
> 1. How concerned are we with sticking to accepted programming practices on
> the various platforms?
> 2. Do we want to build more platform specific code modules into the
> libraries or do we want to stick to having the majority of the code
> cross-platform compatible?
> 3. Do we want this library tweaked for maximum performance on the platform
> it is being run on?

To answer all three questions, generally, at once:

We don't need to use the most common programming paradigms for each
platform, on each platform.   But we do need to make things work,
reliably.  If using "select()" on Windows works well enough, then it
would be fine to use that instead of async i/o.  

The library should, as much as possible, abstract the platform
differences from the library's clients; including platform-specific
code for both networking and audio.

Right now, we want something with acceptable performance on each target
platform;  The actual computational requirements of an audio client
should be relatively modest, and as long as we don't have unnecessary
spin-loops in the code, or vastly inefficient algorithms, we should be
able to make a client that works on any reasonably modern machine.  We
shouldn't need to go to great lengths to optimize anything, but we
shouldn't make any silly architectural decisions either.

> To the end users utilizing the library, they won't see any difference no
> matter what are decisions are on these items.  They will see a smaller
> subset of the current IAX functions that deal with initiating a call,
> sending audio, sending DTMF, hanging up, and other phone functions.  The
> client will no longer deal directly with the socket or event handling
> functions.  As well, I am looking to put in a callback function structure to
> have the library call a client based function perform all the audio
> processing on the received audio packets (after the header has been stripped
> and the IAX event dealt with accordingly).

I think it would make sense to put the audio processing code into the
library as well, including the platform-specific audio device code.

Perhaps this could later be made optional, so that a client that wanted
to handle audio i/o itself could do so, but I suspect that keeping the
audio driver code inside the library would be useful to most clients, by
reducing the duplication of code, since most IAX clients will definately
need audio i/o.

> I want to be close to having the library debugged and ready to send out with
> the Win32 client (steps 1 and 2) towards the end of the week.  However, I
> think we need to make some decisions on these issues before I get the whole
> library wrapped up and ready to send out to the rest of the world.  Let me
> know what your thoughts are on the items listed above.  I will be continuing
> my work on the library and starting on the client program tomorrow and will
> incorporate our decisions on these items after the discussions are
> completed.

That sounds good.  As I said, feel free to send me your code in any
incomplete statement, and perhaps I can give you some feedback (maybe
just stupid comments, but maybe something useful).

-SteveK


-- 
  Steve Kann - Chief Engineer - 520 8th Ave #2300 NY 10018 -  (212) 533-1775
                HorizonLive.com - collaborate . interact . learn
   "The box said 'Requires Windows 95, NT, or better,' so I installed Linux."