[Iaxclient-devel] Some proposed feature additions to the library

[Steven M. S. <ss...@so...>]

Steve & Company,

First, thank you for the codec code.  I will be releasing a new version 
of the free version of IAX Phone with the codec-enabled library included.

I'm working on a few improvements (or at least additions) to the 
library, and I want some response.  Some of these are already coded, 
while others are still on the drawing board.  These include:

- New registration routine
- Client-Side Call Recording
- Multi-Path Audio Output (and possibly input)
- Onboard Mixing for 3-way Calling
- URL Send/URL Event
- DTMF Received Event

-= New Registration Routines =-

I've created an iaxc_register2 function that creates a managed list of 
registrations and their states.  It required a single change to the 
existing registration structure -- the addition of a regid integer to 
hold the registration identifier.  This value (simply an incrmenting 
integer) is generated when the registration is created and is used to 
identify the registration during callbacks (events).  The function call 
returns the newly generated registration ID, or negative values to 
indicate errors.

The registration event (appended to the main event structure) returns 
the registration ID, the registration result, and the message count for 
the mailbox associated with the user account (if any).  A new event 
type, IAXC_EVENT_REG is used as the event type.  The registration event 
is generated by the iaxc_regmsg function which is called from 
iaxc_handle_regreply.

The general purpose of this change was to allow the client to display 
registration status on an account-by-account basis, and to track message 
counts for each account.  This is a very minor tweak (and I suppose we 
could have returned the registration information in the user message, 
but this seems to be a more elegant approach).

-= Client-Side Call Recording =-

This set of routines makes use of the libsndfile project code to 
integrate call recording into the library.  The same function can be 
handled by the server using the monitor, but in accordance with Mark's 
belief that the greatest amount of processing power is available at the 
edge, it can be quite useful.  It also fits a need for a client who 
/really/ wanted client-side recording and file playback.

The code to make this happen comes, largely, from the libsndfile 
project.  The libsndfile functions handle the creation of the file 
header (if any) plus the proper formatting of the data for the storage 
format.  My implementation makes use of it as an external DLL, but it 
could readily be compiled into iaxClientLib.  It is cross-platform 
capable in most respects.  The one issue I have not been able to make 
fully cross-platform is timing.

Windows supports "multimedia timers" which are accurate down to 1 ms.  
This allows for accurate playback.  I don't know much about timers in 
Linux (only that the 2.6 kernel is the first to support innate 
high-frequency timers) and absolutely nothing about timers in Mac OSX.  
Thus it /may/ be best to keep this as a patch outside of the main code 
base.  My implementation segements it with #ifdef  WIN32.

The recording function captures the output audio stream (as linear PCM) 
in the send_encoded_audio function in audio_encode.c.  It captures the 
input audio stream (also as linear PCM) in the decode_audio function.  
At the present time, both streams are written to separate files which 
can be optionally mixed using a separate mixing function.

Playback of files is handled in a timer callback function in the WIN32 
section of the code.  It encodes and sends the audio using the existing 
send_encoded_audio function.  I may add an option of playing the audio 
back over the local audio.output channel as well.

The recording and playback functions generate a series of events which 
are added to the main event structure.  These include and event type of 
IAXC_EVENT_FILE_IO, an event sub-type (start, stop, etc.), and io 
sub-type (recording or playback), and a cause (timeout, end-of-file, 
digit, or stop command).

This function currently exists in the version of the library that 
handled only GSM.  I am working on porting it into the new multi-codec 
version.

-= Multi-Path Audio Output (and Input) =-

This feature is still on the drawing board.  It services two functions.  
First, it allows for phones with multiple audio devices to direct the 
audio from one call to a given device and audio from another to a 
separate device.  The classic example of this is placing and "on hold" 
call on one device (the speakers) while communicating on another call 
using a headset.  The second function is a bit more esoteric and relates 
to a specific customer request.

My thought here would be to have a pool of audio objects which can be 
dynamically assigned to the pool of calls.  Instead of having the 
"selected" call be the sole consumer/source of audio, the call would 
check to see if an audio object was assigned.  If so, it would make use 
of that object.  If not, the audio would be discarded.  This still 
allows for the use of the "selected" call as the primary (if the 
application needs a "primary") but also allows for other calls to route 
audio.

We currently create a single audio object for the client, then assign 
various devices (in, out, ring) to that audio object.  All calls make 
use of that single audio object, but only when the call is the 
"selected" call.  In order for the multi-destination system to work, it 
would seem logical to create separate audio objects and assign them each 
an input, output and ring device.  Then, each call could be assigned a 
different audio object.  The function would take as parameters the call 
ID and the object ID and would handle properly passing the references, 
returning an error if the audio object is already in use or does not exist.

I suspect that this will require quite a bit of reworking of the current 
audio driver structure.  Can the structure be expanded to support 
multiple audio i/o devices?  Can we simply instantiate and initialize 
several driver objects as it stants?  Do we run into any kind of issue 
in terms of threading if we try to read and/or write on multiple devices 
simultaneously?

It will also require at least a small change to the call structure -- 
adding the audio object as a memeber.  The audio handler routines would, 
obviously, need to be altered to make use of the audio object associated 
with the call, rather than the single central audio object.

-= Onboard Mixing For 3-way Calling =-

I originally started to work out a routine for passing feature requests 
back to the server over IAX primarily to enable a server-side 
conferencing function.  Mark steared me away from that, as again he 
feels the client should handle trivial functions like mixing for 3-way 
calling.  As such I want to implement client native simple conferencing.

                    -----------
CALL1 SEND--------->|         |------>CALL1 RCV
                    |   MIX   |
CALL2 SEND--------->|    &    |------>CALL2 RCV
                    |   ECHO  |
AUDIO IN (MIC)----->|  CANCEL |------>AUDIO OUT (SPK)
                    -----------

I suppose I could spend a bit of time making myself familliar with the 
code in app_meetme, but this may be a problem that other more familliar 
with the specifics of mixing and echo cancellation would be better able 
to address.  If nothing else, pointers would be happily accepted!

-= URL Send/URL Event =-

This is simply the addition of a wrapper to the innate IAX2 URL function 
and event.  It should look much like the text event.  This will make it 
easy for the application to either "pop" a browser, or display a web 
page in a browser window incorporated into the client.

-= DTMF Event =-

This simply converts the DTMF events that are currently handled as part 
of the user message stream and provides a separate event that can be 
used by the client to recognize DTMF from the far end of the call.  This 
function is being added to facilitate some basic client-side automation 
that a customer has requested.

Please give me thoughts on these features.  Let me know if they would be 
useful to you, and also if you see any reason NOT to include them.  
Obviously Steve K will be the final decision maker, but I would like to 
see what others think of these additions.

Thanks,

Steve Sokol
Sokol & Associates/AstriCon