Re: [Iaxclient-devel] Some proposed feature additions to the library

[Michael V. D. <mic...@va...>]

Hi Steve (and Steves)

On Sun, 24 Oct 2004 14:05:08 -0500, "Steven M. Sokol"
<ss...@so...> wrote:

>Steve & Company,
>
>First, thank you for the codec code.  I will be releasing a new version=20
>of the free version of IAX Phone with the codec-enabled library =
included.

Same here.  I'm loving iLBC.

>I'm working on a few improvements (or at least additions) to the=20
>library, and I want some response.  Some of these are already coded,=20
>while others are still on the drawing board.  These include:
>
>- New registration routine
>- Client-Side Call Recording
>- Multi-Path Audio Output (and possibly input)
>- Onboard Mixing for 3-way Calling
>- URL Send/URL Event
>- DTMF Received Event
>
>-=3D New Registration Routines =3D-
>
>I've created an iaxc_register2 function that creates a managed list of=20
>registrations and their states.  It required a single change to the=20
>existing registration structure -- the addition of a regid integer to=20
>hold the registration identifier.  This value (simply an incrmenting=20
>integer) is generated when the registration is created and is used to=20
>identify the registration during callbacks (events).  The function call=20
>returns the newly generated registration ID, or negative values to=20
>indicate errors.

Great idea. I'm all for it.
>
>The registration event (appended to the main event structure) returns=20
>the registration ID, the registration result, and the message count for=20
>the mailbox associated with the user account (if any).  A new event=20
>type, IAXC_EVENT_REG is used as the event type.  The registration event=20
>is generated by the iaxc_regmsg function which is called from=20
>iaxc_handle_regreply.
>
>The general purpose of this change was to allow the client to display=20
>registration status on an account-by-account basis, and to track message=
=20
>counts for each account.  This is a very minor tweak (and I suppose we=20
>could have returned the registration information in the user message,=20
>but this seems to be a more elegant approach).

I agree, this sounds more elegant.  We'll still need to patch asterisk to=
 get
your message count sent correctly, right?

>-=3D Client-Side Call Recording =3D-
>
>This set of routines makes use of the libsndfile project code to=20
>integrate call recording into the library.  The same function can be=20
>handled by the server using the monitor, but in accordance with Mark's=20
>belief that the greatest amount of processing power is available at the=20
>edge, it can be quite useful.  It also fits a need for a client who=20
>/really/ wanted client-side recording and file playback.
>
>The code to make this happen comes, largely, from the libsndfile=20
>project.  The libsndfile functions handle the creation of the file=20
>header (if any) plus the proper formatting of the data for the storage=20
>format.  My implementation makes use of it as an external DLL, but it=20
>could readily be compiled into iaxClientLib.  It is cross-platform=20
>capable in most respects.  The one issue I have not been able to make=20
>fully cross-platform is timing.

I have no problem with this, but probably wouldn't use it myself if it =
isn't
portable.

>Windows supports "multimedia timers" which are accurate down to 1 ms. =20
>This allows for accurate playback.  I don't know much about timers in=20
>Linux (only that the 2.6 kernel is the first to support innate=20
>high-frequency timers) and absolutely nothing about timers in Mac OSX. =20
>Thus it /may/ be best to keep this as a patch outside of the main code=20
>base.  My implementation segements it with #ifdef  WIN32.
>
>The recording function captures the output audio stream (as linear PCM)=20
>in the send_encoded_audio function in audio_encode.c.  It captures the=20
>input audio stream (also as linear PCM) in the decode_audio function. =20
>At the present time, both streams are written to separate files which=20
>can be optionally mixed using a separate mixing function.
>
>Playback of files is handled in a timer callback function in the WIN32=20
>section of the code.  It encodes and sends the audio using the existing=20
>send_encoded_audio function.  I may add an option of playing the audio=20
>back over the local audio.output channel as well.
>
>The recording and playback functions generate a series of events which=20
>are added to the main event structure.  These include and event type of=20
>IAXC_EVENT_FILE_IO, an event sub-type (start, stop, etc.), and io=20
>sub-type (recording or playback), and a cause (timeout, end-of-file,=20
>digit, or stop command).
>
>This function currently exists in the version of the library that=20
>handled only GSM.  I am working on porting it into the new multi-codec=20
>version.
>
>-=3D Multi-Path Audio Output (and Input) =3D-
>
>This feature is still on the drawing board.  It services two functions. =
=20
>First, it allows for phones with multiple audio devices to direct the=20
>audio from one call to a given device and audio from another to a=20
>separate device.  The classic example of this is placing and "on hold"=20
>call on one device (the speakers) while communicating on another call=20
>using a headset.  The second function is a bit more esoteric and relates=
=20
>to a specific customer request.
>
>My thought here would be to have a pool of audio objects which can be=20
>dynamically assigned to the pool of calls.  Instead of having the=20
>"selected" call be the sole consumer/source of audio, the call would=20
>check to see if an audio object was assigned.  If so, it would make use=20
>of that object.  If not, the audio would be discarded.  This still=20
>allows for the use of the "selected" call as the primary (if the=20
>application needs a "primary") but also allows for other calls to route=20
>audio.
>
>We currently create a single audio object for the client, then assign=20
>various devices (in, out, ring) to that audio object.  All calls make=20
>use of that single audio object, but only when the call is the=20
>"selected" call.  In order for the multi-destination system to work, it=20
>would seem logical to create separate audio objects and assign them each=
=20
>an input, output and ring device.  Then, each call could be assigned a=20
>different audio object.  The function would take as parameters the call=20
>ID and the object ID and would handle properly passing the references,=20
>returning an error if the audio object is already in use or does not =
exist.
>
>I suspect that this will require quite a bit of reworking of the current=
=20
>audio driver structure.  Can the structure be expanded to support=20
>multiple audio i/o devices?  Can we simply instantiate and initialize=20
>several driver objects as it stants?  Do we run into any kind of issue=20
>in terms of threading if we try to read and/or write on multiple devices=
=20
>simultaneously?
>
>It will also require at least a small change to the call structure --=20
>adding the audio object as a memeber.  The audio handler routines would,=
=20
>obviously, need to be altered to make use of the audio object associated=
=20
>with the call, rather than the single central audio object.
>
>-=3D Onboard Mixing For 3-way Calling =3D-
>
>I originally started to work out a routine for passing feature requests=20
>back to the server over IAX primarily to enable a server-side=20
>conferencing function.  Mark steared me away from that, as again he=20
>feels the client should handle trivial functions like mixing for 3-way=20
>calling.  As such I want to implement client native simple conferencing.
>
>                    -----------
>CALL1 SEND--------->|         |------>CALL1 RCV
>                    |   MIX   |
>CALL2 SEND--------->|    &    |------>CALL2 RCV
>                    |   ECHO  |
>AUDIO IN (MIC)----->|  CANCEL |------>AUDIO OUT (SPK)
>                    -----------
>
>I suppose I could spend a bit of time making myself familliar with the=20
>code in app_meetme, but this may be a problem that other more familliar=20
>with the specifics of mixing and echo cancellation would be better able=20
>to address.  If nothing else, pointers would be happily accepted!
>

Would you run into the same timing source problem?  That is, asterisk on =
i86
linux requires a  (1ms??) timing source to clock its stream mixer.  They =
need a
zaptel board or a USB timer hack that is sometimes difficult to =
configure.

>-=3D URL Send/URL Event =3D-
>
>This is simply the addition of a wrapper to the innate IAX2 URL function=
=20
>and event.  It should look much like the text event.  This will make it=20
>easy for the application to either "pop" a browser, or display a web=20
>page in a browser window incorporated into the client.

Great idea!  I already have used an html window to send back billing =
info, etc.
It is controlled by the iaxclient app.  It would be great to have the =
asterisk
server send the URLs.

>-=3D DTMF Event =3D-
>
>This simply converts the DTMF events that are currently handled as part=20
>of the user message stream and provides a separate event that can be=20
>used by the client to recognize DTMF from the far end of the call.  This=
=20
>function is being added to facilitate some basic client-side automation=20
>that a customer has requested.

I don't think I would use this.

>Please give me thoughts on these features.  Let me know if they would be=
=20
>useful to you, and also if you see any reason NOT to include them. =20
>Obviously Steve K will be the final decision maker, but I would like to=20
>see what others think of these additions.

I see no reason to not include any of these changes.

I would love to see the new registration scheme implemented.

I would like to see the new audio mixer, to allow local music on hold.

>Thanks,
>
>Steve Sokol
>Sokol & Associates/AstriCon
>
>
>
>
>-------------------------------------------------------
>This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
>Use IT products in your business? Tell us what you think of them. Give =
us
>Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out =
more
>http://productguide.itmanagersjournal.com/guidepromo.tmpl
>_______________________________________________
>Iaxclient-devel mailing list
>Iax...@li...
>https://lists.sourceforge.net/lists/listinfo/iaxclient-devel