During some test I see that if test mode is set the program doesn't open the connection correctly and no audio will be received. The callback IAXC_EVENT_AUDIO will not be set by the library. You can test it with the stresstest.c, you will see no audio frame will be received. If you disable the testmode you can receive the audio frames.
Tested on 2.1beta 3 and svn current (20090416)