Halef / Tickets / #55 HALEF keeps waiting for user response (potential VAD issue)

Vikram Ramanarayanan - 2015-07-07

Another instance of this was when I was using Peers to call into Patrick's Kaldi instance (Patrick, please provide more details here...)

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Vikram Ramanarayanan - 2015-07-07

assigned_to: Patrick Lange
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Vikram Ramanarayanan - 2015-07-08

This is an interesting call to listen to:

dfab4df655fbaba66cabf8974dffce56@141.31.8.62
HALEF_MixMonitor_audio_ext_3333_05072015_161111.wav
2015-07-05 10:14:09
191
3333
NULL

Also, this one:

b4ff7da5e8cbd682433efb2d8f02c2e7@141.31.8.62
HALEF_MixMonitor_audio_ext_3333_06072015_071137.wav
2015-07-06 01:13:42
162
3333
NULL

In both these cases, the audio quality gets spotty in the middle of the call, and either the system or caller or both get garbled. Also, these are a couple of examples with a longer call duration.

Last edit: Vikram Ramanarayanan 2015-07-08

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Vikram Ramanarayanan - 2015-07-08

What is saw is that this problem occurred twice during the interview item call administration on MTurk (ext. 3333) between July 4 and 6 (see call IDs above).

Last edit: David Suendermann-Oeft 2015-07-13

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Description has changed:

Diff:

--- old
+++ new
@@ -1,3 +1 @@
-During full calls, the system keeps waiting even though the caller has spoken already. For an example of this, listen to the full-call corresponding to the following call ID:
-
-6bb786dba2e3364d69f10b67ce5376ba@141.31.8.62 (extension 3333)
+During full calls, the system keeps waiting even though the caller has spoken already

David Suendermann-Oeft - 2015-07-13

Out of the 22 NULL calls collected in the period July 4 to 6 on Ext 3333, there are exactly the aforementioned two calls suffering from the described problem. In both calls, the call proceeds OK during the first few prompts until the third question at which the system stops listening and does not hear the caller anymore. The latter hand up out of frustration.

This is how a good recognition event should look like:

| 2967836 | 2015-07-05 10:11:39 | App type: application/srgs+xml |
| 2967837 | 2015-07-05 10:11:39 | Content: http://141.31.8.128:8080/7709/-/resources/PizzaInterview_Voice/Default/yesno.gram
|
| 2967838 | 2015-07-05 10:11:39 | processing application/srgs+xml |
| 2967839 | 2015-07-05 10:11:39 | Content type 'application/srgs+xml' not supported |
| 2967840 | 2015-07-05 10:11:39 | Using fallback to 'application/x-jsgf' |
| 2967841 | 2015-07-05 10:11:39 | Before doing the work, status code is -1 |
| 2967842 | 2015-07-05 10:11:39 | Recognition Mode is : normal So hotword flag is now: false |
| 2967843 | 2015-07-05 10:11:39 | No input timeout value is -1 |
| 2967844 | 2015-07-05 10:11:39 | OK, processor was null |
| 2967845 | 2015-07-05 10:11:39 | destination count now: 1 |
| 2967846 | 2015-07-05 10:11:39 | Creating realized processor... |
| 2967847 | 2015-07-05 10:11:39 | getContentType() |
| 2967848 | 2015-07-05 10:11:39 | getContentType() |
| 2967849 | 2015-07-05 10:11:39 | getStreams() |
| 2967850 | 2015-07-05 10:11:39 | getDuration() |
| 2967851 | 2015-07-05 10:11:39 | getControl() request received: controlType=javax.media.CachingControl |
| 2967852 | 2015-07-05 10:11:39 | getFormat() |
| 2967853 | 2015-07-05 10:11:39 | setTransferHandler(): bufferTransferHandler.class=class com.sun.media.parser.RawBufferParser$FrameTr |
| 2967854 | 2015-07-05 10:11:39 | DONE set settransferhandler()... |
| 2967855 | 2015-07-05 10:11:39 | start() |
| 2967856 | 2015-07-05 10:11:39 | getControl() request received: controlType=com.sun.media.util.RTPInfo |
| 2967857 | 2015-07-05 10:11:39 | stop() |
| 2967858 | 2015-07-05 10:11:39 | setTransferHandler(): bufferTransferHandler.class=class com.sun.media.parser.RawBufferParser$FrameTr |
| 2967859 | 2015-07-05 10:11:39 | DONE set settransferhandler()... |
| 2967860 | 2015-07-05 10:11:39 | setTransferHandler(): bufferTransferHandler.class=class com.sun.media.parser.RawBufferParser$FrameTr |
| 2967861 | 2015-07-05 10:11:39 | DONE set settransferhandler()... |
| 2967862 | 2015-07-05 10:11:39 | setTransferHandler(): bufferTransferHandler.class=class com.sun.media.parser.RawBufferParser$FrameTr |
| 2967863 | 2015-07-05 10:11:39 | DONE set settransferhandler()... |
| 2967864 | 2015-07-05 10:11:39 | getDuration()

However, this is what is happening at the third user input:

| 2968426 | 2015-07-05 10:12:03 | App type: application/srgs+xml |
| 2968427 | 2015-07-05 10:12:03 | Content: http://141.31.8.128:8080/7709/-/resources/PizzaInterview_Voice/Default/yesno.gram
|
| 2968428 | 2015-07-05 10:12:03 | processing application/srgs+xml |
| 2968429 | 2015-07-05 10:12:03 | Content type 'application/srgs+xml' not supported |
| 2968430 | 2015-07-05 10:12:03 | Using fallback to 'application/x-jsgf' |
| 2968431 | 2015-07-05 10:12:03 | Before doing the work, status code is -1 |
| 2968432 | 2015-07-05 10:12:03 | Recognition Mode is : normal So hotword flag is now: false |
| 2968433 | 2015-07-05 10:12:03 | No input timeout value is -1 |
| 2968434 | 2015-07-05 10:12:03 | OK, processor was null |
| 2968435 | 2015-07-05 10:12:03 | destination count now: 1 |
| 2968436 | 2015-07-05 10:12:03 | Creating realized processor... |
| 2968437 | 2015-07-05 10:12:03 | getContentType() |
| 2968438 | 2015-07-05 10:12:03 | getContentType() |
| 2968439 | 2015-07-05 10:12:03 | getStreams() |
| 2968440 | 2015-07-05 10:12:03 | getDuration() |
| 2968441 | 2015-07-05 10:12:03 | getControl() request received: controlType=javax.media.CachingControl |
| 2968442 | 2015-07-05 10:12:03 | getFormat() |
| 2968443 | 2015-07-05 10:12:03 | setTransferHandler(): bufferTransferHandler.class=class com.sun.media.parser.RawBufferParser$FrameTr |
| 2968444 | 2015-07-05 10:12:03 | DONE set settransferhandler()... |
| 2968445 | 2015-07-05 10:12:03 | start() |
| 2968446 | 2015-07-05 10:14:04 |
------------- RECEIVED A SIP REQUEST ---------------
Received a BYE SIP request

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

David Suendermann-Oeft - 2015-07-30

I was able to reproduce the problem. Calling Ext 7711, I went through almost the entire call until a point where Halef stopped responding. These are the last Cairo log entries:

2015-07-30 20:55:40,746 DEBUG ~.~.~.SESSION [IoThreadPool-3]: READ:
MRCP/2.0 187 SPEAK 1438278888341^M
Channel-Identifier:14ee04120bd@speechsynth^M
Content-Type:text/uri-list^M
Content-Length:60^M
^M
http://141.31.8.128:8080/7711/-/resources/cheese_pizza.wav^M

2015-07-30 20:55:40,754 DEBUG ~.~.~.MrcpRequestProcessorImpl [Thread-45]: MrcpRequestProcessorImpl.processRequest()...
2015-07-30 20:55:40,760 DEBUG ~.~.~.~.~.MrcpSpeechSynthChannel [Thread-45]: http://141.31.8.128:8080/7711/-/resources/cheese_pizza.wav^M

2015-07-30 20:55:40,767 DEBUG ~.~.~.~.~.MrcpSpeechSynthChannel [Thread-45]: http://141.31.8.128:8080/7711/-/resources/cheese_pizza.wav^M

2015-07-30 20:55:40,772 WARN ~.~.~.~.~.MrcpSpeechSynthChannel [Thread-45]: Multiple URIs not supported yet. Just playing the first URI.
2015-07-30 20:55:40,782 DEBUG ~.~.~.~.~.MrcpSpeechSynthChannel [Thread-45]: http://141.31.8.128:8080/7711/-/resources/cheese_pizza.wav text/html;charset=utf-8
2015-07-30 20:55:40,789 WARN ~.~.~.~.~.MrcpSpeechSynthChannel [Thread-45]: Unsupported content type for in the speak request: text/html;charset=utf-8
2015-07-30 20:55:40,795 DEBUG ~.~.~.MrcpRequestProcessorImpl [Thread-45]: MrcpRequestProcessorImpl got response from request handler.
2015-07-30 20:55:40,800 DEBUG ~.~.~.SESSION [IoThreadPool-2]: WRITTEN:
MRCP/2.0 85 1438278888341 -1 COMPLETE^M
Channel-Identifier:14ee04120bd@speechsynth^M

It turns out that the prompt

http://141.31.8.128:8080/7711/-/resources/cheese_pizza.wav

is not available and throws a 404 error, as tested by attempting to load the file in a web browser.

Conversely, the last prompt that was played

http://141.31.8.128:8080/7711/-/resources/Hello_Voice/Default/pizza_empty_catch.wav

is available.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

David Suendermann-Oeft - 2015-07-30

It turns out that the prompt cheese_pizza.wav is missing altogether in 7711.war.

@Vikram, please make sure all wave files are available in 7711, list all the missing ones you found and close this ticket. I will open a separate ticket for the creation of a tool ("preflight") which is to check on issues such as this one even before deploying.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

David Suendermann-Oeft - 2015-07-30

assigned_to: Patrick Lange --> Vikram Ramanarayanan
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

David Suendermann-Oeft - 2015-08-19

status: open --> closed
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

David Suendermann-Oeft - 2015-08-19

This was resolved and will be checked by the Pythia tool being developed.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

HALEF keeps waiting for user response (potential VAD issue)

Open Source VXML-Based Spoken Dialog System

Milestone

Searches

Help

#55 HALEF keeps waiting for user response (potential VAD issue)

Discussion