Android PocketSphinx (bound to use C# with Xamarin) how to use a keyphrase list file

[Jim]

I have a simple app built on a C# conversion of the sample program supplied the Android PocketSphinx package,

The program works fine as long as I use a simple grammar.

However, I would like to use a keyphrase list file and use keyphrase recognition for more accurary.

I can't figure out what the proper incantations are in SpeechRecognizerSetup and elsewhere are to make this happen.

The keyphrase file is found and loaded but the attempt to recognize ends with:

[SpeechRecognizer] Start recognition "jarvis"
[trySphinx]  [2016/03/22 15:16:25.1057] - **DEBUG** OnCreate Error - Decoder_setSearch returned -1
[trySphinx]   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () <0x9ba250b8 + 0x00024> in <filename unknown>:0 
[trySphinx]   at Android.Runtime.JNIEnv.CallBooleanMethod (IntPtr jobject, IntPtr jmethod, Android.Runtime.JValue* parms) <0x9ba23e68 + 0x000c3> in <filename unknown>:0 
[trySphinx]   at Edu.Cmu.Pocketsphinx.SpeechRecognizer.StartListening (System.String p0, Int32 p1) <0x9ba23b08 + 0x001ff> in <filename unknown>:0 
[trySphinx]   at trySphinx.MainActivity.SwitchSearch (System.String searchName) <0x9ba23478 + 0x00047> in <filename unknown>:0 
[trySphinx]   at trySphinx.MainActivity.OnCreate (Android.OS.Bundle bundle) <0x9baa6478 + 0x00243> in <filename unknown>:0 

My code to setup is:

        private void SetupRecognizer(File assetsDir) {

            log.Debug(progName, "SetupRecognizer Entered");

            try {
                _recognizer = SpeechRecognizerSetup
                    .DefaultSetup()

                    .SetAcousticModel(new File(_assetDir, "en-us-ptm"))
                    .SetDictionary(new File(_assetDir, "cmudict-en-us.dict"))

                    .SetKeywordThreshold(1e-20f)

                    .SetBoolean("-allphone_ci", true)

                    .SetString("-kws", _assetDir.AbsolutePath + "/" + "keyphrase.list")

                    .Recognizer;

                _recognizer.AddListener(this);

                _isInitialised = true;

                log.Debug(progName, "SetupRecognizer done.");

            } catch (Exception) {
                throw new Exception ("SetupRecognizer failed");
            }
        }

And the keyphrase.list file is:

jarvis show navigation      /1e-40/
jarvis show weather         /1e-40/ 
jarvis show what's ahead    /1e-40/

Any help or comments would be greatly appreciated.

Regards,
Jim

[Nickolay V. Shmyrev]

Hello Jim

Something went wrong, for example, some word might be missing in the dictionary or file path might be wrong. You might find additional details in logcat output:

https://developer.xamarin.com/guides/android/deployment,_testing,_and_metrics/android_debug_log/

SetKeywordThreshold(1e-20f)
.SetBoolean("-allphone_ci", true)

You do not need those too settings anymore

[Jim]

Nickolay,

In Android 5.1.1 (and above 4.1) looking at logcat requires root. If I sent the Sphinx raw log to a file I can get to would that help?

Do I read that what I have is correct (except for the two lines you mention) and should work?

All of the words in the keyphrase list are in the dictionary and they work when I use the same phrases in a grammar.

But with a grammar I am not getting complete/exact recognition. For example, the phrase "jarvis show navigation" will be returned with the utterance of "jarvis navigation" which is not what I want. I would like as close to an exact match as possible and it seemed from posts I've read elsewhere that a keyphrase list is the way to go.

This will be a command and control Android service with a very limited list of phrases. The program I am wokring on now is a proof of concept. I may well go back to doing a keyword spot for "jarvis" and then using keyphrases after that but when I tried the keyword spot of "jarvis" in a noisey environment (music and talking) I did get some false positiove but I also got crashes and stops for which there was no data. I assume there may be something in logcat but I can't get to it unless you know a way.

Your thoughts?

Regards.
Jim

[Jim]

Nickolay,

I should have asked in the previous post:

1. Do you have an example of using a keyphrase list. I don't care what the language is, C#, C++, Java, even python, because after 50 years doing this commputer stuff they have all begun to look the same?

2. Do you have a list or a link to all of the -xxxx setings for pocketsphinx? I have some old code I wrote in Java 10 years ago for the desktop linux sphinx and want to see if some of the parms I used then are available. By the way I had great success with sphinx on that project.

Regards,
Jim

[Jim]

Nickolay,

I found what to do.

I had to remove the "-kws"option and add the line after setup as follows:

_regognizer.Decoder.SetKws("jarvis", _assetDir.AbsolutePath + "/" +"keyphrase.list");

The recognition worked BUT unlike what I had read, it still recognized partial phrases just like it did with a grammar, that is, th utterance "jarvis navigation" gave the result "jarvis show navigation".

Do you have any instght on this or was the article I read incorrect?

[Nickolay V. Shmyrev]

This is expected to have false alarms, to reduce number of false alarms you need to tune the threshold, try values like 1e-30 instead 1e-40