Sphinix crush when trying to decode a new word

[Amran Zeouay]

Hi everyone,
I'm using Sphinix for an android project, and I modified the dictionnary and also menu, when I start the application it reconigze the KeyPhrase that i added and then when i say "Menu" it crushs because after that i need to say "Amrane" but it crushs before I can say the word.

and the bug is in the switchSearch's function in the bold line

**        else if (text.equals(AMRANE))
**            switchSearch(AMRANE);

private void switchSearch(String searchName) {
        recognizer.stop();
        Log.v("switchSearch", " "+searchName);

        // If we are not spotting, start listening with timeout (10000 ms or 10 seconds).
        if (searchName.equals(KWS_SEARCH))
            recognizer.startListening(searchName);
        else
***            recognizer.startListening(searchName, 10000);
*
        String caption = getResources().getString(captions.get(searchName));
        speechtoText.setText(caption);
    }

And one more question how can i display everything that I can say, like if i said a word from the dictionnary it will displayed in the application ?

Thank you,

Here is the log of the crush :

I/SpeechRecognizer: Stop recognition
V/switchSearch:  menu
I/SpeechRecognizer: Start recognition "menu"
D/DisplayManager: DisplayManager()
D/SpeechRecognizer: Starting decoding
I/cmusphinx: INFO: pocketsphinx.c(986): Writing raw audio file: /storage/emulated/0/Android/data/fr.amrane.myapplication/files/sync/000000001.raw
I/cmusphinx: INFO: cmn_live.c(120): Update from < 
I/cmusphinx: 25.61 
I/cmusphinx:  6.32 
I/cmusphinx: -8.67 
I/cmusphinx: 10.05 
I/cmusphinx: 11.28 
I/cmusphinx:  4.54 
I/cmusphinx: -1.85 
I/cmusphinx:  1.64 
I/cmusphinx:  2.96 
I/cmusphinx: -1.93 
I/cmusphinx: -0.33 
I/cmusphinx:  2.85 
I/cmusphinx: -1.95 
I/cmusphinx: >
I/cmusphinx: INFO: cmn_live.c(138): Update to   < 
I/cmusphinx: 25.01 
I/cmusphinx:  6.57 
I/cmusphinx: -7.41 
I/cmusphinx: 10.23 
I/cmusphinx: 11.90 
I/cmusphinx:  4.46 
I/cmusphinx: -0.61 
I/cmusphinx:  2.40 
I/cmusphinx:  2.44 
I/cmusphinx: -1.99 
I/cmusphinx: -0.54 
I/cmusphinx:  3.11 
I/cmusphinx: -0.99 
I/cmusphinx: >
I/cmusphinx: INFO: fsg_search.c(859): 72 frames, 2247 HMMs (31/fr), 6586 senones (91/fr), 141 history entries (1/fr)
I/cmusphinx: INFO: fsg_search.c(869): fsg 0.29 CPU 0.397 xRT
I/cmusphinx: INFO: fsg_search.c(871): fsg 2.41 wall 3.302 xRT
I/SpeechRecognizer: Stop recognition
V/switchSearch:  amrane
I/SpeechRecognizer: Start recognition "amrane"
D/AndroidRuntime: Shutting down VM
E/AndroidRuntime: FATAL EXCEPTION: main
                  Process: fr.amrane.myapplication, PID: 6397
                  java.lang.RuntimeException: Decoder_setSearch returned -1
                      at edu.cmu.pocketsphinx.PocketSphinxJNI.Decoder_setSearch(Native Method)
                      at edu.cmu.pocketsphinx.Decoder.setSearch(Decoder.java:181)
                      at edu.cmu.pocketsphinx.SpeechRecognizer.startListening(SpeechRecognizer.java:141)
                      at fr.amrane.myapplication.MainActivity.switchSearch(MainActivity.java:129)
                      at fr.amrane.myapplication.MainActivity.onPartialResult(MainActivity.java:230)
                      at edu.cmu.pocketsphinx.SpeechRecognizer$ResultEvent.execute(SpeechRecognizer.java:427)
                      at edu.cmu.pocketsphinx.SpeechRecognizer$RecognitionEvent.run(SpeechRecognizer.java:391)
                      at android.os.Handler.handleCallback(Handler.java:739)
                      at android.os.Handler.dispatchMessage(Handler.java:95)
                      at android.os.Looper.loop(Looper.java:135)
                      at android.app.ActivityThread.main(ActivityThread.java:5910)
                      at java.lang.reflect.Method.invoke(Native Method)
                      at java.lang.reflect.Method.invoke(Method.java:372)
                      at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1405)
                      at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1200)

[Nickolay V. Shmyrev]

   java.lang.RuntimeException: Decoder_setSearch returned -1

There are earlier errors caused this, you need to provide the full code and data and also the full logcat output.

Most likely error is caused by the missing word in the dictionary.

And one more question how can i display everything that I can say, like if i said a word from the dictionnary it will displayed in the application ?

No, phone is too slow for that.

[Amran Zeouay]

So can you give me like an example for catching words like which function I need to call because i saw in the tutorial but there is some steps I didn't get them.

And thank you,

Here is my source code

public class MainActivity extends AppCompatActivity implements RecognitionListener {
    /* Named searches allow to quickly reconfigure the decoder */
    private static final String KWS_SEARCH = "wakeup";
    private static final String AMRANE = "amrane";
    private static final String SOS = "sos";
    private static final String TEST = "test";
    private static final String HELLO = "hello";
    private static final String DIGITS_SEARCH = "digits";
    private static final String MENU_SEARCH = "menu";


    /* Keyword we are looking for to activate menu */
    private static final String KEYPHRASE = "android";

    /* Used to handle permission request */
    private static final int PERMISSIONS_REQUEST_RECORD_AUDIO = 1;

    private SpeechRecognizer recognizer;
    private HashMap<String, Integer> captions;


    private TextView speechtoText;
    private ImageButton btSpeak;


    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        btSpeak = (ImageButton) findViewById(R.id.btnSpeak);
        speechtoText = (TextView) findViewById(R.id.speechToText);

        captions = new HashMap<String, Integer>();
        captions.put(AMRANE, R.string.amrane);
        captions.put(KWS_SEARCH, R.string.kws_caption);
        captions.put(SOS, R.string.sos);
        captions.put(KEYPHRASE, R.string.myoxy);
        captions.put(TEST, R.string.test);
        captions.put(HELLO, R.string.hello);
        captions.put(MENU_SEARCH, R.string.menu_caption);

        speechtoText.setText("Preparing the recognizer");
        // Check if user has given permission to record audio
        int permissionCheck = ContextCompat.checkSelfPermission(getApplicationContext(), Manifest.permission.RECORD_AUDIO);
        if (permissionCheck == PackageManager.PERMISSION_DENIED) {
            ActivityCompat.requestPermissions(this, new String[]{Manifest.permission.RECORD_AUDIO}, PERMISSIONS_REQUEST_RECORD_AUDIO);
            return;
        }
        runRecognizerSetup();
    }

    private void runRecognizerSetup() {
        // Recognizer initialization is a time-consuming and it involves IO,
        // so we execute it in async task
        new AsyncTask<Void, Void, Exception>() {
            @Override
            protected Exception doInBackground(Void... params) {
                try {
                    Assets assets = new Assets(MainActivity.this);
                    File assetDir = assets.syncAssets();
                    setupRecognizer(assetDir);
                } catch (IOException e) {
                    return e;
                }
                return null;
            }

            @Override
            protected void onPostExecute(Exception result) {
                if (result != null) {
                    speechtoText.setText("Failed to init recognizer " + result);
                } else {
                    switchSearch(KWS_SEARCH);
                }
            }
        }.execute();
    }

    private void switchSearch14(String searchName) {
        recognizer.stop();
        Log.v("MainActivity", " "+searchName);
        // If we are not spotting, start listening with timeout (10000 ms or 10 seconds).
        recognizer.startListening(searchName, 100000);

        String caption = getResources().getString(captions.get(searchName));
        Log.v("MainActivity", " "+caption);
        speechtoText.setText(caption);
    }


    private void switchSearch(String searchName) {
        recognizer.stop();
        Log.v("switchSearch", " "+searchName);

        // If we are not spotting, start listening with timeout (10000 ms or 10 seconds).
        if (searchName.equals(KWS_SEARCH))
            recognizer.startListening(searchName);
        else
            recognizer.startListening(searchName, 10000);

        String caption = getResources().getString(captions.get(searchName));
        speechtoText.setText(caption);
    }

    private void setupRecognizer(File assetsDir) throws IOException {
        // The recognizer can be configured to perform multiple searches
        // of different kind and switch between them

        recognizer = SpeechRecognizerSetup.defaultSetup()
                .setAcousticModel(new File(assetsDir, "en-us-ptm"))
                .setDictionary(new File(assetsDir, "cmudict-en-us.dict"))
                .setRawLogDir(assetsDir) // To disable logging of raw audio comment out this call (takes a lot of space on the device)
                .getRecognizer();
        recognizer.addListener(this);

        /** In your application you might not need to add all those searches.
         * They are added here for demonstration. You can leave just one.
         */

        // Create keyword-activation search.
        recognizer.addKeyphraseSearch(KWS_SEARCH, KEYPHRASE);

        File menuGrammar = new File(assetsDir, "menu.gram");
        recognizer.addGrammarSearch(MENU_SEARCH, menuGrammar);
        // Next search for digits
        File digitsGrammar = new File(assetsDir, "digits.gram");
        recognizer.addGrammarSearch(DIGITS_SEARCH, digitsGrammar);

        /*recognizer.addKeyphraseSearch(AMRANE, KEYPHRASE);
        recognizer.addKeyphraseSearch(SOS, KEYPHRASE);
        recognizer.addKeyphraseSearch(TEST, KEYPHRASE);
        recognizer.addKeyphraseSearch(SOS, KEYPHRASE);*/
    }


    @Override
    public void onRequestPermissionsResult(int requestCode,
                                           String[] permissions, int[] grantResults) {
        super.onRequestPermissionsResult(requestCode, permissions, grantResults);

        if (requestCode == PERMISSIONS_REQUEST_RECORD_AUDIO) {
            if (grantResults.length > 0 && grantResults[0] == PackageManager.PERMISSION_GRANTED) {
                runRecognizerSetup();
            } else {
                finish();
            }
        }
    }

    @Override
    public void onDestroy(){
        super.onDestroy();
        if (recognizer != null) {
            recognizer.cancel();
            recognizer.shutdown();
        }
    }

    /**
     * Receiving speech input
     * */
    @Override
    protected void onActivityResult(int requestCode, int resultCode, Intent data) {
        super.onActivityResult(requestCode, resultCode, data);

    }

    @Override
    public void onBeginningOfSpeech() {

    }

    @Override
    public void onEndOfSpeech() {
        if (!recognizer.getSearchName().equals(KWS_SEARCH)){
            makeText(getApplicationContext(), "onENdofSpeech : "+recognizer.getSearchName(), Toast.LENGTH_SHORT).show();
            switchSearch(KWS_SEARCH);
        }
    }

    @Override
    public void onPartialResult(Hypothesis hypothesis) {
        if (hypothesis == null)
            return;

        String text = hypothesis.getHypstr();
        makeText(getApplicationContext(), "Result : "+text, Toast.LENGTH_SHORT).show();
        speechtoText.setText(text);
        if (text.equals(KEYPHRASE))
            switchSearch(MENU_SEARCH);

        else if (text.equals(DIGITS_SEARCH))
            switchSearch(DIGITS_SEARCH);

        else if (text.equals(TEST))
            switchSearch(TEST);

        else if (text.equals(HELLO))
            switchSearch(HELLO);

        else if (text.equals(AMRANE))
            switchSearch(AMRANE);

        else if (text.equals(SOS))
            switchSearch(SOS);

        else
            speechtoText.setText(text);
    }

    @Override
    public void onResult(Hypothesis hypothesis) {
        speechtoText.setText("");
        if (hypothesis != null) {
            String text = hypothesis.getHypstr();
            makeText(getApplicationContext(), text, Toast.LENGTH_SHORT).show();
            speechtoText.setText(text);
        }
    }

    @Override
    public void onError(Exception e) {
        speechtoText.setText(e.getMessage());

    }

    @Override
    public void onTimeout() {
        switchSearch(KEYPHRASE);
    }
}

and here is my dictionnary

android AE N DH ER AO IH DH
amrane AE M R EY N
amrane(2) AA M ER AE N
sos S AO S
sos(2) EH OW EH S
test T EH S T
hello HH EH L OW
digits D IH JH AH T S
digits(2) D IH JH IH T S
menu M EH N Y UW
menus M EH N Y UW Z
oh OW
zero Z IH R OW
zero(2) Z IY R OW
one W AH N
one(2) HH W AH N
two T UW
three TH R IY
four F AO R
five F AY V
six S IH K S
seven S EH V AH N
eight EY TH
nine N AY N
wakeup W EY K AH P

[Nickolay V. Shmyrev]

In your code you never add "AMRANE" search, the relevant part is commented out, so it can not switch to non-existent search.

You first need to decide which searches do you want to have in your application.

[Amran Zeouay]

What I want is that the application will hear some words like 8 words and for each word an acitivity will open. so that's why I want to know if there is a way to use your framework only for detecting words not like i need to saythe keyphrase and then i need to say something

[Nickolay V. Shmyrev]

I would implement an activation keyphrase and then an additional keyphrase search for 8 words you want to detect.