CMU Sphinx

[ijain]

I have copied so file in libs folder and sync folder and assets.lst file in the assets.I also added
pocketsphinx-android-0.8-nolib.jar to the application.
I also added:
<uses-permission android:name="android.permission.RECEIVE_BOOT_COMPLETED">
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE">
permissions to the manifest file. </uses-permission></uses-permission>

Then I added code for set up SppechRecognizer as

        private void setUpRecognizer(){
           Assets assets = new Assets(context);
    File assetDir = assets.syncAssets();
            File modelsDir = new File(assetsDir, "models");
    recognizer = defaultSetup()
            .setAcousticModel(new File(assetsDir, "hmm/en-us-semi"))
            .setDictionary(new File(assetsDir, "dict/cmu07a.dic"))
            .setRawLogDir(assetsDir).setKeywordThreshold(1e-40f)
            .getRecognizer();
    recognizer.addListener(this);
    File modelsDir = new File(assetsDir, "models");
    recognizer = defaultSetup()
            .setAcousticModel(new File(assetsDir, "hmm/en-us-semi"))
            .setDictionary(new File(assetsDir, "dict/cmu07a.dic"))
            .setRawLogDir(assetsDir).setKeywordThreshold(1e-40f)
            .getRecognizer();
    recognizer.addListener(this);
    recognizer.addKeyphraseSearch(KWS_SEARCH, KEYPHRASE);
    File menuGrammer = new File(modelsDir, "grammar/menu.gram");
    recognizer.addGrammarSearch(MENU_SEARCH, menuGrammer);
    File digitGrammer = new File(modelsDir, "grammar/digits.gram");
    recognizer.addGrammarSearch(DIGITS_SEARCH, digitGrammer);
    File languageModel = new File(modelsDir, "lm/weather.dmp");
    recognizer.addNgramSearch(FORECAST_SEARCH, languageModel);
            }

After running the application, I am getting the exception:

08-26 13:42:38.301: E/cmusphinx(14405): ERROR: "acmod.c", line 90: Folder '/storage/sdcard0/Android/data/com.perscitussln.sherbet/files/sync/hmm/en-us-semi' does not contain acoustic model definition 'mdef'

Can anyone help me to solve this problem or give me some advice.

[Alexander Solovets]

http://cmusphinx.sourceforge.net/wiki/tutorialandroid#including_resource_files.

[ijain]

Hi Thanks for reply, but the demo is running is absolutely fine. This error is coming when I include the library in my application.

[Alexander Solovets]

I pointed out to a concrete section - "Including resource files". Read it thoroughly.

[ijain]

Hi, I am using eclipse for building the project. Can you tell how to build script using ant so that it can create md5 file of the assets? I have ant binary and my eclipse doesnot have ant plugin.

[Alexander Solovets]

Apparently you need to install the Ant plugin for your version of Eclipse.

[ijain]

Actually the problem is with assets file sync. So can you tell me the procedure to create md5 and assets.lst files?

Actually the package name of my application is also different from edu.cmu... . Can I use asset files by just copying the sync folder from demo application.

[Nickolay V. Shmyrev]

Hello ijain

It doesn't matter which package name you use. To properly sync asset you need to use the asset builder. To do that you need the following:

1. Install Java Development Tools bundle in you Eclipse
2. Copy assets.xml file from demo to your project
3. Copy .externalToolBuilders folder to your project from the demo
4. Make sure Eclipse uses external tool builder in project properties (Builders section)
5. If you incorporate external builder into your project everything should work properly.

[ijain]

Thanks for reply but after adding external folder to my project I am getting the following error in the ecipse:

Errors occurred during the build.
Errors running builder 'Integrated External Tool Builder' on project 'AppName'.
Launch configuration type id "org.eclipse.ant.AntBuilderLaunchConfigurationType" does not exist.
Possible causes:
Missing specification of a launch type (missing plug-in)
Incorrect launch configuration XML
Launch configuration type id "org.eclipse.ant.AntBuilderLaunchConfigurationType" does not exist.
Possible causes:
Missing specification of a launch type (missing plug-in)
Incorrect launch configuration XML

I also tried with adding ant plugin to eclipse.

Can you tell me how to run asset "Asset List Builder.launch" in .externalToolBuilders folder without using eclipse, or is ther any help regarding eclipse IDE.

[Nickolay V. Shmyrev]

Launch configuration type id "org.eclipse.ant.AntBuilderLaunchConfigurationType" does not exist.

This error means you don't have Java Development Toolkit installed in your eclipse. I wrote you above that you need to install it. Tutorial also tells you need to install it. So just install it and launcher will work properly.

[Nickolay V. Shmyrev]

go to Help→Install New Software. Select Eclipse Repository and install from Programming Languages section “Eclipse Java Development Tools”.

[ijain]

Thanks Nickolay for your help, I am able to run assets.xml now. But the thing is when I am running this code in my application:

            recognizer = defaultSetup()
            .setAcousticModel(new File(assetsDir, "hmm/en-us-semi"))
            .setDictionary(new File(assetsDir, "dict/cmu07a.dic"))
            .setRawLogDir(assetsDir).setKeywordThreshold(1e-40f)
            .getRecognizer();
    recognizer.addListener(this);

I am getting the same exception as:

"08-27 17:08:06.923: E/cmusphinx(28129): ERROR: "acmod.c", line 90: Folder '/storage/sdcard0/Android/data/com.example.sherbet/files/sync/hmm/en-us-semi' does not contain acoustic model definition 'mdef' ".

So the problem is still not solved.

[Nickolay V. Shmyrev]

Before creating the recognizer you need to sync assets. You can do that with

~~~~~~~~~~~
Assets assets = new Assets(PocketSphinxActivity.this);
File assetDir = assets.syncAssets();
~~~~~~~~~~

See the demo for details

[ijain]

Hi, Nickolay I did not posted the code but I am doing that.

Assets assets = new Assets(context);
File assetDir = assets.syncAssets();

still the problem persists.

[ijain]

If you want to have a look then you can see the above code.

[Nickolay V. Shmyrev]

Probably files are corrupted somehow. You can do the following:

1) Delete files from sdcard and make it sync again
2) Make sure your original files in assets folder is not corrupted
3) Provide full logcat output, not just the last line.

[ijain]

Hi, tried above steps: Application gave following logs:

08-27 18:53:36.878: I/Assets(4405): Copying asset models/grammar/menu.gram to /storage/sdcard0/Android/data/com.perscitussln.sherbet/files/sync/models/grammar/menu.gram
08-27 18:53:37.558: I/Assets(4405): Copying asset models/hmm/en-us-semi/sendump to /storage/sdcard0/Android/data/com.perscitussln.sherbet/files/sync/models/hmm/en-us-semi/sendump
08-27 18:53:38.848: I/Assets(4405): Copying asset models/hmm/en-us-semi/mdef to /storage/sdcard0/Android/data/com.perscitussln.sherbet/files/sync/models/hmm/en-us-semi/mdef
08-27 18:53:38.848: I/Assets(4405): Copying asset models/hmm/en-us-semi/README to /storage/sdcard0/Android/data/com.perscitussln.sherbet/files/sync/models/hmm/en-us-semi/README
08-27 18:53:38.858: I/Assets(4405): Copying asset models/grammar/digits.gram to /storage/sdcard0/Android/data/com.perscitussln.sherbet/files/sync/models/grammar/digits.gram
08-27 18:53:38.858: I/Assets(4405): Copying asset models/hmm/en-us-semi/feat.params to /storage/sdcard0/Android/data/com.perscitussln.sherbet/files/sync/models/hmm/en-us-semi/feat.params
08-27 18:53:38.868: I/Assets(4405): Copying asset models/hmm/en-us-semi/means to /storage/sdcard0/Android/data/com.perscitussln.sherbet/files/sync/models/hmm/en-us-semi/means
08-27 18:53:38.878: I/Assets(4405): Copying asset models/hmm/en-us-semi/noisedict to /storage/sdcard0/Android/data/com.perscitussln.sherbet/files/sync/models/hmm/en-us-semi/noisedict
08-27 18:53:39.008: I/Assets(4405): Copying asset models/lm/weather.dmp to /storage/sdcard0/Android/data/com.perscitussln.sherbet/files/sync/models/lm/weather.dmp
08-27 18:53:39.018: I/Assets(4405): Copying asset models/hmm/en-us-semi/variances to /storage/sdcard0/Android/data/com.perscitussln.sherbet/files/sync/models/hmm/en-us-semi/variances
08-27 18:53:39.018: I/Assets(4405): Copying asset models/hmm/en-us-semi/transition_matrices to /storage/sdcard0/Android/data/com.perscitussln.sherbet/files/sync/models/hmm/en-us-semi/transition_matrices
08-27 18:53:39.708: I/Assets(4405): Copying asset models/dict/cmu07a.dic to /storage/sdcard0/Android/data/com.perscitussln.sherbet/files/sync/models/dict/cmu07a.dic
08-27 18:53:39.728: D/dalvikvm(4405): Trying to load lib /data/data/com.perscitussln.sherbet/lib/libpocketsphinx_jni.so 0x417340c8
08-27 18:53:39.728: D/dalvikvm(4405): Added shared lib /data/data/com.perscitussln.sherbet/lib/libpocketsphinx_jni.so 0x417340c8
08-27 18:53:39.728: D/dalvikvm(4405): No JNI_OnLoad found in /data/data/com.perscitussln.sherbet/lib/libpocketsphinx_jni.so 0x417340c8, skipping init
08-27 18:53:39.728: I/cmusphinx(4405): INFO: cmd_ln.c(696): Parsing command line:
08-27 18:53:39.728: I/cmusphinx(4405): Current configuration:
08-27 18:53:39.788: I/cmusphinx(4405): INFO: feat.c(715): Initializing feature stream to type: '1s_c_d_dd', ceplen=13, CMN='current', VARNORM='no', AGC='none'
08-27 18:53:39.788: I/cmusphinx(4405): INFO: cmn.c(143): mean[0]= 12.00, mean[1..12]= 0.0

""08-27 18:53:39.788: E/cmusphinx(4405): ERROR: "acmod.c", line 90: Folder '/storage/sdcard0/Android/data/com.perscitussln.sherbet/files/sync/hmm/en-us-semi' does not contain acoustic model definition 'mdef'"

08-27 18:53:39.788: I/SpeechRecognizer(4405): Load JSGF /storage/sdcard0/Android/data/com.perscitussln.sherbet/files/sync/models/grammar/digits.gram
08-27 18:53:39.788: I/cmusphinx(4405): INFO: jsgf.c(697): Defined rule: <digits.digit>
08-27 18:53:39.798: I/cmusphinx(4405): INFO: jsgf.c(697): Defined rule: <digits.g00001>
08-27 18:53:39.798: I/cmusphinx(4405): INFO: jsgf.c(697): Defined rule: PUBLIC <digits.digits></digits.digits></digits.g00001></digits.digit>

"08-27 18:53:39.798: A/libc(4405): Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread 4510 (AsyncTask #1)"

08-27 18:53:40.358: E/Trace(4536): error opening trace file: No such file or directory (2)
08-27 18:53:40.368: D/ActivityThread(4536): setTargetHeapUtilization:0.25
08-27 18:53:40.368: D/ActivityThread(4536): setTargetHeapIdealFree:8388608
08-27 18:53:40.368: D/ActivityThread(4536): setTargetHeapConcurrentStart:2097152
08-27 18:53:40.538: D/dalvikvm(4536): GC_FOR_ALLOC freed 88K, 36% free 12836K/19779K, paused 18ms, total 19ms
08-27 18:53:40.548: I/dalvikvm-heap(4536): Grow heap (frag case) to 28.615MB for 7680016-byte allocation
08-27 18:53:40.608: D/dalvikvm(4536): GC_CONCURRENT freed 1K, 26% free 20335K/27335K, paused 12ms+3ms, total 51ms
08-27 18:53:40.678: D/libEGL(4536): loaded /system/lib/egl/libEGL_adreno200.so
08-27 18:53:40.678: D/libEGL(4536): loaded /system/lib/egl/libGLESv1_CM_adreno200.so
08-27 18:53:40.688: D/libEGL(4536): loaded /system/lib/egl/libGLESv2_adreno200.so
08-27 18:53:40.688: I/Adreno200-EGL(4536): <qegldrvapi_eglinitialize:299>: EGL 1.4 QUALCOMM build: (Merge)
08-27 18:53:40.688: I/Adreno200-EGL(4536): Build Date: 07/09/13 Tue
08-27 18:53:40.688: I/Adreno200-EGL(4536): Local Branch: AU_41
08-27 18:53:40.688: I/Adreno200-EGL(4536): Remote Branch:
08-27 18:53:40.688: I/Adreno200-EGL(4536): Local Patches:
08-27 18:53:40.688: I/Adreno200-EGL(4536): Reconstruct Branch:
08-27 18:53:40.738: D/OpenGLRenderer(4536): Enabling debug mode 0</qegldrvapi_eglinitialize:299>

[Nickolay V. Shmyrev]

Ok, if you study the log, you'll see a difference between two lines:

  08-27 18:53:38.848: I/Assets(4405): Copying asset models/hmm/en-us-semi/mdef to /storage/sdcard0/Android/data/com.perscitussln.sherbet/files/sync/models/hmm/en-us-semi/mdef

and

  08-27 18:53:39.788: E/cmusphinx(4405): ERROR: "acmod.c", line 90: Folder '/storage/sdcard0/Android/data/com.perscitussln.sherbet/files/sync/hmm/en-us-semi' does not contain acoustic model definition 'mdef'"

The second path miss 'models' folder after sync folder. So there is no such file indeed. To fix your issue you need to fix your code:

  recognizer = defaultSetup()
        .setAcousticModel(new File(assetsDir, "hmm/en-us-semi"))

to

 recognizer = defaultSetup()
        .setAcousticModel(new File(assetsDir, "models/hmm/en-us-semi"))

[ijain]

Thanks a lot Nickolay, Actually I have implemted the same code as given in demo project. THe issue was same that it was taking the wrong path and the issue has been solved now.

[ijain]

Hi, Nickolay I am able to implement the required functionality but the thing is that the functionality is not optimized. The files recorded using phone and tablet seems to be noisy. It seems to be very sensitive to background noise and gives patial hypothesis even I don't speak. how can I reduce the sensitivity of the noise and improve the accuracy to the level of the recognition.

Note: The optimization is very important beacuse I am creating an android application in which tha app plays videos, so while testing the functionality, the app takes the voice command even there is voice from video also and it is not needed. The app should takes voice command from person only.

[Nickolay V. Shmyrev]

It's almost impossible to recognize commands if they are mixed with other audio. If you have access to audio data you play you can subtract it from recording to cancel it, you can get an idea here:

http://sound.stackexchange.com/questions/23374/subtracting-specific-audio-using-an-isolated-track-from-the-same-mix

If you cancel playback this way you should get pretty good results.

For continuous listening demo uses keyword spotting mode, you can see how it works.

[ijain]

Hi, How can I reduce the sensitivity of the recognizer?