Menu

French language on PocketSphinx Android demo master

Help
2018-03-22
2018-03-22
  • AlexLutinNoir

    AlexLutinNoir - 2018-03-22

    Hello there,
    I'm trying to create an android application with voice recognition using pocketsphinx android demo, to simplfy my application, I'm only trying the words recogition (Like in the "DEGITS" demo) the application works fine with it's default language English-US.
    I want to add French to my application, so I can use both English and French recognition, so for testing purpose I'm trying to start the words recognition for French language as it is already a supported language.
    I downloaded the:
    1)cmusphinx-fr-ptm-8khz-5.2.tar.gz language model, and I'm not sure it is the correct file for my android application.
    2)I also downloaded fr.dict since I'm only using words recognition.
    I simply copied and pasted the downloaded files to my android project assets folder, and in my application, the MD5 files were generated since I have the sync assets already integrated in my application .
    here is the link of my downloads:
    in my java code I simply replaced and renamed the files like so:

     private void setupRecognizer(File assetsDir) throws IOException {
            recognizer = SpeechRecognizerSetup.defaultSetup()
                    .setAcousticModel(new File(assetsDir, "fr-fr-ptm"))
                    .setDictionary(new File(assetsDir, "cmudict-fr-fr.dict"))
                    .setRawLogDir(assetsDir) // To disable logging of raw audio comment out this call (takes a lot of space on the device)
                    .getRecognizer();
            recognizer.addListener(this);
    

    For the grammar, I created a new .gram file with 3 number words in French that exist in the dictionary like so:

    #JSGF V1.0;
    grammar numeros;
    <numero> = un | deux | trois;
    public <numeros> = <numero>+;
    

    then simply replaced the digits.gram with my numero.gram in my java code:

    File digitsGrammar = new File(assetsDir, "numeros.gram");
            recognizer.addGrammarSearch(DIGITS_SEARCH, digitsGrammar);
    

    After all these, I tried to run my application and it just crashes all the time I start the recognition here is the run log:

    03/22 13:06:58: Launching PocketSphinxActivity
    $ adb push C:\Users\Administrateur\AndroidStudioProjects\pocketsphinx-android-demo-master\app\build\outputs\apk\debug\app-debug.apk /data/local/tmp/edu.cmu.sphinx.pocketsphinx
    $ adb shell pm install -t -r "/data/local/tmp/edu.cmu.sphinx.pocketsphinx"
        pkg: /data/local/tmp/edu.cmu.sphinx.pocketsphinx
    Success
    
    
    $ adb shell am start -n "edu.cmu.sphinx.pocketsphinx/edu.cmu.pocketsphinx.demo.PocketSphinxActivity" -a android.intent.action.MAIN -c android.intent.category.LAUNCHER
    Connected to process 14706 on device sony-d2502-YT9114ZHWZ
    Capturing and displaying logcat messages from application. This behavior can be disabled in the "Logcat output" section of the "Debugger" settings page.
    W/ContextImpl: Failed to ensure directory: /storage/sdcard1/Android/data/edu.cmu.sphinx.pocketsphinx/files
    I/Assets: Skipping asset cmudict-en-us.dict: checksums are equal
    I/Assets: Skipping asset en-us-ptm/transition_matrices: checksums are equal
    I/Assets: Skipping asset fr-fr-ptm/sendump: checksums are equal
    I/Assets: Skipping asset en-us-ptm/mdef: checksums are equal
    I/Assets: Skipping asset en-us-ptm/README: checksums are equal
    I/Assets: Skipping asset en-us-ptm/sendump: checksums are equal
    I/Assets: Skipping asset cmudict-fr-fr.dict: checksums are equal
    I/Assets: Skipping asset en-us-ptm/means: checksums are equal
    I/Assets: Skipping asset menu.gram: checksums are equal
    I/Assets: Skipping asset fr-fr-ptm/mixture_weights: checksums are equal
    I/Assets: Skipping asset en-us-ptm/feat.params: checksums are equal
    I/Assets: Skipping asset fr-fr-ptm/transition_matrices: checksums are equal
    I/Assets: Skipping asset fr-fr-ptm/variances: checksums are equal
    I/Assets: Skipping asset fr-fr-ptm/README: checksums are equal
    I/Assets: Skipping asset fr-fr-ptm/feat.params: checksums are equal
    I/Assets: Skipping asset weather.dmp: checksums are equal
    I/Assets: Skipping asset fr-fr-ptm/mdef: checksums are equal
    I/Assets: Skipping asset fr-fr-ptm/noisedict: checksums are equal
    I/Assets: Skipping asset en-us-ptm/noisedict: checksums are equal
    I/Assets: Skipping asset en-phone.dmp: checksums are equal
    I/Assets: Skipping asset digits.gram: checksums are equal
    I/Assets: Skipping asset en-us-ptm/variances: checksums are equal
    I/Assets: Skipping asset fr-fr-ptm/means: checksums are equal
    I/Assets: Copying asset numeros.gram to /storage/emulated/0/Android/data/edu.cmu.sphinx.pocketsphinx/files/sync/numeros.gram
    W/linker: libpocketsphinx_jni.so: unused DT entry: type 0x6ffffffe arg 0xe3c0
    W/linker: libpocketsphinx_jni.so: unused DT entry: type 0x6fffffff arg 0x2
    I/cmusphinx: INFO: pocketsphinx.c(152): Parsed model-specific feature parameters from /storage/emulated/0/Android/data/edu.cmu.sphinx.pocketsphinx/files/sync/fr-fr-ptm/feat.params
    D/OpenGLRenderer: Use EGL_SWAP_BEHAVIOR_PRESERVED: true
    D/Atlas: Validating map...
    I/cmusphinx: INFO: feat.c(715): Initializing feature stream to type: '1s_c_d_dd', ceplen=13, CMN='batch', VARNORM='no', AGC='none'
    I/cmusphinx: INFO: acmod.c(162): Using subvector specification 0-12/13-25/26-38
    I/cmusphinx: INFO: mdef.c(518): Reading model definition: /storage/emulated/0/Android/data/edu.cmu.sphinx.pocketsphinx/files/sync/fr-fr-ptm/mdef
    I/Adreno-EGL: <qeglDrvAPI_eglInitialize:410>: EGL 1.4 QUALCOMM build: AU_LINUX_ANDROID_LA.BF.1.1.1_RB1.05.01.00.042.030_msm8226_LA.BF.1.1.1_RB1__release_AU ()
                  OpenGL ES Shader Compiler Version: E031.25.03.06
                  Build Date: 06/10/15 Wed
                  Local Branch: 
                  Remote Branch: quic/LA.BF.1.1.1_rb1.24
                  Local Patches: NONE
                  Reconstruct Branch: AU_LINUX_ANDROID_LA.BF.1.1.1_RB1.05.01.00.042.030 + 6151be1 +  NOTHING
    I/OpenGLRenderer: Initialized EGL, version 1.4
    D/OpenGLRenderer: Enabling debug mode 0
    I/Timeline: Timeline: Activity_idle id: android.os.BinderProxy@3472ce05 time:334411884
    I/cmusphinx: INFO: bin_mdef.c(181): Allocating 101051 * 8 bytes (789 KiB) for CD tree
    D/AndroidRuntime: Shutting down VM
    E/AndroidRuntime: FATAL EXCEPTION: main
                      Process: edu.cmu.sphinx.pocketsphinx, PID: 14706
                      java.lang.NullPointerException: Attempt to invoke virtual method 'boolean edu.cmu.pocketsphinx.SpeechRecognizer.startListening(java.lang.String, int)' on a null object reference
                          at edu.cmu.pocketsphinx.demo.PocketSphinxActivity.switchSearch(PocketSphinxActivity.java:202)
                          at edu.cmu.pocketsphinx.demo.PocketSphinxActivity.access$000(PocketSphinxActivity.java:59)
                          at edu.cmu.pocketsphinx.demo.PocketSphinxActivity$1.onClick(PocketSphinxActivity.java:81)
                          at android.view.View.performClick(View.java:4856)
                          at android.view.View$PerformClick.run(View.java:19956)
                          at android.os.Handler.handleCallback(Handler.java:739)
                          at android.os.Handler.dispatchMessage(Handler.java:95)
                          at android.os.Looper.loop(Looper.java:211)
                          at android.app.ActivityThread.main(ActivityThread.java:5371)
                          at java.lang.reflect.Method.invoke(Native Method)
                          at java.lang.reflect.Method.invoke(Method.java:372)
                          at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:945)
                          at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:740)
    I/cmusphinx: INFO: tmat.c(149): Reading HMM transition probability matrices: /storage/emulated/0/Android/data/edu.cmu.sphinx.pocketsphinx/files/sync/fr-fr-ptm/transition_matrices
    I/cmusphinx: INFO: acmod.c(113): Attempting to use PTM computation module
    I/cmusphinx: INFO: ms_gauden.c(127): Reading mixture gaussian parameter: /storage/emulated/0/Android/data/edu.cmu.sphinx.pocketsphinx/files/sync/fr-fr-ptm/means
    I/cmusphinx: INFO: ms_gauden.c(242): 36 codebook, 3 feature, size: 
    I/cmusphinx: INFO: ms_gauden.c(244):  128x13
    I/cmusphinx: INFO: ms_gauden.c(244):  128x13
    I/cmusphinx: INFO: ms_gauden.c(244):  128x13
    I/cmusphinx: INFO: ms_gauden.c(127): Reading mixture gaussian parameter: /storage/emulated/0/Android/data/edu.cmu.sphinx.pocketsphinx/files/sync/fr-fr-ptm/variances
    I/cmusphinx: INFO: ms_gauden.c(242): 36 codebook, 3 feature, size: 
    I/cmusphinx: INFO: ms_gauden.c(244):  128x13
    I/cmusphinx: INFO: ms_gauden.c(244):  128x13
    I/cmusphinx: INFO: ms_gauden.c(244):  128x13
    I/cmusphinx: INFO: ms_gauden.c(304): 65 variance values floored
    I/cmusphinx: INFO: ptm_mgau.c(476): Loading senones from dump file /storage/emulated/0/Android/data/edu.cmu.sphinx.pocketsphinx/files/sync/fr-fr-ptm/sendump
    I/cmusphinx: INFO: ptm_mgau.c(500): BEGIN FILE FORMAT DESCRIPTION
    I/cmusphinx: INFO: ptm_mgau.c(563): Rows: 128, Columns: 2108
    I/cmusphinx: INFO: ptm_mgau.c(595): Using memory-mapped I/O for senones
    I/cmusphinx: INFO: ptm_mgau.c(838): Maximum top-N: 4
    I/cmusphinx: INFO: phone_loop_search.c(114): State beam -225 Phone exit beam -225 Insertion penalty 0
    I/cmusphinx: INFO: dict.c(320): Allocating 109102 * 20 bytes (2130 KiB) for word entries
    I/cmusphinx: INFO: dict.c(333): Reading main dictionary: /storage/emulated/0/Android/data/edu.cmu.sphinx.pocketsphinx/files/sync/cmudict-fr-fr.dict
    I/cmusphinx: INFO: dict.c(213): Dictionary size 105003, allocated 1018 KiB for strings, 1375 KiB for phones
    I/cmusphinx: INFO: dict.c(336): 105003 words read
    I/cmusphinx: INFO: dict.c(358): Reading filler dictionary: /storage/emulated/0/Android/data/edu.cmu.sphinx.pocketsphinx/files/sync/fr-fr-ptm/noisedict
    I/cmusphinx: INFO: dict.c(213): Dictionary size 105006, allocated 0 KiB for strings, 0 KiB for phones
    I/cmusphinx: INFO: dict.c(361): 3 words read
    I/cmusphinx: INFO: dict2pid.c(396): Building PID tables for dictionary
    I/cmusphinx: INFO: dict2pid.c(406): Allocating 36^3 * 2 bytes (91 KiB) for word-initial triphones
    I/cmusphinx: INFO: dict2pid.c(132): Allocated 15696 bytes (15 KiB) for word-final triphones
    I/cmusphinx: INFO: dict2pid.c(196): Allocated 15696 bytes (15 KiB) for single-phone word triphones
    I/SpeechRecognizer: Load JSGF /storage/emulated/0/Android/data/edu.cmu.sphinx.pocketsphinx/files/sync/numeros.gram
    E/cmusphinx: ERROR: "jsgf_parser.y", line 155: syntax error, unexpected '=', expecting ';' or '|' at line 3 current token '='
    E/cmusphinx: ERROR: "jsgf.c", line 924: Failed to parse JSGF grammar from '/storage/emulated/0/Android/data/edu.cmu.sphinx.pocketsphinx/files/sync/numeros.gram'
    Application terminated.
    

    I also got the error saying that the Fr-fr.dict is 3225097 bytes which exeeds the 2560000 bytes and that the code insight features are not available.
    In the log file it says grammar error while the syntaxe is the same found in the the "DIGITS" .gram.
    Please consider my issue and help me solving it out.
    Thank you for further replies.

     

    Last edit: AlexLutinNoir 2018-03-22
    • Nickolay V. Shmyrev

      It says your JSGF file is not properly formatted. You missed ; before teh second line, it should be like this:

      #JSGF V1.0;
      grammar numeros;
      <numero> = un | deux | trois ;
      public <numeros> = <numero>+;
      

      If you fix JSGF it might work

       
  • AlexLutinNoir

    AlexLutinNoir - 2018-03-22

    Yes I did miss that one ";" there and I realized that just after posting I'm sorry, it's working like a charm now thank you, but is there any light dictionary I can use ? or do I have to create one myself ?

     
    • Nickolay V. Shmyrev

      You can easily create it yourself.

       

Log in to post a comment.

Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.