Menu

Changed Pocketsphinx on Android

Help
2015-03-19
2015-05-07
  • Nikita Furin

    Nikita Furin - 2015-03-19

    Hello, World!
    If I added several files in pocketsphinx, how can I compile it for Android?
    I don't understand.
    P.S. I want to put the compiled project in Unity.

     
    • Nickolay V. Shmyrev

      You can find pocketsphinx-android project in our repository

      You can list additional files to add to the library in jni/Android.mk.

       
  • Nikita Furin

    Nikita Furin - 2015-03-19

    Thanks for quick reply!
    I'm trying to build, but I'm getting error.
    History of my console commands:

    ant-jar
    

    BUILD FAILED: ... Cannot run program "swig": error=2

    sudo apt-get install swig2.0
    ant-jar
    

    BUILD FAILED: ... Cannot run program "swig": error=2

    sudo apt-get install swig2.0
    ant-jar
    
    Buildfile: /home/nikita/Исходники/pocketsph/pocketsphinx-android/build.xml
    
    swig:
    
    compile:
        [mkdir] Created dir: /home/nikita/Исходники/pocketsph/pocketsphinx-android/build/classes
        [javac] Compiling 43 source files to /home/nikita/Исходники/pocketsph/pocketsphinx-android/build/classes
        [javac] /home/nikita/Исходники/pocketsph/pocketsphinx-android/src/edu/cmu/pocketsphinx/Assets.java:51: error: package android.content does not exist
        [javac] import android.content.Context;
        [javac]                       ^
        [javac] /home/nikita/Исходники/pocketsph/pocketsphinx-android/src/edu/cmu/pocketsphinx/Assets.java:52: error: package android.content.res does not exist
        [javac] import android.content.res.AssetManager;
        [javac]                           ^
        [javac] /home/nikita/Исходники/pocketsph/pocketsphinx-android/src/edu/cmu/pocketsphinx/Assets.java:53: error: package android.os does not exist
        [javac] import android.os.Environment;
        [javac]                  ^
        [javac] /home/nikita/Исходники/pocketsph/pocketsphinx-android/src/edu/cmu/pocketsphinx/Assets.java:54: error: package android.util does not exist
        [javac] import android.util.Log;
        [javac]                    ^
        [javac] /home/nikita/Исходники/pocketsph/pocketsphinx-android/src/edu/cmu/pocketsphinx/Assets.java:79: error: cannot find symbol
        [javac]     private final AssetManager assetManager;
        [javac]                   ^
        [javac]   symbol:   class AssetManager
        [javac]   location: class Assets
        [javac] /home/nikita/Исходники/pocketsph/pocketsphinx-android/src/edu/cmu/pocketsphinx/Assets.java:94: error: cannot find symbol
        [javac]     public Assets(Context context) throws IOException {
        [javac]                   ^
        [javac]   symbol:   class Context
        [javac]   location: class Assets
        [javac] /home/nikita/Исходники/pocketsph/pocketsphinx-android/src/edu/cmu/pocketsphinx/Assets.java:111: error: cannot find symbol
        [javac]     public Assets(Context context, String dest) {
        [javac]                   ^
        [javac]   symbol:   class Context
        [javac]   location: class Assets
        [javac] /home/nikita/Исходники/pocketsph/pocketsphinx-android/src/edu/cmu/pocketsphinx/SpeechRecognizer.java:40: error: package android.media does not exist
        [javac] import android.media.AudioFormat;
        [javac]                     ^
        [javac] /home/nikita/Исходники/pocketsph/pocketsphinx-android/src/edu/cmu/pocketsphinx/SpeechRecognizer.java:41: error: package android.media does not exist
        [javac] import android.media.AudioRecord;
        [javac]                     ^
        [javac] /home/nikita/Исходники/pocketsph/pocketsphinx-android/src/edu/cmu/pocketsphinx/SpeechRecognizer.java:42: error: package android.media.MediaRecorder does not exist
        [javac] import android.media.MediaRecorder.AudioSource;
        [javac]                                   ^
        [javac] /home/nikita/Исходники/pocketsph/pocketsphinx-android/src/edu/cmu/pocketsphinx/SpeechRecognizer.java:43: error: package android.os does not exist
        [javac] import android.os.Handler;
        [javac]                  ^
        [javac] /home/nikita/Исходники/pocketsph/pocketsphinx-android/src/edu/cmu/pocketsphinx/SpeechRecognizer.java:44: error: package android.os does not exist
        [javac] import android.os.Looper;
        [javac]                  ^
        [javac] /home/nikita/Исходники/pocketsph/pocketsphinx-android/src/edu/cmu/pocketsphinx/SpeechRecognizer.java:45: error: package android.util does not exist
        [javac] import android.util.Log;
        [javac]                    ^
        [javac] /home/nikita/Исходники/pocketsph/pocketsphinx-android/src/edu/cmu/pocketsphinx/SpeechRecognizer.java:63: error: cannot find symbol
        [javac]     private final AudioRecord recorder;
        [javac]                   ^
        [javac]   symbol:   class AudioRecord
        [javac]   location: class SpeechRecognizer
        [javac] /home/nikita/Исходники/pocketsph/pocketsphinx-android/src/edu/cmu/pocketsphinx/SpeechRecognizer.java:67: error: cannot find symbol
        [javac]     private final Handler mainHandler = new Handler(Looper.getMainLooper());
        [javac]                   ^
        [javac]   symbol:   class Handler
        [javac]   location: class SpeechRecognizer
        [javac] /home/nikita/Исходники/pocketsph/pocketsphinx-android/src/edu/cmu/pocketsphinx/Assets.java:98: error: cannot find symbol
        [javac]                     + "external storage state is " + Environment.getExternalStorageState());
        [javac]                                                      ^
        [javac]   symbol:   variable Environment
        [javac]   location: class Assets
        [javac] /home/nikita/Исходники/pocketsph/pocketsphinx-android/src/edu/cmu/pocketsphinx/Assets.java:260: error: cannot find symbol
        [javac]                 Log.i(TAG,
        [javac]                 ^
        [javac]   symbol:   variable Log
        [javac]   location: class Assets
        [javac] /home/nikita/Исходники/pocketsph/pocketsphinx-android/src/edu/cmu/pocketsphinx/Assets.java:270: error: cannot find symbol
        [javac]             Log.i(TAG, String.format("Copying asset %s to %s", path, file));
        [javac]             ^
        [javac]   symbol:   variable Log
        [javac]   location: class Assets
        [javac] /home/nikita/Исходники/pocketsph/pocketsphinx-android/src/edu/cmu/pocketsphinx/Assets.java:276: error: cannot find symbol
        [javac]             Log.i(TAG, String.format("Removing asset %s", file));
        [javac]             ^
        [javac]   symbol:   variable Log
        [javac]   location: class Assets
        [javac] /home/nikita/Исходники/pocketsph/pocketsphinx-android/src/edu/cmu/pocketsphinx/SpeechRecognizer.java:67: error: cannot find symbol
        [javac]     private final Handler mainHandler = new Handler(Looper.getMainLooper());
        [javac]                                             ^
        [javac]   symbol:   class Handler
        [javac]   location: class SpeechRecognizer
        [javac] /home/nikita/Исходники/pocketsph/pocketsphinx-android/src/edu/cmu/pocketsphinx/SpeechRecognizer.java:67: error: cannot find symbol
        [javac]     private final Handler mainHandler = new Handler(Looper.getMainLooper());
        [javac]                                                     ^
        [javac]   symbol:   variable Looper
        [javac]   location: class SpeechRecognizer
        [javac] /home/nikita/Исходники/pocketsph/pocketsphinx-android/src/edu/cmu/pocketsphinx/SpeechRecognizer.java:82: error: cannot find symbol
        [javac]         recorder = new AudioRecord(
        [javac]                        ^
        [javac]   symbol:   class AudioRecord
        [javac]   location: class SpeechRecognizer
        [javac] /home/nikita/Исходники/pocketsph/pocketsphinx-android/src/edu/cmu/pocketsphinx/SpeechRecognizer.java:83: error: cannot find symbol
        [javac]                 AudioSource.VOICE_RECOGNITION, sampleRate,
        [javac]                 ^
        [javac]   symbol:   variable AudioSource
        [javac]   location: class SpeechRecognizer
        [javac] /home/nikita/Исходники/pocketsph/pocketsphinx-android/src/edu/cmu/pocketsphinx/SpeechRecognizer.java:84: error: cannot find symbol
        [javac]                 AudioFormat.CHANNEL_IN_MONO,
        [javac]                 ^
        [javac]   symbol:   variable AudioFormat
        [javac]   location: class SpeechRecognizer
        [javac] /home/nikita/Исходники/pocketsph/pocketsphinx-android/src/edu/cmu/pocketsphinx/SpeechRecognizer.java:85: error: cannot find symbol
        [javac]                 AudioFormat.ENCODING_PCM_16BIT, bufferSize * 2);
        [javac]                 ^
        [javac]   symbol:   variable AudioFormat
        [javac]   location: class SpeechRecognizer
        [javac] /home/nikita/Исходники/pocketsph/pocketsphinx-android/src/edu/cmu/pocketsphinx/SpeechRecognizer.java:87: error: cannot find symbol
        [javac]         if (recorder.getState() == AudioRecord.STATE_UNINITIALIZED) {
        [javac]                                    ^
        [javac]   symbol:   variable AudioRecord
        [javac]   location: class SpeechRecognizer
        [javac] /home/nikita/Исходники/pocketsph/pocketsphinx-android/src/edu/cmu/pocketsphinx/SpeechRecognizer.java:121: error: cannot find symbol
        [javac]         Log.i(TAG, format("Start recognition \"%s\"", searchName));
        [javac]         ^
        [javac]   symbol:   variable Log
        [javac]   location: class SpeechRecognizer
        [javac] /home/nikita/Исходники/pocketsph/pocketsphinx-android/src/edu/cmu/pocketsphinx/SpeechRecognizer.java:140: error: cannot find symbol
        [javac]         Log.i(TAG, format("Start recognition \"%s\"", searchName));
        [javac]         ^
        [javac]   symbol:   variable Log
        [javac]   location: class SpeechRecognizer
        [javac] /home/nikita/Исходники/pocketsph/pocketsphinx-android/src/edu/cmu/pocketsphinx/SpeechRecognizer.java:172: error: cannot find symbol
        [javac]             Log.i(TAG, "Stop recognition");
        [javac]             ^
        [javac]   symbol:   variable Log
        [javac]   location: class SpeechRecognizer
        [javac] /home/nikita/Исходники/pocketsph/pocketsphinx-android/src/edu/cmu/pocketsphinx/SpeechRecognizer.java:188: error: cannot find symbol
        [javac]             Log.i(TAG, "Cancel recognition");
        [javac]             ^
        [javac]   symbol:   variable Log
        [javac]   location: class SpeechRecognizer
        [javac] /home/nikita/Исходники/pocketsph/pocketsphinx-android/src/edu/cmu/pocketsphinx/SpeechRecognizer.java:233: error: cannot find symbol
        [javac]         Log.i(TAG, format("Load JSGF %s", file));
        [javac]         ^
        [javac]   symbol:   variable Log
        [javac]   location: class SpeechRecognizer
        [javac] /home/nikita/Исходники/pocketsph/pocketsphinx-android/src/edu/cmu/pocketsphinx/SpeechRecognizer.java:246: error: cannot find symbol
        [javac]         Log.i(TAG, format("Load N-gram model %s", file));
        [javac]         ^
        [javac]   symbol:   variable Log
        [javac]   location: class SpeechRecognizer
        [javac] /home/nikita/Исходники/pocketsph/pocketsphinx-android/src/edu/cmu/pocketsphinx/SpeechRecognizer.java:313: error: cannot find symbol
        [javac]             if (recorder.getRecordingState() == AudioRecord.RECORDSTATE_STOPPED) {
        [javac]                                                 ^
        [javac]   symbol:   variable AudioRecord
        [javac]   location: class SpeechRecognizer.RecognizerThread
        [javac] /home/nikita/Исходники/pocketsph/pocketsphinx-android/src/edu/cmu/pocketsphinx/SpeechRecognizer.java:321: error: cannot find symbol
        [javac]             Log.d(TAG, "Starting decoding");
        [javac]             ^
        [javac]   symbol:   variable Log
        [javac]   location: class SpeechRecognizer.RecognizerThread
        [javac] 34 errors
    
    BUILD FAILED
    /home/nikita/Исходники/pocketsph/pocketsphinx-android/build.xml:32: Compile failed; see the compiler error output for details.
    
     
    • Nickolay V. Shmyrev

      You need to set sdk.path in build.properties. Check README file for details.

       
  • Nikita Furin

    Nikita Furin - 2015-03-20

    My build.properties:

    sdk.dir=/home/nikita/Android/Sdk
    ndk.dir=/home/nikita/ndk
    sdk.version=21.1.1
    

    It is in /home/nikita/Исходники/pocketsph/pocketsphinx-android

     
  • Nikita Furin

    Nikita Furin - 2015-03-20

    The error persists.

     
  • Nikita Furin

    Nikita Furin - 2015-03-20

    I changed sdk.version from 21.1.1 to 21. This error disappeared, but new error appeared.

    Buildfile: /home/nikita/source/pocketsph/pocketsphinx-android/build.xml
    
    swig:
    
    compile:
        [javac] Compiling 43 source files to /home/nikita/source/pocketsph/pocketsphinx-android/build/classes
    
    jni:
        [exec] [arm64-v8a] Compile        : pocketsphinx <= acmod.c
        ...
        [exec] [arm64-v8a] Compile        : sphinxutil <= err.c
        [exec] make: *** No rule to make target `../sphinxbase/src/libsphinxbase
    /util/errno.c', needed by `obj/local/arm64-v8a/objs/sphinxutil/errno.o'.  Stop.
    
    BUILD FAILED
    /home/nikita/source/pocketsph/pocketsphinx-android/build.xml:62: exec returned: 2
    
     
    • Nickolay V. Shmyrev

      Looks like you accidentally deleted errno.c, this file must be there. You can refresh your checkout or start from a clean one.

       
      • Nickolay V. Shmyrev

        Another thing is that you edited the file with windows editor and introduced windows-style newline in it. The build process didn't respect this change, that's why you have newline in file path here:

            [exec] make: *** No rule to make target `../sphinxbase/src/libsphinxbase
            /util/errno.c', needed by `obj/local/arm64-v8a/objs/sphinxutil/errno.o'.  Stop.
        

        Overall, please make sure you run from a clean checkout

         
  • Nikita Furin

    Nikita Furin - 2015-03-21

    Yes. There was not errno.c, because I did incorrectly checkout. Now, it's all right.
    How can I add my files in pocketsphinx for Android? I added mymodule.h in pocketsphinx/include and mymodule.c in pocketsphinx/src/libpocketsphinx.
    Next, I added "mymodule.c \" in Android.mk on line 139. It's in block:

    include $(CLEAR_VARS)
    LOCAL_C_INCLUDES := \
      $(SPHINX_PATH)/sphinxbase/include/android \
      $(SPHINX_PATH)/sphinxbase/include \
      $(SPHINX_PATH)/pocketsphinx/include
    LOCAL_CFLAGS += -DHAVE_CONFIG_H
    LOCAL_CFLAGS += -DANDROID_NDK
    
    LOCAL_PATH := $(SPHINX_PATH)/pocketsphinx/src/libpocketsphinx
    LOCAL_MODULE := pocketsphinx
    
    LOCAL_SRC_FILES := \
      acmod.c     \
      allphone_search.c \
      bin_mdef.c    \
      blkarray_list.c   \
      dict.c     \
      dict2pid.c    \
      fsg_history.c   \
      fsg_lextree.c   \
      fsg_search.c   \
      kws_detections.c \
      kws_search.c   \
      hmm.c.arm     \
      mdef.c     \
      mymodule.c \
      ms_gauden.c.arm    \
      ms_mgau.c.arm    \
      ms_senone.c.arm    \
      ngram_search.c   \
      ngram_search_fwdtree.c \
      ngram_search_fwdflat.c \
      phone_loop_search.c  \
      pocketsphinx.c \
      ps_lattice.c   \
      ps_mllr.c    \
      ptm_mgau.c.arm    \
      s2_semi_mgau.c.arm   \
      tmat.c     \
      vector.c
    
    include $(BUILD_STATIC_LIBRARY)
    

    Next, I executed

    ant jar
    

    (BUILD SUCCESSFUL)

    It, certainly, didn't work. How to make correctly?

     
  • Nickolay V. Shmyrev

    It, certainly, didn't work. How to make correctly?

    You did everything correctly, maybe you need to run "ant clean" to rebuild after makefile change.

     
  • Nikita Furin

    Nikita Furin - 2015-03-22

    But I can not find my functions in build/.

    nikita@nikita-N550JK ~/source/pocketsph/pocketsphinx-android/build $ find | grep -i mymodule
    nikita@nikita-N550JK ~/source/pocketsph/pocketsphinx-android/build $ grep -i mm_ classes/edu/cmu/pocketsphinx/*
    nikita@nikita-N550JK ~/source/pocketsph/pocketsphinx-android/build $ 
    
     
    • Nickolay V. Shmyrev

      I presume you do not need java wrapper part for Unity, you can just load library from C#.

      Java classes are created with SWIG. In case you still want to access API in java, you need to add interface in SWIG. You can read documentation on SWIG for details.

       
  • Nikita Furin

    Nikita Furin - 2015-05-05

    Hello.
    I began to test the library. It don't work. Unity see the library (.so files), but it cannot find my function. So I have EntryPointNotFoundException.

    I have copied files from /pocketsphinx-android/lib/ into Unity.

     

    Last edit: Nikita Furin 2015-05-05
  • Nikita Furin

    Nikita Furin - 2015-05-05

    During compiling the library, mymodule.c is used. So I suppose that mymodule.h make trouble.

     
    • Nickolay V. Shmyrev

      You can check so public symbols with arm-linux-androideabi-nm (part of ndk). It is hard to say why symbols is not exported, maybe you define it as static.
      You can share your files if you need help.

       
  • Nikita Furin

    Nikita Furin - 2015-05-05

    Yes, I need help.

     
    • Nickolay V. Shmyrev

      You need to share other files including so file and C# code.

       
  • Nikita Furin

    Nikita Furin - 2015-05-06

    I attach them.

     
    • Nickolay V. Shmyrev

      Binary does not export your functions which means that module is not included into so file. You need to check build process yourself or provide makefile you use.

       
  • Nikita Furin

    Nikita Furin - 2015-05-06

    I just added "mymodule.c \" into Android.mk, as I wrote above.
    Should I change files named "Makefile.am"?

     
  • Nikita Furin

    Nikita Furin - 2015-05-06

    I changed /pocketsphinx/include/Makefile.am and /pocketsphinx/src/libpocketsphinx and rebuild(ant clean, ant jar) the library.
    This did not help, because

    cat libpocketsphinx_jni.so | grep mm_f

    return nothing, i.e. binary does not export my functions.

     

    Last edit: Nikita Furin 2015-05-06
    • Nickolay V. Shmyrev

      Makefile.am is irrelevant here, it is only used on Linux automake-based build. You are building with ndk-build and only Android.mk is relevant.

      Your issue is that you included your wrapper into pocketsphinx static library. During build all unused symbols from static libraries are excluded, only functions referenced from "whole" static libraries are kept. You need to add module as LOCAL_WHOLE_STATIC_LIBRARY, not inside pocketsphinx. An example Android.mk is attached.

      However, there is another issue. You used ad.h for audio recording which is not supported in android. You need to record audio with Android native framework like openal instead of sphinxad.

      http://mobilepearls.com/labs/native-android-api/ndk/docs/opensles/

      Alternatively you can record with C# and then pass audio buffers into pocketsphinx through interop. In that case you will have to modify the wrapper you wrote.

       
  • Nikita Furin

    Nikita Furin - 2015-05-07

    Thank you, Nickolay!

     

Log in to post a comment.

Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.