Menu

Pocketsphinx + Qt + Android

Help
2015-06-16
2015-06-21
  • Maxim Sosnov

    Maxim Sosnov - 2015-06-16

    Hello, I've developed simple speech recognition app with Qt and it works nice on Windows and Ubuntu. But I can't build this app for android.

    When I'm trying to build my project I get these errors:

    warning: skipping incompatible /usr/local/lib/libpocketsphinx.so while searching for pocketsphinx
    error: cannot find -lpocketsphinx
    warning: skipping incompatible /usr/local/lib/libsphinxbase.so while searching for sphinxbase
    error: cannot find -lsphinxbase
    warning: skipping incompatible /usr/local/lib/libsphinxad.so while searching for sphinxad
    error: cannot find -lsphinxad

    I tried to build .so files as it showed at https://github.com/cmusphinx/pocketsphinx-android , but I get only one .so file (libpocketsphinx_jni.so).
    What should I do to get libs which I need?

    My environment:

    Ubuntu 14.04
    Qt 5.4.1
    compiler: Android GCC (arm 4.9)
    kit in Qt: Android for armeabi-v7a

    My pro file:

    TEMPLATE = app
    QT += qml quick widgets multimedia core
    CONFIG += c++11
    VPATH += ./source
    SOURCES += main.cpp \
    qcore.cpp \
    audioinfo.cpp \
    pocketsphinxprocessor.cpp \
    microphoneinterface.cpp \
    logger.cpp
    RESOURCES += qml.qrc
    LIBS += -I/usr/local/include -I/usr/local/include/sphinxbase -I/usr/local/include/pocketsphinx
    LIBS += -L/usr/local/lib -lpocketsphinx -lsphinxbase -lsphinxad
    LIBS += -L/home/crazymax/svn/pocketsphinx-android/libs/armeabi-v7a -lpocketsphinx -lsphinxbase -lsphinxad
    ANDROID_EXTRA_LIBS += $$PWD/pocketsphinx.so $$PWD/sphinxbase.so
    QML_IMPORT_PATH =
    include(deployment.pri)
    HEADERS += \
    qcore.h \
    audioinfo.h \
    pocketsphinxprocessor.h \
    microphoneinterface.h \
    logger.h \
    logLevel.h
    INCLUDEPATH += $$PWD/include/sphinxbase/
    INCLUDEPATH += $$PWD/include/
    INCLUDEPATH += $$PWD/include/pocketsphinx/
    DEPENDPATH += $$PWD/include/sphinxbase/
    DEPENDPATH += $$PWD/include/pocketsphinx/
    OTHER_FILES +=

     
    • bic-user

      bic-user - 2015-06-16

      Hello, Maxim

      -lsphinxad is an audio interface lib, it doesn't support android. If you want to grab audio on native side you'll need to implement recorder with openSLES. Pocketsphinx android demo passes audio from java side.

      What should I do to get libs which I need?

      if you want to keep that separation for -lpocketsphinx and -lsphinxbase, you'll need to build those with ndk yourself. But libpocketsphinx_jni.so contains all the pocketsphinx interfaces and I believe you'll be able to link you sources with this lib.

       
      • Maxim Sosnov

        Maxim Sosnov - 2015-06-17

        Thanks for reply.
        Can you help me a little to understand what I must to do?
        So, I built libpocketsphinx_jni.so and it contains all pocketsphinx interfaces. I need just these functions:

        cmd_ln_init
        ps_init
        ps_process_raw
        ps_get_in_speech
        ps_end_utt
        ps_get_hyp
        ps_start_utt
        ps_free

        How should I link lib to project? Now it's

        LIBS += -L"/home/crazymax/svn/pocketsphinx-android/libs/armeabi-v7a" -lpocketsphinx_jni

        Is somethere example of usage this lib in C++?

         
        • Nickolay V. Shmyrev

          LIBS += -L"/home/crazymax/svn/pocketsphinx-android/libs/armeabi-v7a" -lpocketsphinx_jni

          This is correct

          Is somethere example of usage this lib in C++?

          The example is provided in our tutorial

          http://cmusphinx.sourceforge.net/wiki/tutorialpocketsphinx

          You can find more advanced examples in pocketsphinx sources.

           
  • bic-user

    bic-user - 2015-06-17

    How should I link lib to project?

    I'm not quite sure how it's done in qt android. If you're working directly with Android.mk you link with shared library like this: http://stackoverflow.com/questions/9870435/how-can-i-link-prebuilt-shared-library-to-android-ndk-project

    Is somethere example of usage this lib in C++?

    if you create jni interfaces for your sources that works under linux and win it should be ok. Except sphinxad thing.

     
  • Maxim Sosnov

    Maxim Sosnov - 2015-06-18

    Thanks to bic-user and nsh for replys. I have a new problem, but it's related to topic name, so should I create new topic?

    Current situation:
    I've built application and it runs on my android device, but when I try to initialize pocketsphinx object, ps_init returns me NULL, so ps_decoder_t * points to NULL (causes SIGSEGV fatal error when I try to use it). But cmd_ln_init returns not NULL.
    This code still works fine at desktop.

    What can be wrong?

    Ubuntu 14.04
    Qt 5.4.1
    compiler: Android GCC (arm 4.9)
    kit in Qt: Android for armeabi-v7a
    Device OS: Android 4.0.4

     

    Last edit: Maxim Sosnov 2015-06-18
    • bic-user

      bic-user - 2015-06-18

      pocketsphinx log is redirected to android logcat. There should be a reason of initialization failure. Share that logcat with that info to get help

       
      • Maxim Sosnov

        Maxim Sosnov - 2015-06-18

        I get only this line and it is not on initialization:
        06-18 23:31:45.367: A/libc(3962): Fatal signal 11 (SIGSEGV) at 0x00000008 (code=1)

        I ran debug build without debugging

         
        • bic-user

          bic-user - 2015-06-18

          single line is not informative. Please provide whole log. Use pastebin.

           
          • Maxim Sosnov

            Maxim Sosnov - 2015-06-18

            it's the only line which appears while running.
            full logcat http://pastebin.com/6QZ2wdJK

             

            Last edit: Maxim Sosnov 2015-06-18
          • Maxim Sosnov

            Maxim Sosnov - 2015-06-19

            Here is logcat of android emu
            http://pastebin.com/bAdn83mt

             
            • bic-user

              bic-user - 2015-06-19

              emulator seems to work. It just fails to get access to external storage to load models and to record audio. As for device log, it looks suspicious. There is no info at least about starting activity. Are sure there are no filters selected and you provide relevant log?

               
              • Maxim Sosnov

                Maxim Sosnov - 2015-06-19

                The device is rooted. Maybe this is a reason of such behavior.
                Tried another device and it generates tons of log lines.
                Pastebin to log filtered by Qt
                http://pastebin.com/4C8fKPpc

                 
                • bic-user

                  bic-user - 2015-06-19

                  there are a lot of qt staff indeed.
                  Take a look on this line:

                  E/cmusphinx( 8157): ERROR: "fe_interface.c", line 279: Upper frequency 6855.5 is higher than samprate/2 (4000.0)

                  Do you grab 16 khz audio? how did you initialize ps decoder? Acoustic model you are using expects 16 khz audio, while decoder is configured for 8khz. Either use 8khz model or change config. Log is still incomplete. There should be bunch of ps initialization info like in windows or linux console.

                   
                  • Maxim Sosnov

                    Maxim Sosnov - 2015-06-19

                    this error appears at ps_init line. Is this error related to -upperf in feat.params of acoustic model? At the feat.params file defined -upperf 3700.
                    I use acoustic model which works with 8khz (zero_ru_v3 adapted to my needs).
                    in cmd_ln_init defined lm, hmm, dict and samprate (8000) as args

                     
  • bic-user

    bic-user - 2015-06-19

    Is this error related to -upperf in feat.params of acoustic model?

    yes

    At the feat.params file defined -upperf 3700

    Seems like acoustic model is not loaded. 6855.5 is default value. Make sure acoustic model is reachable, your app has permission for external storage io and correct path is specified on initialization. try to filter full log with "sphinx".

     
    • Maxim Sosnov

      Maxim Sosnov - 2015-06-19

      I deploy acoustic model in assets and it seems that "assets:/hmm/" url is not reachable for cmd_ln_init.

       
      • bic-user

        bic-user - 2015-06-19

        why it should be accessible? is it qt thing? by default you need to unpack assets to external storage by yourself. Try to deploy models on device with "adb push" in certain place and check if it runs

         
        • Maxim Sosnov

          Maxim Sosnov - 2015-06-21

          Great thanks, this problem has been solved.

           

Log in to post a comment.