Menu

using pocketsphinx in unity with csharp on windows

Help
2017-08-25
2019-07-06
  • Daniel Owers

    Daniel Owers - 2017-08-25

    Hi

    I'm trying to develop a unity application that uses pocketsphinx's phoneme detection for facial animation.
    We've have been able to get the unity demo work on a Macbook but not on Windows.

    The exception we are getting from unity is

    DllNotFoundException: libpocketsphinxwrap
    Pocketsphinx.pocketsphinxPINVOKE+SWIGExceptionHelper..cctor () (at Assets/Pocketsphinx/pocketsphinxPINVOKE.cs:105)
    

    I've then tried to libpocketsphinxwrap myself by first following the tutorial for windows on your site by opening the .sln file and batch building the entire library, first for sphinxbase then for pocketsphinx.
    After that I tried to run the makefile in pocketspinx/swig/csharp using cygwin with all the dependencies installed.
    That gives this output

    mkdir -p gen
    swig -I.. -I../../../sphinxbase/swig -I../include -I../../sphinxbase/include \
        -csharp \
        -dllimport "libpocketsphinxwrap.so" \
        -namespace "Pocketsphinx" -o sphinxbase.c \
        -outdir gen ../../../sphinxbase/swig/sphinxbase.i
    swig -I.. -I../../../sphinxbase/swig -I../include -I../../sphinxbase/include \
        -csharp \
        -dllimport "libpocketsphinxwrap.so" \
        -namespace "Pocketsphinx" -o pocketsphinx.c \
        -outdir gen ../pocketsphinx.i
    gcc -fPIC pocketsphinx.c sphinxbase.c `pkg-config --libs --cflags pocketsphinx` -shared -o libpocketsphinxwrap.so
    Package pocketsphinx was not found in the pkg-config search path.
    Perhaps you should add the directory containing `pocketsphinx.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'pocketsphinx' found
    pocketsphinx.c:1:0: warning: -fPIC ignored for target (all code is position independent)
     /* ----------------------------------------------------------------------------
     ^
    pocketsphinx.c:18:26: fatal error: pocketsphinx.h: No such file or directory
    compilation terminated.
    sphinxbase.c:1:0: warning: -fPIC ignored for target (all code is position independent)
     /* ----------------------------------------------------------------------------
     ^
    sphinxbase.c:18:31: fatal error: sphinxbase/cmd_ln.h: No such file or directory
    compilation terminated.
    make: *** [Makefile:19: libpocketsphinxwrap.so] Error 1
    

    The generated C# files are there and so are pocketsphinx.c and sphinxbase.c but there is no libpocketsphinxwrap.

    I'm not too sure what to change the PKG_CONFIG_PATH to. But if I change it to the sphinxbase and pocketsphinx folders and rename the file ending in .pc.in to end in .pc I get

    gcc: error: @LIBS@: No such file or directory
    

    I'd really appreciate the help. I've seen https://github.com/garuma/PiHueSphinx/tree/master/PiHueSphinx and https://github.com/cmuartfab/unitysphinx which both write their own wrapper for pocketsphinx but I don't know how to do that and I fear it will not be an easy concept to grasp.

     
  • Marcel Pfaffhauser

    Hi

    I am runing into the same problem, and I just wanted to ask if you found a solution.

    Here what I found until now:

    • the pc.in files are not the .pc files. You need to run "autogen.sh" in the folders in which the .pc.in files are in order to generate all the make files (including the .pc files from the .pc.in files).

    For this you need python (else there will be an error). (And if it is not the correct python version (in the correct place) you have to define the path to it...)

     

    Last edit: Marcel Pfaffhauser 2017-11-21
  • Prapas Pongdamrong

    I also tried to make libpocketsphinxwrap on windows but in the end I gave up (Solved one problem to find another problem.)

    I moved the process to ubuntu and everything smooth, no error, no problem.
    These was I have done.

    • install sphinxbase and pocketsphinx via
      sudo apt-get install git
      sudo apt-get install automake
      sudo apt-get install libtool
      sudo apt-get install bison
      sudo apt-get install python-dev
      sudo apt-get install swig
      sudo apt-get install make
      sudo apt-get install pkg-config
      git clone https://github.com/cmusphinx/sphinxbase.git
      cd sphinxbase
      ./autogen.sh
      make
      sudo make install
      cd ..
      git clone git://github.com/cmusphinx/pocketsphinx.git
      cd pocketsphinx
      ./autogen.sh
      make
      sudo make install

    • Then generate the wrapper by
      cd swig
      cd csharp
      mkdir -p gen
      make -f Makefile
      ..
      and done. I've got libpocketsphinxwrap.so and got the cs files inside gen folder.

    Still not testing it in unity. If everything work I will update.

    PS. If you want to use it on windows, you have to compile it by using visual studio to build the dll file.

    Update:
    Finally, I can make it work on windows by create libpocketsphinxwrap.dll.
    1. Follow method above on unbuntu. (To gen the cs files, pocketsphinx.c and sphinxbase.c, because there a lot of error while trying to do it in cygwin)
    2. Copy pocketsphinx and sphinxbase from ubuntu to window
    3. Open solution file (.sln) inside sphinxbase with visual studio. Then build it to generate. sphinxbase.dll (make it for both x86 and x64)
    4. Do the same on pocketsphinx.
    5. Then create the new visual studio project with "Visual c++ console application" name it "libpocketsphinxwrap". When the wizard popup select DLL (this project will generate dll wrapper libpocketsphinxwrap)
    6. add pocketsphinx.c and sphinxbase.c to Source Files
    7. Link the sphinxbase.lib and pocketsphinx.lib (getting from step 3 and 4) by adding folder path that contain header files and lib file into VC++ Directories > Include Directories & Library Directory
    8. Add sphinxbase.lib;pocketsphinx.lib to Additional Dependencies in project linker (project properties > Linker > Input > Additional Dependencies)
    9. Build libpocketsphinxwrap to get libpocketsphinxwrap.dll
    10. Copy libpocketsphinxwrap.dll, pocketsphinx.dll, sphinxbase.dll to unity project.
    11. Edit file PocketSphinxPINVOKE.cs by replace libpocketsphinxwrap.so with libpocketsphinxwrap (*without .dll or else unity will not find it)
    12. Test it. If you get Access Violation. It almost becase the decoder initialize fail, try to fix the code by point the dict, model, etc. to the right place.

     

    Last edit: Prapas Pongdamrong 2017-12-18
  • Sergey Panov

    Sergey Panov - 2018-07-11

    Hi, Prapas Pongdamrong!

    Your post is the only I found with instructions about making Pocketsphinx work with Unity3D on windows by create libpocketsphinxwrap.dll. However, I've got
    EntryPointNotFoundException: SWIGRegisterExceptionCallbacks_pocketsphinx
    Pocketsphinx.pocketsphinxPINVOKE+SWIGExceptionHelper..cctor () (at Assets/Pocketsphinx/pocketsphinxPINVOKE.cs:105)

    So could you help?! Here is what i did:
    1-4,5 pocketsphinx.dll and sphninxbase.dll successfully compiled for x64, project libpocketsphinxwrap created.
    5 Added pocketsphinx.cs and sphninxbase.cs into project. They are:
    pocketsphinx.cs:
    namespace Pocketsphinx {
    public class pocketsphinx {
    }
    }
    sphninxbase.cpp:
    namespace Pocketsphinx {
    public class SphinxBase {
    }
    }
    7-11 Done exactly!
    12 Build succeeded , but playing causes the EntryPointNotFoundException already mentioned.

    Do you have any ideas?

     
  • Giedrius Girgzdis

    Hello.
    We figured out what is happening here.
    Sphinx library names inside the built dll are all lowercase when Unity expects all words to start with uppercase letters. No just to find an easy way to fix all of that...

     
    • Nickolay V. Shmyrev

      when Unity expects all words to start with uppercase letters.

      There is definitely no such thing. You need to recheck yourself and provide more information in case you have issues.

       

Log in to post a comment.

Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.