using pocketsphinx in unity with csharp on windows

[Daniel Owers]

Hi

I'm trying to develop a unity application that uses pocketsphinx's phoneme detection for facial animation.
We've have been able to get the unity demo work on a Macbook but not on Windows.

The exception we are getting from unity is

DllNotFoundException: libpocketsphinxwrap
Pocketsphinx.pocketsphinxPINVOKE+SWIGExceptionHelper..cctor () (at Assets/Pocketsphinx/pocketsphinxPINVOKE.cs:105)

I've then tried to libpocketsphinxwrap myself by first following the tutorial for windows on your site by opening the .sln file and batch building the entire library, first for sphinxbase then for pocketsphinx.
After that I tried to run the makefile in pocketspinx/swig/csharp using cygwin with all the dependencies installed.
That gives this output

mkdir -p gen
swig -I.. -I../../../sphinxbase/swig -I../include -I../../sphinxbase/include \
    -csharp \
    -dllimport "libpocketsphinxwrap.so" \
    -namespace "Pocketsphinx" -o sphinxbase.c \
    -outdir gen ../../../sphinxbase/swig/sphinxbase.i
swig -I.. -I../../../sphinxbase/swig -I../include -I../../sphinxbase/include \
    -csharp \
    -dllimport "libpocketsphinxwrap.so" \
    -namespace "Pocketsphinx" -o pocketsphinx.c \
    -outdir gen ../pocketsphinx.i
gcc -fPIC pocketsphinx.c sphinxbase.c `pkg-config --libs --cflags pocketsphinx` -shared -o libpocketsphinxwrap.so
Package pocketsphinx was not found in the pkg-config search path.
Perhaps you should add the directory containing `pocketsphinx.pc'
to the PKG_CONFIG_PATH environment variable
No package 'pocketsphinx' found
pocketsphinx.c:1:0: warning: -fPIC ignored for target (all code is position independent)
 /* ----------------------------------------------------------------------------
 ^
pocketsphinx.c:18:26: fatal error: pocketsphinx.h: No such file or directory
compilation terminated.
sphinxbase.c:1:0: warning: -fPIC ignored for target (all code is position independent)
 /* ----------------------------------------------------------------------------
 ^
sphinxbase.c:18:31: fatal error: sphinxbase/cmd_ln.h: No such file or directory
compilation terminated.
make: *** [Makefile:19: libpocketsphinxwrap.so] Error 1

The generated C# files are there and so are pocketsphinx.c and sphinxbase.c but there is no libpocketsphinxwrap.

I'm not too sure what to change the PKG_CONFIG_PATH to. But if I change it to the sphinxbase and pocketsphinx folders and rename the file ending in .pc.in to end in .pc I get

gcc: error: @LIBS@: No such file or directory

I'd really appreciate the help. I've seen https://github.com/garuma/PiHueSphinx/tree/master/PiHueSphinx and https://github.com/cmuartfab/unitysphinx which both write their own wrapper for pocketsphinx but I don't know how to do that and I fear it will not be an easy concept to grasp.

[Marcel Pfaffhauser]

Hi

I am runing into the same problem, and I just wanted to ask if you found a solution.

Here what I found until now:

• the pc.in files are not the .pc files. You need to run "autogen.sh" in the folders in which the .pc.in files are in order to generate all the make files (including the .pc files from the .pc.in files).

For this you need python (else there will be an error). (And if it is not the correct python version (in the correct place) you have to define the path to it...)

[Prapas Pongdamrong]

I also tried to make libpocketsphinxwrap on windows but in the end I gave up (Solved one problem to find another problem.)

I moved the process to ubuntu and everything smooth, no error, no problem.
These was I have done.

• install sphinxbase and pocketsphinx via
  sudo apt-get install git
  sudo apt-get install automake
  sudo apt-get install libtool
  sudo apt-get install bison
  sudo apt-get install python-dev
  sudo apt-get install swig
  sudo apt-get install make
  sudo apt-get install pkg-config
  git clone https://github.com/cmusphinx/sphinxbase.git
  cd sphinxbase
  ./autogen.sh
  make
  sudo make install
  cd ..
  git clone git://github.com/cmusphinx/pocketsphinx.git
  cd pocketsphinx
  ./autogen.sh
  make
  sudo make install

• Then generate the wrapper by
  cd swig
  cd csharp
  mkdir -p gen
  make -f Makefile
  ..
  and done. I've got libpocketsphinxwrap.so and got the cs files inside gen folder.

Still not testing it in unity. If everything work I will update.

PS. If you want to use it on windows, you have to compile it by using visual studio to build the dll file.

Update:
Finally, I can make it work on windows by create libpocketsphinxwrap.dll.
1. Follow method above on unbuntu. (To gen the cs files, pocketsphinx.c and sphinxbase.c, because there a lot of error while trying to do it in cygwin)
2. Copy pocketsphinx and sphinxbase from ubuntu to window
3. Open solution file (.sln) inside sphinxbase with visual studio. Then build it to generate. sphinxbase.dll (make it for both x86 and x64)
4. Do the same on pocketsphinx.
5. Then create the new visual studio project with "Visual c++ console application" name it "libpocketsphinxwrap". When the wizard popup select DLL (this project will generate dll wrapper libpocketsphinxwrap)
6. add pocketsphinx.c and sphinxbase.c to Source Files
7. Link the sphinxbase.lib and pocketsphinx.lib (getting from step 3 and 4) by adding folder path that contain header files and lib file into VC++ Directories > Include Directories & Library Directory
8. Add sphinxbase.lib;pocketsphinx.lib to Additional Dependencies in project linker (project properties > Linker > Input > Additional Dependencies)
9. Build libpocketsphinxwrap to get libpocketsphinxwrap.dll
10. Copy libpocketsphinxwrap.dll, pocketsphinx.dll, sphinxbase.dll to unity project.
11. Edit file PocketSphinxPINVOKE.cs by replace libpocketsphinxwrap.so with libpocketsphinxwrap (*without .dll or else unity will not find it)
12. Test it. If you get Access Violation. It almost becase the decoder initialize fail, try to fix the code by point the dict, model, etc. to the right place.

[Sergey Panov]

Hi, Prapas Pongdamrong!

Your post is the only I found with instructions about making Pocketsphinx work with Unity3D on windows by create libpocketsphinxwrap.dll. However, I've got
EntryPointNotFoundException: SWIGRegisterExceptionCallbacks_pocketsphinx
Pocketsphinx.pocketsphinxPINVOKE+SWIGExceptionHelper..cctor () (at Assets/Pocketsphinx/pocketsphinxPINVOKE.cs:105)

So could you help?! Here is what i did:
1-4,5 pocketsphinx.dll and sphninxbase.dll successfully compiled for x64, project libpocketsphinxwrap created.
5 Added pocketsphinx.cs and sphninxbase.cs into project. They are:
pocketsphinx.cs:
namespace Pocketsphinx {
public class pocketsphinx {
}
}
sphninxbase.cpp:
namespace Pocketsphinx {
public class SphinxBase {
}
}
7-11 Done exactly!
12 Build succeeded , but playing causes the EntryPointNotFoundException already mentioned.

Do you have any ideas?

[Giedrius Girgzdis]

Hello.
We figured out what is happening here.
Sphinx library names inside the built dll are all lowercase when Unity expects all words to start with uppercase letters. No just to find an easy way to fix all of that...

[Nickolay V. Shmyrev]

when Unity expects all words to start with uppercase letters.

There is definitely no such thing. You need to recheck yourself and provide more information in case you have issues.