I launched a training for a new acoustic model for a subset of the arabic language.
The training failed and I tried to use forced alignment by setting the following in sphinx_train.cfg file:
$CFG_FORCEDALIGN = 'yes';
The problem is that the training fails again without doing forced alignment because the tool is telling me that "sphinx3_align(.exe)" is not installed.
My questions:
1. How can I set up forced alignment to work? Is there a detailed manual or a link to help?
2. Why is it asking for an .exe file where I am training in Linux?
3. Is forced alignment present in the last versions of pocketsphinx/Sphinxbase/Sphinxtrain? because I already installed these tools.
Thanks in advance for your help,
Jamal.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
How can I set up forced alignment to work? Is there a detailed manual or a link to help?
The alignment is described in acoustic model training tutorial
Why is it asking for an .exe file where I am training in Linux?
It asks for optional exe (in parenthesis). In Linux it needs sphinx3_align without exe
Is forced alignment present in the last versions of pocketsphinx/Sphinxbase/Sphinxtrain? because I already installed these tools.
No, alignment is not supported by default sphinxtrain. You have to install sphinx3 additionally and copy sphinx3_align binary to libexec/sphinxtrain folder so that it would be found by scripts.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Yesterday I installed Sphinx3 (in another Virtual machine where I have previous Sphinx versions: Sphinxbase 0.8, Pocketsphinx 0.8 and Sphinxtrain 1.0.8). I retrieved the sphinx3_align binary file and copied it in the current VM in /usr/local/libexec/sphinxtrain/ folder. I then tried force alignment in the current and it did not work as well.
The reason why I installed sphinx3 in another VM is because it did not compile with with last version of sphinxbase (5prealpha). Current VM has Sphinxtrain, pocketsphinx and Sphinxbase 5prealpha versions only.
Training in the current VM gives the following message in step 11:
Phase 1: Cleaning up directories:
logs... output... qmanager...
Phase 3: Creating dictionary for alignment...
Phase 4: Creating transcript for alignment...
Phase 5: Running force alignment in 1 parts
Force alignment starting: (1 of 1)
sphinx3_align Log File
completed
Failed in part 1
After step 11, step 30 starts and fails at first step (most likely because step 11 failed).
After having this error I went to the /usr/local/lib folder in the VM where I installed sphinx3 and copied as well all libraries starting with libs3decoder. I pasted them in the current VM /usr/local/lib folder. I repeated the test and same error.
What could be the problem?
Thanks in advance for your help,
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
You need to configure machine linker to load libraries from /usr/local/lib. You can do it by setting LD_LIBRARY_PATH environment variable or in /etc/ld.so.conf.
You can easily find more information about this problem in Google.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Yesterday I installed Sphinx3 (in another Virtual machine where I have previous Sphinx versions: Sphinxbase 0.8, Pocketsphinx 0.8 and Sphinxtrain 1.0.8). I retrieved the sphinx3_align binary file and copied it in the current VM in /usr/local/libexec/sphinxtrain/ folder. I then tried force alignment in the current and it did not work as well.
For the reference to compile sphinx3 with latest 5prealpha you need to checkout sphinx3 from subversion. Then it will compile.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I installed sphinx3 from the svn and installation was OK. Thanks.
But when I repeat the training I still have an error at the same step (step 11). Log file gives the following:
mkdir: cannot create directory ‘/usr/local/libexec/sphinxtrain/.libs’: Permission denied
/usr/bin/ld: cannot open output file /usr/local/libexec/sphinxtrain/.libs/24907-lt-sphinx3_align: No such file or directory
collect2: error: ld returned 1 exit status
Tue Jun 23 00:51:37 2015
Shall I run "sphinxtrain run" command as root or is there another way of solving this permission problem?
Thanks for your help.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I then ran sphinxtrain run again and still the same problem:
/usr/local/libexec/sphinxtrain/sphinx3_align: error while loading shared libraries: libs3decoder.so.0: cannot open shared object file: No such file or directory
Then I did a test: I checked LD_LIBRARY_PATH before running sphinxtrain run command and it was OK (/usr/local/lib). I then started the training and checked again (during training in another terminal window) and it was empty.
What could be overwriting it?
I think that is the problem, when the training reaches step 11 the LD_LIBRARY_PATH has no value and then it cannot find the libraries.
Thanks in advance for your help,
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
You need to export in the same terminal window to export environment variables. They are not distributed across terminals, just inherited by child processes.
Alternatively you can configure linker in /etc/ld.so.conf
I did not know about the environment variables & terminal windows.
Before reading your previous post, I restarted the Virtual machine, set up the environment variables (with values described in previous post) and restarted a training and it finally passed.
I will check the link you sent me and configure the linker to have a cleaner environment for the next experiments.
The results are 11.5% sentence error rate & 5% word error rate (which I think is quite good for 1800 word in the vocabulary).
I have a few more questions before leaving this post:
This acoustic model should be used in a mobile phone so the ptm acoustic model type should be chosen next. I started with continuous model because in a previous thread you told me that it was better to start with continuous model as it was giving a better baseline.
My questions are the following for the next steps:
What parameters other than senones & densities can I play with to have the best results?
Shall I find the best values (senones & densities & other parameters) in continuous and then switch to ptm with the found values or shall I directly go to the ptm model and find the best parameters in ptm model?
Again I would like to thank you Nickolay for your precious help.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
What parameters other than senones & densities can I play with to have the best results?
Those things are covered in tutorial, you can use 256 densities in ptm and about 2k-3k senones
Shall I find the best values (senones & densities & other parameters) in continuous and then switch to ptm with the found values or shall I directly go to the ptm model and find the best parameters in ptm model?
You can play with PTM directly.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi,
I launched a training for a new acoustic model for a subset of the arabic language.
The training failed and I tried to use forced alignment by setting the following in sphinx_train.cfg file:
$CFG_FORCEDALIGN = 'yes';
The problem is that the training fails again without doing forced alignment because the tool is telling me that "sphinx3_align(.exe)" is not installed.
My questions:
1. How can I set up forced alignment to work? Is there a detailed manual or a link to help?
2. Why is it asking for an .exe file where I am training in Linux?
3. Is forced alignment present in the last versions of pocketsphinx/Sphinxbase/Sphinxtrain? because I already installed these tools.
Thanks in advance for your help,
Jamal.
The alignment is described in acoustic model training tutorial
It asks for optional exe (in parenthesis). In Linux it needs sphinx3_align without exe
No, alignment is not supported by default sphinxtrain. You have to install sphinx3 additionally and copy sphinx3_align binary to libexec/sphinxtrain folder so that it would be found by scripts.
Ok Thanks a lot Nicolay.
I will try that today and inform you about the result.
Hi,
Yesterday I installed Sphinx3 (in another Virtual machine where I have previous Sphinx versions: Sphinxbase 0.8, Pocketsphinx 0.8 and Sphinxtrain 1.0.8). I retrieved the sphinx3_align binary file and copied it in the current VM in /usr/local/libexec/sphinxtrain/ folder. I then tried force alignment in the current and it did not work as well.
The reason why I installed sphinx3 in another VM is because it did not compile with with last version of sphinxbase (5prealpha). Current VM has Sphinxtrain, pocketsphinx and Sphinxbase 5prealpha versions only.
Training in the current VM gives the following message in step 11:
When I check the log file it gives this:
After step 11, step 30 starts and fails at first step (most likely because step 11 failed).
After having this error I went to the /usr/local/lib folder in the VM where I installed sphinx3 and copied as well all libraries starting with libs3decoder. I pasted them in the current VM /usr/local/lib folder. I repeated the test and same error.
What could be the problem?
Thanks in advance for your help,
You need to configure machine linker to load libraries from /usr/local/lib. You can do it by setting LD_LIBRARY_PATH environment variable or in /etc/ld.so.conf.
You can easily find more information about this problem in Google.
For the reference to compile sphinx3 with latest 5prealpha you need to checkout sphinx3 from subversion. Then it will compile.
Thanks a lot Nickolay,
But LD_LIBRARY_PATH, PATH and PKG_CONFIG_PATH are all set to the right values.
I will investigate some more and try to redo the copy/paste of the librairies and make sure it is well done.
If you have other ideas for the source of the problem, please do not hesitate.
Thanks,
Hi,
I installed sphinx3 from the svn and installation was OK. Thanks.
But when I repeat the training I still have an error at the same step (step 11). Log file gives the following:
Shall I run "sphinxtrain run" command as root or is there another way of solving this permission problem?
Thanks for your help.
You need to install sphinx3 first and then copy the binary from /usr/local/bin to /usr/local/libexec/sphinxtrain
You used a helper libtool script instead of a binary.
Hi,
I double checked and made sure that LD_LIBRARY_PATH is set to /usr/local/lib/.
I uninstalled sphinx3, removed libtool (you were right I used libtool). I then made sure /usr/local/lib and /usr/local/bin had no trace of sphinx3.
I reinstalled sphinx3 and copied the binary sphinx3_align to /usr/local/libexec/sphinxtrain
I set the following environment variables:
I then ran sphinxtrain run again and still the same problem:
Then I did a test: I checked LD_LIBRARY_PATH before running sphinxtrain run command and it was OK (/usr/local/lib). I then started the training and checked again (during training in another terminal window) and it was empty.
What could be overwriting it?
I think that is the problem, when the training reaches step 11 the LD_LIBRARY_PATH has no value and then it cannot find the libraries.
Thanks in advance for your help,
You need to export in the same terminal window to export environment variables. They are not distributed across terminals, just inherited by child processes.
Alternatively you can configure linker in /etc/ld.so.conf
This might be helpful for you: https://help.ubuntu.com/community/EnvironmentVariables
Thanks Nickolay,
I did not know about the environment variables & terminal windows.
Before reading your previous post, I restarted the Virtual machine, set up the environment variables (with values described in previous post) and restarted a training and it finally passed.
I will check the link you sent me and configure the linker to have a cleaner environment for the next experiments.
The results are 11.5% sentence error rate & 5% word error rate (which I think is quite good for 1800 word in the vocabulary).
I have a few more questions before leaving this post:
This acoustic model should be used in a mobile phone so the ptm acoustic model type should be chosen next. I started with continuous model because in a previous thread you told me that it was better to start with continuous model as it was giving a better baseline.
My questions are the following for the next steps:
Again I would like to thank you Nickolay for your precious help.
Those things are covered in tutorial, you can use 256 densities in ptm and about 2k-3k senones
You can play with PTM directly.