Since I was having bad recgnition accuracy,I am trying to do the adaption of the acoustic model for the French language such that the pocketsphinx should be able to recognize phonemes such as: syllables( like "de", "re", "se", etc), consonants (like "m", "f", "g", etc), double-consonants(like "kl", "ks", "gr",etc) and vowels(like "a", "o","e",etc). But, while doing the adaption, I get some warnings when I try to run the bw.exe. The warning I get are the following:
Thank you very much for your answer.
It worked for the most of the letters I want to recognize. But, for some of them I still get the same warnings, such as: "pl", "ye", etc. Should I just add them somewhere?
I also tested the adapted model with the letters that I did not have the warnings mentioned above, and the accuracy is improved. But, when I test them, I still get some letters that are not pronounced at all. Could be the reason that the person in the audio file starts to speak after 0.2 second (as I read on the website of the CMUsphinx, the amount of silence in the beginning of the utterance should not exceed 0.2 second, and in my case it starts after the 1st second)? Or, how can I avoid the appearnace of these letters that are not pronounced at all?
Thank you very much in advance!
Best regards,
Leutrim
Last edit: Leo 2016-04-27
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
But, for some of them I still get the same warnings, such as: "pl", "ye", etc. Should I just add them somewhere?
You need to add them to the dictionary file
Could be the reason that the person in the audio file starts to speak after 0.2 second (as I read on the website of the CMUsphinx, the amount of silence in the beginning of the utterance should not exceed 0.2 second, and in my case it starts after the 1st second)?
It is better to follow recommendation of tutorial precisely, however, it is not the reason of extra insertions.
Or, how can I avoid the appearnace of these letters that are not pronounced at all?
You can play with langauge weight and phone insertion probability (-pip) to avoid extra insertions. However, it is not possible to eliminate them altogether. Some insertions are ok.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thank you very much for your answer. It worked, now I do not have missing letters.
Regarding the extra insertions, since my application is supposed to be used by people who have problems with speaking (suffering from aphasia, a brain disorder), they have to learn to pronounce correctly the phonemes(the letters I have mentioned above) I give to them, and I should give a feedback to them whether the pronounciation is correct or not.
Since I have as output even letters that are not pronounced at all, this might be a problem to give the right feedback to the users. I have an idea for solving this, but this is a simple one, and might not be a good one: * each phoneme that is supposed to recognize, would have its representative (according to the tests performed, since we know what should be recognized), and if the representative matches with the recognized result, the pronounciation could be considered as a correct one*.
Since this is a scientific research as well, is that possible to state that phoneme recognition needs still improvements, and argumenting why these extra letters are appearing? Or, could be there other solution for this problem?
Thank you very much in advance!
Leutrim
Last edit: Leo 2016-04-28
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
For scientific research you started too early with experiments without prior art search and theoretical research. I would first build a theory on how are you going to do the scoring and only then start with experiments.
There are many ways to verify correctness of pronunciation, but in general you can't proceed with model only trained on the correct pronunciation. You need an alternative model of bad pronunciation in order to make an optimal decision about pronunciation quality.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
First, I would like to thank you very much for your reply and for the support.
Before choosing Pocketsphinx, I did some research on what speech recognition software (that are applicable to mobile devices) are available nowadays, and which of them best would fit to my problem. Thus, I figured out that Pocketpshinx would be the most appropriate one for solving my problem.
Regarding the alternative model for bad pronunciation, I would like just to make sure myself. So, I should desing an alternative model by adapting the default acoustic model with bad pronunciation of the audio samples, and thus we would have two models, the one trained with the correct pronunciations and the other one trained with bad pronunciations. Is that right? Then, how could we use these models? Couldn't I just put a threshold between what is good and bad, and thus using just one model? I am not sure if I am right, I am a bit confused with this alternative model.
Thank you very much!
Leutrim
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
HI Leutrim
I´m french too and a friend and i , both work on qbo robot on speech reco.
I made a small tkinter py. program to automate recording texts, speak...
both textfiles are automatically created
adaptation done,
missing word on dic appear and program wait for changes before looping.
you can find it on my github elpimous ( tools section : PAMA)
si on peut s´entraider pour obtenir un wer < 15%...
vincent
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hello Elpimous,
.
I solve the problem with the missing letters by just adding them to the dictionary file.
Anyway, thank you very much for writing and trying to help me.
Best,
Leutrm
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hello everyone,
Since I was having bad recgnition accuracy,I am trying to do the adaption of the acoustic model for the French language such that the pocketsphinx should be able to recognize phonemes such as: syllables( like "de", "re", "se", etc), consonants (like "m", "f", "g", etc), double-consonants(like "kl", "ks", "gr",etc) and vowels(like "a", "o","e",etc). But, while doing the adaption, I get some warnings when I try to run the bw.exe. The warning I get are the following:
Also, you can find attached the transcription reference file and the wav files that I want to do the adaption on.
Could any one of you please let me know what how to deal with these issues?
Thank you very much in advance!
Leutrim
Last edit: Nickolay V. Shmyrev 2016-04-26
You need to have spaces after
<s>and before</s>.Hello Nickolay,
Thank you very much for your answer.
It worked for the most of the letters I want to recognize. But, for some of them I still get the same warnings, such as: "pl", "ye", etc. Should I just add them somewhere?
I also tested the adapted model with the letters that I did not have the warnings mentioned above, and the accuracy is improved. But, when I test them, I still get some letters that are not pronounced at all. Could be the reason that the person in the audio file starts to speak after 0.2 second (as I read on the website of the CMUsphinx, the amount of silence in the beginning of the utterance should not exceed 0.2 second, and in my case it starts after the 1st second)? Or, how can I avoid the appearnace of these letters that are not pronounced at all?
Thank you very much in advance!
Best regards,
Leutrim
Last edit: Leo 2016-04-27
You need to add them to the dictionary file
It is better to follow recommendation of tutorial precisely, however, it is not the reason of extra insertions.
You can play with langauge weight and phone insertion probability (-pip) to avoid extra insertions. However, it is not possible to eliminate them altogether. Some insertions are ok.
Hello Nickolay,
Thank you very much for your answer. It worked, now I do not have missing letters.
Regarding the extra insertions, since my application is supposed to be used by people who have problems with speaking (suffering from aphasia, a brain disorder), they have to learn to pronounce correctly the phonemes(the letters I have mentioned above) I give to them, and I should give a feedback to them whether the pronounciation is correct or not.
Since I have as output even letters that are not pronounced at all, this might be a problem to give the right feedback to the users. I have an idea for solving this, but this is a simple one, and might not be a good one: * each phoneme that is supposed to recognize, would have its representative (according to the tests performed, since we know what should be recognized), and if the representative matches with the recognized result, the pronounciation could be considered as a correct one*.
Since this is a scientific research as well, is that possible to state that phoneme recognition needs still improvements, and argumenting why these extra letters are appearing? Or, could be there other solution for this problem?
Thank you very much in advance!
Leutrim
Last edit: Leo 2016-04-28
For scientific research you started too early with experiments without prior art search and theoretical research. I would first build a theory on how are you going to do the scoring and only then start with experiments.
There are many ways to verify correctness of pronunciation, but in general you can't proceed with model only trained on the correct pronunciation. You need an alternative model of bad pronunciation in order to make an optimal decision about pronunciation quality.
Hello Nickolay,
First, I would like to thank you very much for your reply and for the support.
Before choosing Pocketsphinx, I did some research on what speech recognition software (that are applicable to mobile devices) are available nowadays, and which of them best would fit to my problem. Thus, I figured out that Pocketpshinx would be the most appropriate one for solving my problem.
Regarding the alternative model for bad pronunciation, I would like just to make sure myself. So, I should desing an alternative model by adapting the default acoustic model with bad pronunciation of the audio samples, and thus we would have two models, the one trained with the correct pronunciations and the other one trained with bad pronunciations. Is that right? Then, how could we use these models? Couldn't I just put a threshold between what is good and bad, and thus using just one model? I am not sure if I am right, I am a bit confused with this alternative model.
Thank you very much!
Leutrim
HI Leutrim
I´m french too and a friend and i , both work on qbo robot on speech reco.
I made a small tkinter py. program to automate recording texts, speak...
both textfiles are automatically created
adaptation done,
missing word on dic appear and program wait for changes before looping.
you can find it on my github elpimous ( tools section : PAMA)
si on peut s´entraider pour obtenir un wer < 15%...
vincent
Hello Elpimous,
.
I solve the problem with the missing letters by just adding them to the dictionary file.
Anyway, thank you very much for writing and trying to help me.
Best,
Leutrm