I have a list of movie names and song names (millions). The list keeps getting updated with new names (songs and movie names) every week. It is required to recognize movie/song names from user utterances. For such a requirement
which is better model:
1)large grammar file being updated regularly
OR
2)Hybrid language model to detect out of vocabulary words .
Thanks
Srinidhi
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
For the requirement of recognizing against millions of names (being updated regularly) will recognition work against large grammar (being appended regularly) or an hybrid language model needs to be built to recognize OOV names (for new songname/moviename)
For user utterance the best possible match from the list of songnames/movienames
must be recognized.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
If a language model is used for my requirement then an open vocabulary language model would be required to recognize OOV words. Does sphinx support open vocabulary language models ? How can I use sphinx to recognize OOV words using language model?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
If a language model is used for my requirement then an open vocabulary language model would be required to recognize OOV words.
This statement is not quite correct. Speech recognizers can not work with open vocabulary language models. You have to modify speech recognizer itself to be able to recognize arbitrary words, not just replace the language model.
Does sphinx support open vocabulary language models ?
No
How can I use sphinx to recognize OOV words using language model?
You can not use it without modifications.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
As we get new data (song and movie names) regularly what are the options we can consider for using sphinx with language model:
1)With new data(song and movie names) is it required to retrain and build the new language model every time ?
2)Is there any option where we can add new data without rebuilding language model?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
1)With new data(song and movie names) is it required to retrain and build the new language model every time ?
Language model requires you to have proper name frequencies, not just the names themselves. If you don't have frequencies you should use grammars, not language models.
2)Is there any option where we can add new data without rebuilding language model?
It is usually not a problem to rebuild the language model given you have source data for it.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi,
I have a list of movie names and song names (millions). The list keeps getting updated with new names (songs and movie names) every week. It is required to recognize movie/song names from user utterances. For such a requirement
which is better model:
1)large grammar file being updated regularly
OR
2)Hybrid language model to detect out of vocabulary words .
Thanks
Srinidhi
What do you mean by "better"?
For the requirement of recognizing against millions of names (being updated regularly) will recognition work against large grammar (being appended regularly) or an hybrid language model needs to be built to recognize OOV names (for new songname/moviename)
For user utterance the best possible match from the list of songnames/movienames
must be recognized.
Yes.
You can do that way too if you don't want to update the grammar.
If grammar file is used and updated regularly then is there any size limit on grammar file.
Limits are pretty high (MAX_INT32).
If a language model is used for my requirement then an open vocabulary language model would be required to recognize OOV words. Does sphinx support open vocabulary language models ? How can I use sphinx to recognize OOV words using language model?
This statement is not quite correct. Speech recognizers can not work with open vocabulary language models. You have to modify speech recognizer itself to be able to recognize arbitrary words, not just replace the language model.
No
You can not use it without modifications.
As we get new data (song and movie names) regularly what are the options we can consider for using sphinx with language model:
1)With new data(song and movie names) is it required to retrain and build the new language model every time ?
2)Is there any option where we can add new data without rebuilding language model?
Language model requires you to have proper name frequencies, not just the names themselves. If you don't have frequencies you should use grammars, not language models.
It is usually not a problem to rebuild the language model given you have source data for it.