How to add new acoustic model to sphinx 4

[Mohammad M.]

I finished training my acoustic model. but how i can use it in sphinx 4?
the files that resulted from the Sphinxtrain are:
- mdef
- feat.params
- mixture_weights
- means
- noisedict
- transition_matrices
- variances

I tried to follow this tutorial, but the link is broken.
http://cmusphinx.sourceforge.net/wiki/tutorialsphinx4/doc/UsingSphinxTrainModels.html

[manu khandelwal]

Hi Mohammad,
I was interested in creating acoustic model for the indian accent english.
If you also created for the same, can you share me the acoustic, dictionary and language model for the indian accent.
If you did not do for indian accent english, do you know from where i can get one.

Thanks,
Manu

[Nickolay V. Shmyrev]

The tutorial is here:

http://cmusphinx.sourceforge.net/wiki/tutorialsphinx4

You need to set the path to the acoustic model in Configuration:

 configuration.setAcousticModelPath("file:acoustic_model_folder");

[Godfrey]

@Nickolay I have the question.I added the absolutely path,but eclipse can't recognize the Configuration and "The import edu.cmu.sphinx.api cannot be resolved".Here is my code.
package demo.sphinx.myowntest;

import java.net.URL;
import edu.cmu.sphinx.api.Configuration;
import edu.cmu.sphinx.api.SpeechResult;
import edu.cmu.sphinx.result.WordResult;
import edu.cmu.sphinx.api.LiveSpeechRecognizer;

public class TranscriberLive {

public static void main(String[] args) throws Exception {

    System.out.println("Loading models...");

    Configuration configuration = new Configuration();

    // Load model from the jar

    configuration.setAcousticModelPath("resource:/D:\\sphinxtrain\bergtrain\\model_parameters\\bergtrain.cd_cont_200");

    // You can also load model from folder
    // configuration.setAcousticModelPath("file:en-us");

    configuration.setDictionaryPath("resource:/WSJ_8gau_13dCep_16k_40mel_130Hz_6800Hz/dict/cmudict.0.6d");

    configuration.setLanguageModelPath("models/language/en-us.lm.dmp");

    SpeechResult result;

    recognizer.startRecognition(true);

    while ((result = recognizer.getResult()) != null ) {

        if ( recognizer.getResult().length() == 0) {
            System.out.format("Say Again!";
         else
            System.out.format("You Said: %s\n",
            recognizer.getResult()); }

        System.out.println("List of recognized words and their times:");
        for (WordResult r : result.getWords()) { System.out.println(r);

}

    recognizer.stopRecognition();
}

}

[Nickolay V. Shmyrev]

Make sure you downloaded the latest release (5prealpha).

[Godfrey]

Thank you!

[manu khandelwal]

Hello Nickolay,

Do you know if acoustic model is available for the indian accent english.
Thanks for the help in advance.

Thanks,
Manu

[Mohammad M.]

wow, that's fantastic. now, the system is working.
Thank you Nickolay

[Amar]

I have this same problem.I was also following the same link and now that page no longer exists. Can you be more specific on how to use the sphinx 4 in a IDE as netbeans or eclipse. Which all jar file i need to add and do i need to create a xml file?

[Nickolay V. Shmyrev]

I was also following the same link and now that page no longer exists.

Please check

http://cmusphinx.sourceforge.net/wiki/tutorialsphinx4

Can you be more specific on how to use the sphinx 4 in a IDE as netbeans or eclipse.

As any other library you a jar to the project classpath and write the code

Which all jar file i need to add

sphinx4.jar and WSJ_8gau_13dCep_16k_40mel_130Hz_6800Hz.jar

do i need to create a xml file?

No

[Amar]

Thanks for your replay. I have created a small accoustic model, so i guess i dont need to use WSJ_8gau_13dCep_16k_40mel_130Hz_6800Hz.jar, right?

I tried to write some code for the recognition but getting some errors. Please help me.

/*
 * To change this license header, choose License Headers in Project Properties.
 * To change this template file, choose Tools | Templates
 * and open the template in the editor.
 */

package beta;

import edu.cmu.sphinx.api.Configuration;
import edu.cmu.sphinx.api.SpeechResult;
import edu.cmu.sphinx.api.StreamSpeechRecognizer;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;

/**
 *
 * @author amar
 */
public class Beta {

    /**
     * @param args the command line arguments
     */
    public static void main(String[] args) throws IOException {

    Configuration configuration = new Configuration();

    // Set path to acoustic model.
    configuration.setAcousticModelPath("file:new.cd_cont_100");
    // Set path to dictionary.
    configuration.setDictionaryPath("new.dic");
    // Set language model.
    configuration.setLanguageModelPath("new.lm.dmp");

    InputStream inputstream = new FileInputStream("/home/amar/tutorial/output.wav");

    StreamSpeechRecognizer recognizer = new StreamSpeechRecognizer(configuration);
    recognizer.startRecognition(inputstream);
    SpeechResult result = recognizer.getResult();
    recognizer.stopRecognition();

    // Print utterance string without filler words.
    System.out.println(result.getHypothesis());
    // Save lattice in a graphviz format.
   // result.getLattice().dumpDot("lattice.dot", "lattice");

    }

}

These are the errors i am getting.

Exception in thread "main" Property exception component:'null' property:'null' - java.io.FileNotFoundException: new.cd_cont_100/means (No such file or directory)
edu.cmu.sphinx.util.props.PropertyException: java.io.FileNotFoundException: new.cd_cont_100/means (No such file or directory)
at edu.cmu.sphinx.frontend.AutoCepstrum.newProperties(AutoCepstrum.java:122)
at edu.cmu.sphinx.util.props.PropertySheet.getOwner(PropertySheet.java:508)
at edu.cmu.sphinx.util.props.ConfigurationManager.lookup(ConfigurationManager.java:161)
at edu.cmu.sphinx.util.props.PropertySheet.getComponentList(PropertySheet.java:422)
at edu.cmu.sphinx.frontend.FrontEnd.newProperties(FrontEnd.java:160)
at edu.cmu.sphinx.util.props.PropertySheet.getOwner(PropertySheet.java:508)
at edu.cmu.sphinx.util.props.PropertySheet.getComponent(PropertySheet.java:290)
at edu.cmu.sphinx.decoder.scorer.SimpleAcousticScorer.newProperties(SimpleAcousticScorer.java:46)
at edu.cmu.sphinx.decoder.scorer.ThreadedAcousticScorer.newProperties(ThreadedAcousticScorer.java:130)
at edu.cmu.sphinx.util.props.PropertySheet.getOwner(PropertySheet.java:508)
at edu.cmu.sphinx.util.props.PropertySheet.getComponent(PropertySheet.java:290)
at edu.cmu.sphinx.decoder.search.WordPruningBreadthFirstSearchManager.newProperties(WordPruningBreadthFirstSearchManager.java:201)
at edu.cmu.sphinx.util.props.PropertySheet.getOwner(PropertySheet.java:508)
at edu.cmu.sphinx.util.props.PropertySheet.getComponent(PropertySheet.java:290)
at edu.cmu.sphinx.decoder.AbstractDecoder.newProperties(AbstractDecoder.java:71)
at edu.cmu.sphinx.decoder.Decoder.newProperties(Decoder.java:37)
at edu.cmu.sphinx.util.props.PropertySheet.getOwner(PropertySheet.java:508)
at edu.cmu.sphinx.util.props.PropertySheet.getComponent(PropertySheet.java:290)
at edu.cmu.sphinx.recognizer.Recognizer.newProperties(Recognizer.java:90)
at edu.cmu.sphinx.util.props.PropertySheet.getOwner(PropertySheet.java:508)
at edu.cmu.sphinx.util.props.ConfigurationManager.lookup(ConfigurationManager.java:161)
at edu.cmu.sphinx.api.Context.<init>(Context.java:77)
at edu.cmu.sphinx.api.Context.<init>(Context.java:49)
at edu.cmu.sphinx.api.AbstractSpeechRecognizer.<init>(AbstractSpeechRecognizer.java:37)
at edu.cmu.sphinx.api.StreamSpeechRecognizer.<init>(StreamSpeechRecognizer.java:33)
at beta.Beta.main(Beta.java:43)
Caused by: java.io.FileNotFoundException: new.cd_cont_100/means (No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at java.io.FileInputStream.<init>(FileInputStream.java:97)
at sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:90)
at sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:188)
at java.net.URL.openStream(URL.java:1037)
at edu.cmu.sphinx.linguist.acoustic.tiedstate.Sphinx3Loader.getDataStream(Sphinx3Loader.java:260)
at edu.cmu.sphinx.linguist.acoustic.tiedstate.Sphinx3Loader.readS3BinaryHeader(Sphinx3Loader.java:513)
at edu.cmu.sphinx.linguist.acoustic.tiedstate.Sphinx3Loader.loadDensityFile(Sphinx3Loader.java:441)
at edu.cmu.sphinx.linguist.acoustic.tiedstate.Sphinx3Loader.loadModelFiles(Sphinx3Loader.java:333)
at edu.cmu.sphinx.linguist.acoustic.tiedstate.Sphinx3Loader.load(Sphinx3Loader.java:280)
at edu.cmu.sphinx.frontend.AutoCepstrum.newProperties(AutoCepstrum.java:120)
... 25 more

[Nickolay V. Shmyrev]

I have created a small accoustic model, so i guess i dont need to use WSJ_8gau_13dCep_16k_40mel_130Hz_6800Hz.jar, right?

Correct

edu.cmu.sphinx.util.props.PropertyException: java.io.FileNotFoundException: new.cd_cont_100/means (No such file or directory)

It says that there is no such file or directory, you can read the exception before you post it. You need to specify a proper path to a model folder. If you are not sure what is the relative path of the model folder, try to use an absolute path.

[SriRam Govardhanam]

this is my code ,is not working with indian accoustic model
plz help

package com.example;

import java.io.*;
import java.io.File;
import java.io.FileInputStream;
import java.io.InputStream;
import java.util.logging.Logger;

import edu.cmu.sphinx.api.Configuration;
import edu.cmu.sphinx.api.LiveSpeechRecognizer;
import edu.cmu.sphinx.api.SpeechResult;
import edu.cmu.sphinx.api.StreamSpeechRecognizer;

public class TranscriberDemo {

public static void main(String[] args) throws Exception {

    Configuration configuration = new Configuration();

    configuration.setAcousticModelPath("resource:/edu/cmu/sphinx/models/en-us/en_in.cd_cont_5000");
    configuration.setDictionaryPath("resource:/edu/cmu/sphinx/models/en-us/cmudict-en-us.dict");
    configuration.setLanguageModelPath("resource:/edu/cmu/sphinx/models/en-us/en-us.lm.bin");

    LiveSpeechRecognizer recognizer = new LiveSpeechRecognizer(configuration);

    recognizer.startRecognition(true);

SpeechResult result;
    while ((result = recognizer.getResult()) != null) {
    System.out.format("Hypothesis: %s\n", result.getHypothesis());
}
recognizer.stopRecognition();
}

}
output like

SEVERE: Missing HMM for unit IY with lc=R rc=M
08:31:32.803 SEVERE lexTreeLinguist Bad HMM Unit: M
Feb 20, 2018 8:31:32 AM edu.cmu.sphinx.linguist.lextree.HMMTree addPronunciation
SEVERE: Missing HMM for unit M with lc=IY rc=AA
08:31:32.804 SEVERE lexTreeLinguist Bad HMM Unit: AA
Feb 20, 2018 8:31:32 AM edu.cmu.sphinx.linguist.lextree.HMMTree addPronunciation
SEVERE: Missing HMM for unit AA with lc=M rc=L
08:31:32.804 SEVERE lexTreeLinguist Bad HMM Unit: L
Feb 20, 2018 8:31:32 AM edu.cmu.sphinx.linguist.lextree.HMMTree addPronunciation
SEVERE: Missing HMM for unit L with lc=AA rc=D
08:31:32.804 SEVERE lexTreeLinguist Bad HMM Unit: D
Feb 20, 2018 8:31:32 AM edu.cmu.sphinx.linguist.lextree.HMMTree addPronunciation
SEVERE: Missing HMM for unit D with lc=L rc=IY
08:31:32.804 SEVERE lexTreeLinguist Bad HMM Unit: S
Feb 20, 2018 8:31:32 AM edu.cmu.sphinx.linguist.lextree.HMMTree addPronunciation
SEVERE: Missing HMM for unit S with lc=EH rc=T
08:31:32.804 SEVERE lexTreeLinguist Bad HMM Unit: T
Feb 20, 2018 8:31:32 AM edu.cmu.sphinx.linguist.lextree.HMMTree addPronunciation
SEVERE: Missing HMM for unit T with lc=S rc=R
08:31:32.805 SEVERE lexTreeLinguist Bad HMM Unit: R
Feb 20, 2018 8:31:32 AM edu.cmu.sphinx.linguist.lextree.HMMTree addPronunciation
SEVERE: Missing HMM for unit R with lc=T rc=IH
08:31:32.805 SEVERE lexTreeLinguist Bad HMM Unit: IH
Feb 20, 2018 8:31:32 AM edu.cmu.sphinx.linguist.lextree.HMMTree addPronunciation
SEVERE: Missing HMM for unit IH with lc=R rc=K
08:31:32.805 SEVERE lexTreeLinguist Bad HMM Unit: AA
Feb 20, 2018 8:31:32 AM edu.cmu.sphinx.linguist.lextree.HMMTree addPronunciation
SEVERE: Missing HMM for unit AA with lc=CH rc=R
08:31:32.805 SEVERE lexTreeLinguist Bad HMM Unit: R
Feb 20, 2018 8:31:32 AM edu.cmu.sphinx.linguist.lextree.HMMTree addPronunciation
SEVERE: Missing HMM for unit R with lc=AA rc=T
08:31:32.805 SEVERE lexTreeLinguist Bad HMM Unit: T
Feb 20, 2018 8:31:32 AM edu.cmu.sphinx.linguist.lextree.HMMTree addPronunciation
SEVERE: Missing HMM for unit T with lc=R rc=ER
08:31:32.805 SEVERE lexTreeLinguist Bad HMM Unit: ER
Feb 20, 2018 8:31:32 AM edu.cmu.sphinx.linguist.lextree.HMMTree addPronunciation
SEVERE: Missing HMM for unit ER with lc=T rc=HH
08:31:32.805 SEVERE lexTreeLinguist Bad HMM Unit: HH
Feb 20, 2018 8:31:32 AM edu.cmu.sphinx.linguist.lextree.HMMTree addPronunciation
SEVERE: Missing HMM for unit HH with lc=ER rc=AW
08:31:32.805 SEVERE lexTreeLinguist Bad HMM Unit: AW
Feb 20, 2018 8:31:32 AM edu.cmu.sphinx.linguist.lextree.HMMTree addPronunciation
SEVERE: Missing HMM for unit AW with lc=HH rc=S
08:31:32.805 SEVERE lexTreeLinguist Bad HMM Unit: EY
Feb 20, 2018 8:31:32 AM edu.cmu.sphinx.linguist.lextree.HMMTree addPronunciation
SEVERE: Missing HMM for unit EY with lc=S rc=N
08:31:32.805 SEVERE lexTreeLinguist Bad HMM Unit: OW
Feb 20, 2018 8:31:32 AM edu.cmu.sphinx.linguist.lextree.HMMTree addPronunciation

[Nickolay V. Shmyrev]

You didn't update the dictionary

[Amar]

Actually i saw that error saying no such file or directory. But i thought i had the file in the right place. Anyway i solved it by using absolute path as you suggested. I tested the system with an4 database and the system was able to recognise its test sounds.

However i had no luck with the accoustic model i created. I gave a the same file i used for training and the system was unable to recognise it. I am trying to develop a speech recognition for a Indian language malayalam. I just want it to recognise about 10 sentences. How many speech data will i be needing and how can i use malayalam with sphinx ??

Thanks in advance.

[Nickolay V. Shmyrev]

How many speech data will i be needing and how can i use malayalam with sphinx ??

This issue is covered in tutorial

http://cmusphinx.sourceforge.net/wiki/tutorialam

You need at least 1 hour of speech data

However i had no luck with the accoustic model i created.

To get more help on this issue you need to share your model training folder

[Amar]

1 hour of speech data for 10 sentences? Do i record the same sentences over and over?
Using other languages is not clearly explained there. Can i just use type malayalam in english ie, typing every malayalam word using english alphabets ?

[Nickolay V. Shmyrev]

1 hour of speech data for 10 sentences? Do i record the same sentences over and over?

Yes, it's statistical software. You need a lot of data to estimate parameters. You can read same sentences 50-100 times. You can combine chunks in different variants too if you are bored.

Can i just use type malayalam in english ie, typing every malayalam word using english alphabets ?

You can use english letter or your native alphabet. Text file must be in UTF-8 encoding.

[Amar]

Even if i am using my native alphabet i should still use english alphabets for phonemes right?
I was thinking of getting 30 of my friends and have each of them read each sentence 10 times. Will this helps in making the model speaker independent?

[Nickolay V. Shmyrev]

Even if i am using my native alphabet i should still use english alphabets for phonemes right?

Yes.

I was thinking of getting 30 of my friends and have each of them read each sentence 10 times. Will this helps in making the model speaker independent?

Yes, sure.

[Amar]

I have successfully developed my speech recognition , thanks to you. I collected about 1.5 hr of data from 15 different people. The recognition is okay. However i still have some problem with the live speech recognizer. My recognition code is(i followed the tutorial) -

/*
 * To change this license header, choose License Headers in Project Properties.
 * To change this template file, choose Tools | Templates
 * and open the template in the editor.
 */

package beta;

import edu.cmu.sphinx.api.Configuration;
import edu.cmu.sphinx.api.LiveSpeechRecognizer;
import edu.cmu.sphinx.api.SpeechResult;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;

/**
 *
 * @author amar
 */
public class betacall {

    public String call() throws IOException{

         Configuration configuration = new Configuration();

    // Set path to acoustic model.
    configuration.setAcousticModelPath("src\\alpha_mala.cd_cont_200");
    // Set path to dictionary.
    configuration.setDictionaryPath("src\\alpha_mala.dic");
    // Set language model.
    configuration.setLanguageModelPath("src\\alpha_mala.dmp");

 LiveSpeechRecognizer recognizer = new LiveSpeechRecognizer(configuration);
// Start recognition process pruning previously cached data.
recognizer.startRecognition(true);
SpeechResult result = recognizer.getResult();
// Pause recognition process. It can be resumed then with startRecognition(false).
recognizer.stopRecognition();

System.out.println(result.getHypothesis());   

return result.getHypothesis();

    }

}

I am calling "beta" using a button and displaying the result to a jTextbox. My problem is that the recognizer does not the recognize the first part of the speech correctly. That is, the result is improved if i press the button and wait some time before saying anything.

My second problem is that i cannot the press the button again to recognize another sentence, i have the run the program again. This problem exists only in windows. I am using linux for training and windows for recognition and recording of sound (because for some reason i cant record sound without noise in linux).

[Nickolay V. Shmyrev]

You should not initialize the recognizer in button callback. You need to create the recognizer when application starts. In button callback you just need to start the recognition process and print the hypothesis starting from this line:

 recognizer.startRecognition(true);

everything else must be done on application init. You can reuse recognizer object multiple time.

That will solve both your problems.

[Amar]

Thanks a lot. It worked. Both my problems are solved.

Cheers

[SriRam Govardhanam]

thanks a lot man!..u r the only answer for any sphinx related questions!
it solves my problem

[Amar]

I did this speech recognition as a part of my academic project. So i need to study the basics about both sphinx 4 and sphinxtrain. Can you point me to some useful pages.