Trouble using SphinxTrain models in Spinx 4

[John Hall]

I recently trained a model called "commands" with Sphinx Train. I have followed the directions here:
http://cmusphinx.sourceforge.net/sphinx4/doc/UsingSphinxTrainModels.html
to use them with Sphinx 4. I have just edited the helloworld demo to do this. When I use build.xml to run, however, Netbeans responds with the following:
all:
Compiling 1 source file to C:\sphinx4\sphinx4-1.0beta\bld\classes
run:
Loading...
Exception in thread "main" java.lang.IndexOutOfBoundsException: Index: 123, Size: 123
at java.util.ArrayList.RangeCheck(ArrayList.java:547)
at java.util.ArrayList.get(ArrayList.java:322)
at edu.cmu.sphinx.linguist.acoustic.tiedstate.Pool.get(Pool.java:60)
at edu.cmu.sphinx.model.acoustic.commands.ModelLoader.getSenoneSequence(ModelLoader.java:1260)
at edu.cmu.sphinx.model.acoustic.commands.ModelLoader.loadHMMPool(ModelLoader.java:1216)
at edu.cmu.sphinx.model.acoustic.commands.ModelLoader.loadModelFiles(ModelLoader.java:564)
at edu.cmu.sphinx.model.acoustic.commands.ModelLoader.load(ModelLoader.java:476)
at edu.cmu.sphinx.model.acoustic.commands.Model.allocate(Model.java:177)
at edu.cmu.sphinx.linguist.flat.FlatLinguist.allocateAcousticModel(FlatLinguist.java:336)
at edu.cmu.sphinx.linguist.flat.FlatLinguist.allocate(FlatLinguist.java:318)
at edu.cmu.sphinx.decoder.search.SimpleBreadthFirstSearchManager.allocate(SimpleBreadthFirstSearchManager.java:602)
at edu.cmu.sphinx.decoder.Decoder.allocate(Decoder.java:109)
at edu.cmu.sphinx.recognizer.Recognizer.allocate(Recognizer.java:182)
at demo.sphinx.helloworld.HelloWorld.main(HelloWorld.java:54)
Java Result: 1
BUILD SUCCESSFUL (total time: 0 seconds)

I made a few extra changes that the website didn't mention just to get it this far. I can post the files that I changed if needed. Can anyone identify what is causing this error? Thanks.

[Nickolay V. Shmyrev]

Hm, looks rather strange. Can you share yor files or at least your configuration file.

[John Hall]

Here is my configuration file. The name of the acoustic model is "commands", so the associated .jar file is "commands.jar".

<?xml version="1.0" encoding="UTF-8"?>

<!--
Sphinx-4 Configuration file
-->

<!-- ******** -->
<!-- an4 configuration file -->
<!-- ******** -->

<config>

<!-- ******************************************************** -->
<!-- frequently tuned properties                              -->
<!-- ******************************************************** -->

<property name="logLevel" value="WARNING"/>

<property name="absoluteBeamWidth"  value="-1"/>
<property name="relativeBeamWidth"  value="1E-80"/>
<property name="wordInsertionProbability" value="1E-36"/>
<property name="languageWeight"     value="8"/>

<property name="frontend" value="epFrontEnd"/>
<property name="recognizer" value="recognizer"/>
<property name="showCreations" value="false"/>

<!-- ******************************************************** -->
<!-- word recognizer configuration                            -->
<!-- ******************************************************** -->

<component name="recognizer" type="edu.cmu.sphinx.recognizer.Recognizer">
    <property name="decoder" value="decoder"/>
    <propertylist name="monitors">
        <item>accuracyTracker </item>
        <item>speedTracker </item>
        <item>memoryTracker </item>
    </propertylist>
</component>

<!-- ******************************************************** -->
<!-- The Decoder   configuration                              -->
<!-- ******************************************************** -->

<component name="decoder" type="edu.cmu.sphinx.decoder.Decoder">
    <property name="searchManager" value="searchManager"/>
</component>

<component name="searchManager" 
    type="edu.cmu.sphinx.decoder.search.SimpleBreadthFirstSearchManager">
    <property name="logMath" value="logMath"/>
    <property name="linguist" value="flatLinguist"/>
    <property name="pruner" value="trivialPruner"/>
    <property name="scorer" value="threadedScorer"/>
    <property name="activeListFactory" value="activeList"/>
</component>

<component name="activeList" 
         type="edu.cmu.sphinx.decoder.search.PartitionActiveListFactory">
    <property name="logMath" value="logMath"/>
    <property name="absoluteBeamWidth" value="${absoluteBeamWidth}"/>
    <property name="relativeBeamWidth" value="${relativeBeamWidth}"/>
</component>

<component name="trivialPruner" 
            type="edu.cmu.sphinx.decoder.pruner.SimplePruner"/>

<component name="threadedScorer" 
            type="edu.cmu.sphinx.decoder.scorer.ThreadedAcousticScorer">
    <property name="frontend" value="${frontend}"/>
    <property name="isCpuRelative" value="true"/>
    <property name="numThreads" value="0"/>
    <property name="minScoreablesPerThread" value="10"/>
    <property name="scoreablesKeepFeature" value="true"/>
</component>

<!-- ******************************************************** -->
<!-- The linguist  configuration                              -->
<!-- ******************************************************** -->

<component name="flatLinguist" 
            type="edu.cmu.sphinx.linguist.flat.FlatLinguist">
    <property name="logMath" value="logMath"/>
    <property name="grammar" value="jsgfGrammar"/>
    <property name="acousticModel" value="commands"/>
    <property name="wordInsertionProbability" 
            value="${wordInsertionProbability}"/>
    <property name="languageWeight" value="${languageWeight}"/>
    <property name="unitManager" value="unitManager"/>
</component>

<!-- ******************************************************** -->
<!-- The Grammar  configuration                               -->
<!-- ******************************************************** -->

<component name="jsgfGrammar" type="edu.cmu.sphinx.jsapi.JSGFGrammar">
    <property name="dictionary" value="dictionary"/>
    <property name="grammarLocation" 
         value="resource:/demo.sphinx.helloworld.HelloWorld!/demo/sphinx/helloworld/"/>
    <property name="grammarName" value="hello"/>
<property name="logMath" value="logMath"/>
</component>

<!-- ******************************************************** -->
<!-- The Dictionary configuration                            -->
<!-- ******************************************************** -->
<component name="dictionary"
    type="edu.cmu.sphinx.linguist.dictionary.FullDictionary">
    <property name="dictionaryPath"
        value="resource:/edu.cmu.sphinx.model.acoustic.commands.Model!/edu/cmu/sphinx/model/acoustic/commands/dict/commands.dic"/>
    <property name="fillerPath" 
     value="resource:/edu.cmu.sphinx.model.acoustic.commands.Model!/edu/cmu/sphinx/model/acoustic/commands/dict/commands.filler"/>
    <property name="addSilEndingPronunciation" value="false"/>
    <property name="allowMissingWords" value="false"/>
    <property name="unitManager" value="unitManager"/>
</component>

<!-- ******************************************************** -->
<!-- The acoustic model configuration                         -->
<!-- ******************************************************** -->
<component name="commands" 
        type="edu.cmu.sphinx.model.acoustic.commands.Model">
    <property name="loader" value="sphinx3Loader"/>
    <property name="unitManager" value="unitManager"/>
</component>

<component name="sphinx3Loader"
        type="edu.cmu.sphinx.model.acoustic.commands.ModelLoader">
    <property name="logMath" value="logMath"/>
    <property name="unitManager" value="unitManager"/>
</component>

<!-- ******************************************************** -->
<!-- The unit manager configuration                           -->
<!-- ******************************************************** -->

<component name="unitManager" 
    type="edu.cmu.sphinx.linguist.acoustic.UnitManager"/>

<!-- ******************************************************** -->
<!-- The frontend configuration                               -->
<!-- ******************************************************** -->

<component name="frontEnd" type="edu.cmu.sphinx.frontend.FrontEnd">
    <propertylist name="pipeline">
        <item>microphone </item>
        <item>premphasizer </item>
        <item>windower </item>
        <item>fft </item>
        <item>melFilterBank </item>
        <item>dct </item>
        <item>liveCMN </item>
        <item>featureExtraction </item>
    </propertylist>
</component>

<!-- ******************************************************** -->
<!-- The live frontend configuration                          -->
<!-- ******************************************************** -->
<component name="epFrontEnd" type="edu.cmu.sphinx.frontend.FrontEnd">
    <propertylist name="pipeline">
        <item>microphone </item>
        <item>speechClassifier </item>
        <item>speechMarker </item>
        <item>nonSpeechDataFilter </item>
        <item>premphasizer </item>
        <item>windower </item>
        <item>fft </item>
        <item>melFilterBank </item>
        <item>dct </item>
        <item>liveCMN </item>
        <item>featureExtraction </item>
    </propertylist>
</component>

<!-- ******************************************************** -->
<!-- The frontend pipelines                                   -->
<!-- ******************************************************** -->

<component name="speechClassifier" 
           type="edu.cmu.sphinx.frontend.endpoint.SpeechClassifier">
    <property name="threshold" value="13"/>
</component>

<component name="nonSpeechDataFilter" 
           type="edu.cmu.sphinx.frontend.endpoint.NonSpeechDataFilter"/>

<component name="speechMarker" 
           type="edu.cmu.sphinx.frontend.endpoint.SpeechMarker" >
    <property name="speechTrailer" value="50"/>
</component>

<component name="premphasizer" 
           type="edu.cmu.sphinx.frontend.filter.Preemphasizer"/>

<component name="windower" 
           type="edu.cmu.sphinx.frontend.window.RaisedCosineWindower">
</component>

<component name="fft" 
        type="edu.cmu.sphinx.frontend.transform.DiscreteFourierTransform">
</component>

<component name="melFilterBank" 
    type="edu.cmu.sphinx.frontend.frequencywarp.MelFrequencyFilterBank">
</component>

<component name="dct" 
        type="edu.cmu.sphinx.frontend.transform.DiscreteCosineTransform"/>

<component name="liveCMN" 
           type="edu.cmu.sphinx.frontend.feature.LiveCMN"/>

<component name="featureExtraction" 
           type="edu.cmu.sphinx.frontend.feature.DeltasFeatureExtractor"/>

<component name="microphone" 
           type="edu.cmu.sphinx.frontend.util.Microphone">
    <property name="closeBetweenUtterances" value="false"/>
</component>

<!-- ******************************************************* -->
<!--  monitors                                               -->
<!-- ******************************************************* -->

<component name="accuracyTracker" 
            type="edu.cmu.sphinx.instrumentation.AccuracyTracker">
    <property name="recognizer" value="${recognizer}"/>
    <property name="showAlignedResults" value="false"/>
    <property name="showRawResults" value="false"/>
</component>

<component name="memoryTracker" 
            type="edu.cmu.sphinx.instrumentation.MemoryTracker">
    <property name="recognizer" value="${recognizer}"/>
<property name="showSummary" value="false"/>
<property name="showDetails" value="false"/>
</component>

<component name="speedTracker" 
            type="edu.cmu.sphinx.instrumentation.SpeedTracker">
    <property name="recognizer" value="${recognizer}"/>
    <property name="frontend" value="${frontend}"/>
<property name="showSummary" value="true"/>
<property name="showDetails" value="false"/>
</component>

<!-- ******************************************************* -->
<!--  Miscellaneous components                               -->
<!-- ******************************************************* -->

<component name="logMath" type="edu.cmu.sphinx.util.LogMath">
    <property name="logBase" value="1.0001"/>
    <property name="useAddTable" value="true"/>
</component>

</config>

Here is build.xml:

<?xml version="1.0" encoding="UTF-8"?>

<project basedir="." default="all" name="The hello world demo.">

<description>
This file is used to run the hello world demo.

NOTE: Before running the tests, you must have already built
the sphinx-4 sources by typing "ant" in the top level
sphinx4 directory.

</description>

<!-- ********************************************************** -->
<!-- *                                                        * -->
<!-- * Properties common to all tests                         * -->
<!-- *                                                        * -->
<!-- ********************************************************** -->
<property name="top_dir"       value="../../.."/>
<property name="lib_dir"            value="${top_dir}/lib"/>
<property name="build_dir"     value="${top_dir}/bld"/>
<property name="classes_dir"   value="${build_dir}/classes"/>

<path id="run.classpath">
    <pathelement path="${classes_dir}"/>
<pathelement location="${lib_dir}/jsapi.jar"/>
    <pathelement location="${lib_dir}/commands.jar"/>
</path>

<!-- ********************************************************** -->
<!-- *                                                        * -->
<!-- * The 'none' target.                                     * -->
<!-- *                                                        * -->
<!-- ********************************************************** -->
<target name="none">
        <echo>Type 'ant -projecthelp' for possible targets.</echo>
</target>

<!-- ********************************************************** -->
<!-- *                                                        * -->
<!-- * Compile all the test code.                             * -->
<!-- *                                                        * -->
<!-- ********************************************************** -->
<target name="all"
        description="Compiles all the tests.">
    <javac debug="true"
           source="1.4"
       deprecation="true"
       destdir="${classes_dir}"
       classpath="${classes_dir}:${lib_dir}/jsapi.jar"
       srcdir="."/>
</target>

<!-- ********************************************************** -->
<!-- *                                                        * -->
<!-- * Runs the hello world demo.                             * -->
<!-- *                                                        * -->
<!-- ********************************************************** -->
<target name="run"
    description="Runs the hello world demo."
    depends="all">
    <java classname="demo.sphinx.helloworld.HelloWorld"
          fork="true"
      maxmemory="128m">
              <sysproperty key="frontend" value="epFrontEnd"/>
      <classpath refid="run.classpath"/>
      <arg value="helloworld.config.xml"/>
    </java>
</target>

</project>

And finally the manifest:

Main-Class: demo.sphinx.helloworld.HelloWorld
Class-Path: ../lib/sphinx4.jar ../lib/jsapi.jar ../lib/commands.jar

HelloWorld.java is unchanged. Any ideas? Thanks.

[Nickolay V. Shmyrev]

Hm, microphone pipeline also include datablocker in original demo. Probably it's the case. Can you please follow the demo more precisely. Also please paste model.props.

[John Hall]

I'm sorry. I haven't been able to find anything about a microphone pipeline or datablocker anywhere in the demo file. s. What exactly are you referring to?

Here is my model.props file:

description = commands acoustic models
modelClass = edu.cmu.sphinx.model.acoustic.commands.Model
modelLoader = edu.cmu.sphinx.model.acoustic.commands.ModelLoader
dataLocation = commands.ci_cont
modelDefinition = etc/commands.1000.mdef

isBinary = true
featureType = ls_c_d_dd
vectorLength = 39
sparseForm = false

numberFftPoint = 512
numberFilters = 40
gaussians = 8
minimumFrequency = 133.33334
maximumFrequency = 6855.4976
sampleRate = 16000

I set these values as best I could determine from SphinxTrain's make_feats.pl file, which I ran before running the RunAll.pl script.

Also, I recently made some hardware changes, leading to my microphone no longer working (I'm working on a fix). Could this be the problem?

Thanks.

[Nickolay V. Shmyrev]

> I'm sorry. I haven't been able to find anything about a microphone pipeline or datablocker anywhere in the demo file. s. What exactly are you referring to?

http://cmusphinx.svn.sourceforge.net/viewvc/cmusphinx/trunk/sphinx4/src/demo/sphinx/helloworld/helloworld.config.xml?revision=6346&view=markup

line 169

[John Hall]

It must be different in the sphinx4-1.0beta-src.zip file on SourceForge. I got it via svn, but I'm having trouble setting it up in Netbeans now. There are no com, edu, and demo folders in $SPHINX_ROOT anymore. Where do I find these to add them to the source/test package folders?

[Holger Brandl]

Hi,

indeed, the directory structure has been changed since s4b1. You need to add all subfolders (excluding 'demo') of the src-folder and the src-folder itself as source-folders in netbeans.

@nshmyrev: What do you think: Should we move the demo folder to an own subfolder? The current state is not nice because the demo-folder is already part of the demo-class-package which leads to the strange procedure of adding children of 'src' and 'src' itself as source-folders.

[Nickolay V. Shmyrev]

I'm all for structure simplification. For example in eclipse current view is very confusing - it shows src folder, src/sphinx4 subfolder and organization of packages is not clear. At least we can remove empty extras folder (in svn you can do it :)) The reason of separation of demos on jsapi and sphinx is not clear for me too.

About original problem, of course it's better to work with trunk, the corresponding code seems missing now. But it seems it's not related to config but to the model itself. Number of tied states is not correct probably, what about state number 123 there? ldb88, can you please give a link to mdef file from a model? Better send me the model jar, by mail for example.

[John Hall]

Here is a link to the .jar file:
http://rapidshare.com/files/84626619/commands.jar.html

Thanks.

[Nickolay V. Shmyrev]

Oh, well, now it's clear.

1) You are using context dependent mdef cd_cont.1000.mdef with context independant means and variances. In model props use ci_cont mdef, it will work fine. also set correct number of gaussians (I suppose it's 1 for ci).

2) For US English there is no sense to train your own model, use wsj instead. You won't get better performance unless you have enormous amount of data.

3) Even if you decided to train, either use more states or use smaller phonetic units (syllables or phones). If you are using unique or big units like syllables or words, do not train context dependant models, they are senseless. Use ci models instead.