No result (WavFile demo with WSJ and Ngram)

[Zach Poff]

Hi. I've been trying to get my head around Sphinx4 for days. (recent SVN) Absolutely bewildering for noobs like me to get simple dictation working, but that's 100% my problem, not Sphinx!

I copied some config.xml options from HelloNGram to WavFile, hoping to increase the vocabulary. I am using the WSJ5K LM and the WSJ acoustic model + dictionary. For now, I'm sticking with the included wav file and stream settings. Classpath, etc have been updated and it doesn't throw any errors. It doesn't produce any results though. The process finishes and it prints "RESULT:"

I first tried converting to the WSJ acoustic model while sticking with the original grammar file. It worked. I was able to add words to the grammar and test new soundfiles and it was fine. Then I jumped into the NGram stuff and the WSJ5K model and here we are. I've tried adjusting beamwidths as per previous posts, and switching the LM to HUB4, but I think there's something much more elemental that I'm missing.

Any ideas?

my config file:

<?xml version="1.0" encoding="UTF-8"?>

<!--
Sphinx-4 Configuration file
-->

<!-- ******** -->
<!-- wavFile configuration file (converted to WSJ + NGRAM) -->
<!-- ******** -->

<config>

<!-- ******************************************************** -->
<!-- frequently tuned properties                              -->
<!-- ******************************************************** -->

<property name="absoluteBeamWidth" value="500"/>
<property name="relativeBeamWidth" value="1E-80"/>
<property name="absoluteWordBeamWidth" value="20"/>
<property name="relativeWordBeamWidth" value="1E-60"/>
<property name="wordInsertionProbability" value="1E-36"/>
<property name="languageWeight" value="7"/>
<property name="silenceInsertionProbability" value=".1"/>
<property name="skip" value="0"/>
<property name="logLevel" value="WARNING"/>

<property name="recognizer" value="recognizer"/>
<property name="linguist" value="flatLinguist"/>
<property name="frontend" value="mfcFrontEnd"/>

<!-- ******************************************************** -->
<!-- The Recognizer configuration                             -->
<!-- ******************************************************** -->

<component name="recognizer" 
                      type="edu.cmu.sphinx.recognizer.Recognizer">
    <property name="decoder" value="decoder"/>
    <propertylist name="monitors">
        <item>accuracyTracker </item>
        <item>speedTracker </item>
        <item>memoryTracker </item>
        <item>recognizerMonitor </item>
    </propertylist>
</component>

<!-- ******************************************************** -->
<!-- The Decoder   configuration                              -->
<!-- ******************************************************** -->

<component name="decoder" type="edu.cmu.sphinx.decoder.Decoder">
    <property name="searchManager" value="wordPruningSearchManager"/>
    <property name="featureBlockSize" value="50"/>
</component>

<!-- ******************************************************** -->
<!-- The Search Manager                                       -->
<!-- ******************************************************** -->

<component name="wordPruningSearchManager" 
type="edu.cmu.sphinx.decoder.search.WordPruningBreadthFirstSearchManager">
    <property name="logMath" value="logMath"/>
    <property name="linguist" value="lexTreeLinguist"/>
    <property name="pruner" value="trivialPruner"/>
    <property name="scorer" value="threadedScorer"/>
    <property name="activeListManager" value="activeListManager"/>
    <property name="growSkipInterval" value="0"/>
    <property name="checkStateOrder" value="false"/>
    <property name="buildWordLattice" value="false"/>
    <property name="acousticLookaheadFrames" value="1.7"/>
    <property name="relativeBeamWidth" value="${relativeBeamWidth}"/>
</component>

<!-- ******************************************************** -->
<!-- The Active Lists                                         -->
<!-- ******************************************************** -->

<component name="activeListManager" 
         type="edu.cmu.sphinx.decoder.search.SimpleActiveListManager">
    <propertylist name="activeListFactories">
    <item>standardActiveListFactory</item>
    <item>wordActiveListFactory</item>
    <item>wordActiveListFactory</item>
    <item>standardActiveListFactory</item>
    <item>standardActiveListFactory</item>
    <item>standardActiveListFactory</item>
</propertylist>
</component>

<component name="standardActiveListFactory" 
         type="edu.cmu.sphinx.decoder.search.PartitionActiveListFactory">
    <property name="logMath" value="logMath"/>
    <property name="absoluteBeamWidth" value="${absoluteBeamWidth}"/>
    <property name="relativeBeamWidth" value="${relativeBeamWidth}"/>
</component>

<component name="wordActiveListFactory" 
         type="edu.cmu.sphinx.decoder.search.PartitionActiveListFactory">
    <property name="logMath" value="logMath"/>
    <property name="absoluteBeamWidth" value="${absoluteWordBeamWidth}"/>
    <property name="relativeBeamWidth" value="${relativeWordBeamWidth}"/>
</component>

<!-- ******************************************************** -->
<!-- The Pruner                                               -->
<!-- ******************************************************** --> 
<component name="trivialPruner" 
            type="edu.cmu.sphinx.decoder.pruner.SimplePruner"/>

<!-- ******************************************************** -->
<!-- TheScorer                                                -->
<!-- ******************************************************** --> 
<component name="threadedScorer" 
            type="edu.cmu.sphinx.decoder.scorer.ThreadedAcousticScorer">
    <property name="frontend" value="${frontend}"/>
    <property name="isCpuRelative" value="true"/>
    <property name="numThreads" value="0"/>
    <property name="minScoreablesPerThread" value="10"/>
    <property name="scoreablesKeepFeature" value="true"/>
</component>

<!-- ******************************************************** -->
<!-- The linguist  configuration                              -->
<!-- ******************************************************** -->

<component name="lexTreeLinguist" 
            type="edu.cmu.sphinx.linguist.lextree.LexTreeLinguist">
    <property name="logMath" value="logMath"/>
    <property name="acousticModel" value="wsj"/>
    <property name="languageModel" value="trigramModel"/>
    <property name="dictionary" value="dictionary"/>
    <property name="addFillerWords" value="false"/>
    <property name="fillerInsertionProbability" value="1E-10"/>
    <property name="generateUnitStates" value="false"/>
    <property name="wantUnigramSmear" value="true"/>
    <property name="unigramSmearWeight" value="1"/>
    <property name="wordInsertionProbability" 
            value="${wordInsertionProbability}"/>
    <property name="silenceInsertionProbability" 
            value="${silenceInsertionProbability}"/>
    <property name="languageWeight" value="${languageWeight}"/>
    <property name="unitManager" value="unitManager"/>
</component>

<!-- ******************************************************** -->
<!-- The Dictionary configuration                            -->
<!-- ******************************************************** -->
<component name="dictionary" 
    type="edu.cmu.sphinx.linguist.dictionary.FastDictionary">
    <property name="dictionaryPath"
              value="resource:/edu.cmu.sphinx.model.acoustic.WSJ_8gau_13dCep_16k_40mel_130Hz_6800Hz.Model!/edu/cmu/sphinx/model/acoustic/WSJ_8gau_13dCep_16k_40mel_130Hz_6800Hz/dict/cmudict.0.6d"/>
    <property name="fillerPath" 
          value="resource:/edu.cmu.sphinx.model.acoustic.WSJ_8gau_13dCep_16k_40mel_130Hz_6800Hz.Model!/edu/cmu/sphinx/model/acoustic/WSJ_8gau_13dCep_16k_40mel_130Hz_6800Hz/dict/fillerdict"/>
    <property name="addSilEndingPronunciation" value="false"/>
    <property name="wordReplacement" value="<sil>"/>
    <property name="unitManager" value="unitManager"/>
</component>

<!-- ******************************************************** -->
<!-- The Language Model configuration                         -->
<!-- ******************************************************** -->
<component name="trigramModel" 
    type="edu.cmu.sphinx.linguist.language.ngram.large.LargeTrigramModel">
    <property name="location" value="/WSJ/wsj5k.DMP"/>

<!-- value="/HUB4/language_model.arpaformat.DMP"/> -->

    <property name="logMath" value="logMath"/>
    <property name="dictionary" value="dictionary"/>
    <property name="maxDepth" value="3"/>
    <property name="unigramWeight" value=".7"/>
</component>

<!-- ******************************************************** -->
<!-- The acoustic model configuration                         -->
<!-- ******************************************************** -->

<component name="wsj"
           type="edu.cmu.sphinx.model.acoustic.WSJ_8gau_13dCep_16k_40mel_130Hz_6800Hz.Model">
    <property name="loader" value="sphinx3Loader"/>
    <property name="unitManager" value="unitManager"/>
</component>

<component name="sphinx3Loader"
           type="edu.cmu.sphinx.model.acoustic.WSJ_8gau_13dCep_16k_40mel_130Hz_6800Hz.ModelLoader">
    <property name="logMath" value="logMath"/>
    <property name="unitManager" value="unitManager"/>
</component>

<!-- ******************************************************** -->
<!-- The unit manager configuration                           -->
<!-- ******************************************************** -->

<component name="unitManager"
           type="edu.cmu.sphinx.linguist.acoustic.UnitManager"/>

<!-- ******************************************************** -->
<!-- The frontend configuration                               -->
<!-- ******************************************************** -->

<component name="mfcFrontEnd" type="edu.cmu.sphinx.frontend.FrontEnd">
    <propertylist name="pipeline">
        <item>streamDataSource</item>
        <item>premphasizer</item>
        <item>windower</item>
        <item>fft</item>
        <item>melFilterBank</item>
        <item>dct</item>
        <item>batchCMN</item>
        <item>featureExtraction</item>
    </propertylist>
</component>

<component name="streamDataSource"
           type="edu.cmu.sphinx.frontend.util.StreamDataSource">
    <property name="sampleRate" value="16000"/>
    <property name="bitsPerSample" value="16"/>
    <property name="bigEndianData" value="false"/>
    <property name="signedData" value="true"/>
</component>

<component name="premphasizer"
           type="edu.cmu.sphinx.frontend.filter.Preemphasizer"/>

<component name="windower"
           type="edu.cmu.sphinx.frontend.window.RaisedCosineWindower"/>

<component name="fft"
           type="edu.cmu.sphinx.frontend.transform.DiscreteFourierTransform"/>

<component name="melFilterBank"
           type="edu.cmu.sphinx.frontend.frequencywarp.MelFrequencyFilterBank"/>

<component name="dct"
           type="edu.cmu.sphinx.frontend.transform.DiscreteCosineTransform"/>

<component name="batchCMN"
           type="edu.cmu.sphinx.frontend.feature.BatchCMN"/>

<component name="featureExtraction"
           type="edu.cmu.sphinx.frontend.feature.DeltasFeatureExtractor"/>

<!-- ******************************************************* -->
<!--  monitors                                               -->
<!-- ******************************************************* -->

<component name="accuracyTracker" 
            type="edu.cmu.sphinx.instrumentation.BestPathAccuracyTracker">
    <property name="recognizer" value="${recognizer}"/>
    <property name="showRawResults" value="false"/>
    <property name="showAlignedResults" value="false"/>
</component>

<component name="memoryTracker" 
            type="edu.cmu.sphinx.instrumentation.MemoryTracker">
    <property name="recognizer" value="${recognizer}"/>
<property name="showDetails" value="false"/>
<property name="showSummary" value="false"/>
</component>

<component name="speedTracker" 
            type="edu.cmu.sphinx.instrumentation.SpeedTracker">
    <property name="recognizer" value="${recognizer}"/>
    <property name="frontend" value="${frontend}"/>
<property name="showDetails" value="false"/>
</component>

<component name="recognizerMonitor" 
            type="edu.cmu.sphinx.instrumentation.RecognizerMonitor">
    <property name="recognizer" value="${recognizer}"/>
    <propertylist name="allocatedMonitors">
        <item>configMonitor </item>
    </propertylist>
</component>

<component name="configMonitor" 
            type="edu.cmu.sphinx.instrumentation.ConfigMonitor">
    <property name="showConfig" value="false"/>
</component>

<!-- ******************************************************* -->
<!--  Miscellaneous components                               -->
<!-- ******************************************************* -->

<component name="logMath" type="edu.cmu.sphinx.util.LogMath">
    <property name="logBase" value="1.0001"/>
    <property name="useAddTable" value="true"/>
</component>

</config>

[Nickolay V. Shmyrev]

Provide recording you are trying to decode please.

[Zach Poff]

The audio file is the one included with the demo (1234.WAV) I'm not adding any extra arguments at runtime (except increasing RAM) so it refers to the default file hardcoded in the class.
-ZLP

[Nickolay V. Shmyrev]

With the following header:

<property name="absoluteBeamWidth" value="5000"/>
<property name="relativeBeamWidth" value="1E-120"/>
<property name="absoluteWordBeamWidth" value="100"/>
<property name="relativeWordBeamWidth" value="1E-60"/>
<property name="wordInsertionProbability" value="0.7"/>
<property name="languageWeight" value="8.5"/>
<property name="silenceInsertionProbability" value=".1"/>
<property name="skip" value="0"/>
<property name="logLevel" value="WARNING"/>

Released version sphinx4-beta works perfectly:

RESULT: one two three four five

Trunk is quite broken actually. I promise a beer to one who will fix it :)

[Nickolay V. Shmyrev]

Ok, I found the problem, it's the same StartDataSignal in scorer. To make it work properly with trunk you need to add useStreamSignals=false in scorer properties:

<component name="threadedScorer"
type="edu.cmu.sphinx.decoder.scorer.ThreadedAcousticScorer">
<property name="frontend" value="${frontend}"/>
<property name="isCpuRelative" value="true"/>
<property name="numThreads" value="0"/>
<property name="minScoreablesPerThread" value="10"/>
<property name="scoreablesKeepFeature" value="true"/>
<property name="useStreamSignals" value="false"/>
</component>

RESULT: one two three four five

But it looks more like a workaround than a solution.

[Zach Poff]

Thanks! That solved my problem. I put the beamwidth back up to a sane value (5k) and now the WSJ5K LM is running great. I'm grateful that you gave it a look. - ZLP