Tips for improving performance (Sphinx4)?

[d_h_benson]

Hi,

I'm using Sphinx4 to analyze audio files with a highly restrictive grammar. I'm not getting terribly useful results, however. I wonder if any of you might be able to have a quick look at the data and configuration file and suggest how to improve the performance. Any assistance would be greatly appreciated.

Here's a bit of background:

The files I'm analyzing are responses from a psychological experiment investigating reading speed. Subjects were presented with a screen full of letters, and were asked to read them back as quickly as possible. As a result, the audio files contain just a few letters -- J, H, F, U and V -- repeated several times in a random order.

I'm trying to extract both the sequence of letters that were read, and the timings of the utterances.

The program I wrote is based on the HelloWorld demo and the Wav transcription demo. Since the set of possible utterances is so small, though, I wrote a simple new grammar definition as well.

Below are links to relevant files and a complete copy of the configuration file I'm using. If you need anything else, or would prefer these in a different format, I'll gladly oblige.

If any of you have any bright ideas, please send them my way!

Many thanks,

Dave Benson

>>

Here's the grammar definition:
http://www.mediafire.com/file/vyvm0grn8ny/letters.gram

Here's the main class of the application:
http://www.mediafire.com/file/jzwt9dvurbw/WordTimings.java

Here's a sample audio file:
http://www.mediafire.com/file/7baafegaaaz/test_file.wav

Here's an audacity project showing the audio file and the resulting analysis:
http://www.mediafire.com/file/jq3ndhkcna7/test_file_audacity_project.zip

Here's the config file:
http://www.mediafire.com/file/zchocuucgcw/wordtimings_letters.config.xml

>>
<?xml version="1.0" encoding="UTF-8"?>

<!--
Sphinx-4 Configuration file
-->

<!-- ******** -->
<!-- an4 configuration file -->
<!-- ******** -->

<config>

<!-- ******************************************************** -->
<!-- frequently tuned properties                              -->
<!-- ******************************************************** -->

<property name="logLevel" value="WARNING"/>

<property name="absoluteBeamWidth"  value="-1"/>
<property name="relativeBeamWidth"  value="1E-80"/>
<property name="wordInsertionProbability" value="1E-36"/>

<!-- <property name="silenceInsertionProbability" value="0.1"/> -->
<property name="languageWeight" value="8"/>

<property name="frontend" value="epFrontEnd"/>
<property name="recognizer" value="recognizer"/>
<property name="showCreations" value="false"/>

<!-- ******************************************************** -->
<!-- word recognizer configuration                            -->
<!-- ******************************************************** -->

<component name="recognizer" type="edu.cmu.sphinx.recognizer.Recognizer">
    <property name="decoder" value="decoder"/>
    <propertylist name="monitors">
        <item>accuracyTracker </item>
        <item>speedTracker </item>
        <item>memoryTracker </item>
    </propertylist>

</component>

<!-- ******************************************************** -->
<!-- The Decoder   configuration                              -->
<!-- ******************************************************** -->

<component name="decoder" type="edu.cmu.sphinx.decoder.Decoder">
    <property name="searchManager" value="searchManager"/>
</component>

<component name="searchManager" 
    type="edu.cmu.sphinx.decoder.search.SimpleBreadthFirstSearchManager">
    <property name="logMath" value="logMath"/>
    <property name="linguist" value="flatLinguist"/>
    <property name="pruner" value="trivialPruner"/>
    <property name="scorer" value="threadedScorer"/>
    <property name="activeListFactory" value="activeList"/>
</component>

<component name="activeList" 
         type="edu.cmu.sphinx.decoder.search.PartitionActiveListFactory">
    <property name="logMath" value="logMath"/>
    <property name="absoluteBeamWidth" value="${absoluteBeamWidth}"/>
    <property name="relativeBeamWidth" value="${relativeBeamWidth}"/>
</component>

<component name="trivialPruner" 
            type="edu.cmu.sphinx.decoder.pruner.SimplePruner"/>

<component name="threadedScorer" 
            type="edu.cmu.sphinx.decoder.scorer.ThreadedAcousticScorer">
    <property name="frontend" value="${frontend}"/>
    <property name="isCpuRelative" value="true"/>
    <property name="numThreads" value="0"/>
    <property name="minScoreablesPerThread" value="10"/>
    <property name="scoreablesKeepFeature" value="true"/>
</component>

<!-- ******************************************************** -->
<!-- The linguist  configuration                              -->
<!-- ******************************************************** -->

<component name="flatLinguist" 
            type="edu.cmu.sphinx.linguist.flat.FlatLinguist">
    <property name="logMath" value="logMath"/>
    <property name="grammar" value="jsgfGrammar"/>
    <property name="acousticModel" value="wsj"/>
    <property name="wordInsertionProbability" 
            value="${wordInsertionProbability}"/>
    <property name="languageWeight" value="${languageWeight}"/>
    <property name="unitManager" value="unitManager"/>
</component>

<!-- ******************************************************** -->
<!-- The Grammar  configuration                               -->
<!-- ******************************************************** -->

<component name="jsgfGrammar" type="edu.cmu.sphinx.jsapi.JSGFGrammar">
    <property name="dictionary" value="dictionary"/>
<!--     <property name="grammarLocation" 
         value="resource:/demo.sphinx.helloworld.HelloWorld!/demo/sphinx/helloworld/"/>
-->
    <property name="grammarLocation" 
         value="bin/demo/sphinx/wordtimings/"/>
    <property name="grammarName" value="letters"/>
<property name="logMath" value="logMath"/>
</component>

<!-- ******************************************************** -->
<!-- The Dictionary configuration                            -->
<!-- ******************************************************** -->

<component name="dictionary" 
    type="edu.cmu.sphinx.linguist.dictionary.FastDictionary">
    <property name="dictionaryPath"
 value="resource:/edu.cmu.sphinx.model.acoustic.WSJ_8gau_13dCep_16k_40mel_130Hz_6800Hz.Model!/edu/cmu/sphinx/model/acoustic/WSJ_8gau_13dCep_16k_40mel_130Hz_6800Hz/dict/cmudict.0.6d"/>
    <property name="fillerPath" 
 value="resource:/edu.cmu.sphinx.model.acoustic.WSJ_8gau_13dCep_16k_40mel_130Hz_6800Hz.Model!/edu/cmu/sphinx/model/acoustic/WSJ_8gau_13dCep_16k_40mel_130Hz_6800Hz/dict/fillerdict"/>
    <property name="addSilEndingPronunciation" value="true"/>

<!-- <property name="wordReplacement" value="&lt;sil&gt;"/>
<property name="allowMissingWords" value="false"/> -->
<property name="unitManager" value="unitManager"/>
</component>

<!-- ******************************************************** -->
<!-- The acoustic model configuration                         -->
<!-- ******************************************************** -->
<component name="wsj" 
  type="edu.cmu.sphinx.model.acoustic.WSJ_8gau_13dCep_16k_40mel_130Hz_6800Hz.Model">
    <property name="loader" value="wsjLoader"/>
    <property name="unitManager" value="unitManager"/>
</component>

<component name="wsjLoader"
           type="edu.cmu.sphinx.model.acoustic.WSJ_8gau_13dCep_16k_40mel_130Hz_6800Hz.ModelLoader">
    <property name="logMath" value="logMath"/>
    <property name="unitManager" value="unitManager"/>
</component>

<!-- ******************************************************** -->
<!-- The unit manager configuration                           -->
<!-- ******************************************************** -->

<component name="unitManager" 
    type="edu.cmu.sphinx.linguist.acoustic.UnitManager"/>

<!-- ******************************************************** -->
<!-- The live frontend configuration                          -->
<!-- ******************************************************** -->
<component name="epFrontEnd" type="edu.cmu.sphinx.frontend.FrontEnd">
    <propertylist name="pipeline">
        <!-- <item>microphone </item> -->
        <item>streamDataSource </item>
        <item>speechClassifier </item>      
        <item>speechMarker </item>          
        <item>nonSpeechDataFilter </item>   
        <item>premphasizer </item>
        <item>windower </item>
        <item>fft </item>
        <item>melFilterBank </item>
        <item>dct </item>
        <item>liveCMN </item>
        <item>featureExtraction </item>
    </propertylist>
</component>

<!-- ******************************************************** -->
<!-- The frontend without endpointer configuration                          -->
<!-- ******************************************************** -->
<component name="nonEpFrontEnd" type="edu.cmu.sphinx.frontend.FrontEnd">
    <propertylist name="pipeline">
        <!-- <item>microphone </item> -->
        <item>streamDataSource </item>
        <item>premphasizer </item>
        <item>windower </item>
        <item>fft </item>
        <item>melFilterBank </item>
        <item>dct </item>
        <item>liveCMN </item>
        <item>featureExtraction </item>
    </propertylist>
</component>

<!-- ******************************************************** -->
<!-- The frontend pipelines                                   -->
<!-- ******************************************************** -->

<component name="streamDataSource"
            type="edu.cmu.sphinx.frontend.util.StreamDataSource">
    <property name="sampleRate" value="44100"/>
    <property name="bitsPerSample" value="16"/>
    <property name="bigEndianData" value="false"/>
    <property name="signedData" value="true"/>
    <property name="bytesPerRead" value="320"/>
</component>

<!-- <component name="microphone"
type="edu.cmu.sphinx.frontend.util.Microphone">
<property name="msecPerRead" value="10"/>
<property name="closeBetweenUtterances" value="false"/>
</component>
-->

<component name="speechClassifier" 
           type="edu.cmu.sphinx.frontend.endpoint.SpeechClassifier">
    <property name="threshold" value="13"/>
</component>

<component name="nonSpeechDataFilter" 
           type="edu.cmu.sphinx.frontend.endpoint.NonSpeechDataFilter"/>

<component name="speechMarker" 
           type="edu.cmu.sphinx.frontend.endpoint.SpeechMarker" >
    <property name="speechTrailer" value="50"/>
</component>

<component name="premphasizer" 
           type="edu.cmu.sphinx.frontend.filter.Preemphasizer"/>

<component name="windower" 
           type="edu.cmu.sphinx.frontend.window.RaisedCosineWindower">
</component>

<component name="fft" 
        type="edu.cmu.sphinx.frontend.transform.DiscreteFourierTransform">
</component>

<component name="melFilterBank" 
    type="edu.cmu.sphinx.frontend.frequencywarp.MelFrequencyFilterBank">
</component>

<component name="dct" 
        type="edu.cmu.sphinx.frontend.transform.DiscreteCosineTransform"/>

<component name="liveCMN" 
           type="edu.cmu.sphinx.frontend.feature.LiveCMN"/>

<component name="featureExtraction" 
           type="edu.cmu.sphinx.frontend.feature.DeltasFeatureExtractor"/>

<!-- ******************************************************* -->
<!--  monitors                                               -->
<!-- ******************************************************* -->

<component name="accuracyTracker" 
            type="edu.cmu.sphinx.instrumentation.AccuracyTracker">
    <property name="recognizer" value="${recognizer}"/>
    <property name="showAlignedResults" value="false"/>
    <property name="showRawResults" value="false"/>
</component>

<component name="memoryTracker" 
            type="edu.cmu.sphinx.instrumentation.MemoryTracker">
    <property name="recognizer" value="${recognizer}"/>
<property name="showSummary" value="false"/>
<property name="showDetails" value="false"/>
</component>

<component name="speedTracker" 
            type="edu.cmu.sphinx.instrumentation.SpeedTracker">
    <property name="recognizer" value="${recognizer}"/>
    <property name="frontend" value="${frontend}"/>
<property name="showSummary" value="true"/>
<property name="showDetails" value="false"/>
</component>

<!-- ******************************************************* -->
<!--  Miscellaneous components                               -->
<!-- ******************************************************* -->

<component name="logMath" type="edu.cmu.sphinx.util.LogMath">
    <property name="logBase" value="1.0001"/>
    <property name="useAddTable" value="true"/>
</component>

</config>

[d_h_benson]

As a secondary question, in some segments Sphinx4 fails to recognize any words at all. Do you know what settings I might be able to change to avoid this?

[Nickolay V. Shmyrev]

(00:55:25) nshm: d_h_benson: <property name="sampleRate" value="44100"/>  <- this will not work for sure
(00:55:33) nshm: samplerate must be 16000
(00:55:48) nshm: but you can use AudioStreamDataSource that will do conversion for you