Menu

accuracyTracker not showing any results

Help
2015-08-04
2015-09-02
  • Cory J Edwards

    Cory J Edwards - 2015-08-04

    I have been trying to get the accuracyTracker to show what the accuracy, but no matter what I change in my default.comfig.xml all it will show is the speed and memory tracker.
    Here is the config file:

    <config></config>

    <property name="logLevel" value="INFO">
    <property name="recognizer" value="recognizer"></property></property>

    <property name="absoluteBeamWidth" value="20000">
    <property name="relativeBeamWidth" value="1e-60">
    <property name="absoluteWordBeamWidth" value="200">
    <property name="relativeWordBeamWidth" value="1e-40"></property></property></property></property>

    <property name="wordInsertionProbability" value="0.1">
    <property name="silenceInsertionProbability" value="0.1">
    <property name="fillerInsertionProbability" value="1e-2"></property></property></property>

    <property name="phoneticLookaheadWindow" value="5">
    <property name="phoneticLookaheadWeight" value="6">
    <property name="acousticLookaheadWeight" value="1.7">
    <property name="phoneticBeam" value="1e-12"></property></property></property></property>

    <property name="oogProbability" value="1e-30">
    <property name="oogLoopProbability" value="1e-10"></property></property>

    <property name="languageWeight" value="8.0"></property>

    <component name="batch" type="edu.cmu.sphinx.tools.batch.BatchModeRecognizer">
    <propertylist name="inputDataProcessors">
    <item>streamDataSource</item>
    </propertylist>
    <property name="skip" value="0">
    <property name="recognizer" value="${recognizer}">
    <property name="logLevel" value="FINEST">
    <property name="logTerse" value="true">
    </property></property></property></property></component>

    <component name="recognizer" type="edu.cmu.sphinx.recognizer.Recognizer">
    <property name="decoder" value="decoder">
    <propertylist name="monitors">
    <item>speedTracker</item>
    <item>memoryTracker</item>
    <item>accuracyTracker </item>
    <item>recognizerMonitor </item>
    </propertylist>
    </property></component>

    <component name="decoder" type="edu.cmu.sphinx.decoder.Decoder">
    <property name="searchManager" value="wordPruningLookaheadSearchManager">
    </property></component>

    <component name="alignerSearchManager" type="edu.cmu.sphinx.decoder.search.SimpleBreadthFirstSearchManager">
    <property name="linguist" value="alignerLinguist">
    <property name="pruner" value="trivialPruner">
    <property name="scorer" value="trivialScorer">
    <property name="activeListFactory" value="activeList">
    </property></property></property></property></component>

    <component name="allphoneSearchManager" type="edu.cmu.sphinx.decoder.search.SimpleBreadthFirstSearchManager">
    <property name="linguist" value="allphoneLinguist">
    <property name="pruner" value="trivialPruner">
    <property name="scorer" value="trivialScorer">
    <property name="activeListFactory" value="activeList">
    </property></property></property></property></component>

    <component name="simpleSearchManager" type="edu.cmu.sphinx.decoder.search.SimpleBreadthFirstSearchManager">
    <property name="linguist" value="flatLinguist">
    <property name="pruner" value="trivialPruner">
    <property name="scorer" value="trivialScorer">
    <property name="activeListFactory" value="activeList">
    </property></property></property></property></component>

    <component name="wordPruningSearchManager" type="edu.cmu.sphinx.decoder.search.WordPruningBreadthFirstSearchManager">
    <property name="linguist" value="lexTreeLinguist">
    <property name="pruner" value="trivialPruner">
    <property name="scorer" value="trivialScorer">
    <property name="activeListManager" value="activeListManager"></property></property></property></property></component>

    <property name="growSkipInterval" value="0"/>
    
    <property name="buildWordLattice" value="true"/>
    <property name="keepAllTokens" value="true"/>
    
    <property name="acousticLookaheadFrames" value="${acousticLookaheadWeight}"/>
    <property name="relativeBeamWidth" value="${relativeBeamWidth}"/>
    

    <component name="wordPruningLookaheadSearchManager" type="edu.cmu.sphinx.decoder.search.WordPruningBreadthFirstLookaheadSearchManager">
    <property name="linguist" value="lexTreeLinguist">
    <property name="fastmatchLinguist" value="allphoneLinguist">
    <property name="loader" value="acousticModelLoader">
    <property name="pruner" value="trivialPruner">
    <property name="scorer" value="trivialScorer">
    <property name="activeListManager" value="activeListManager">
    <property name="fastmatchActiveListFactory" value="fastmatchActiveList"></property></property></property></property></property></property></property></component>

    <property name="growSkipInterval" value="0"/>
    <property name="buildWordLattice" value="true"/>
    <property name="keepAllTokens" value="true"/>
    
    <property name="lookaheadWindow" value="${phoneticLookaheadWindow}"/>
    <property name="lookaheadPenaltyWeight" value="${phoneticLookaheadWeight}"/>
    <property name="acousticLookaheadFrames" value="${acousticLookaheadWeight}"/>
    <property name="relativeBeamWidth" value="${relativeBeamWidth}"/>
    

    <component name="activeList" type="edu.cmu.sphinx.decoder.search.PartitionActiveListFactory">
    <property name="absoluteBeamWidth" value="${absoluteBeamWidth}">
    <property name="relativeBeamWidth" value="${relativeBeamWidth}">
    </property></property></component>

    <component name="fastmatchActiveList" type="edu.cmu.sphinx.decoder.search.PartitionActiveListFactory">
    <property name="absoluteBeamWidth" value="${absoluteBeamWidth}">
    <property name="relativeBeamWidth" value="${phoneticBeam}">
    </property></property></component>

    <component name="activeListManager" type="edu.cmu.sphinx.decoder.search.SimpleActiveListManager">
    <propertylist name="activeListFactories">
    <item>standardActiveListFactory</item>
    <item>wordActiveListFactory</item>
    <item>wordActiveListFactory</item>
    <item>standardActiveListFactory</item>
    <item>standardActiveListFactory</item>
    <item>standardActiveListFactory</item>
    </propertylist>
    </component>

    <component name="standardActiveListFactory" type="edu.cmu.sphinx.decoder.search.PartitionActiveListFactory">
    <property name="absoluteBeamWidth" value="${absoluteBeamWidth}">
    <property name="relativeBeamWidth" value="${relativeBeamWidth}">
    </property></property></component>

    <component name="wordActiveListFactory" type="edu.cmu.sphinx.decoder.search.PartitionActiveListFactory">
    <property name="absoluteBeamWidth" value="${absoluteWordBeamWidth}">
    <property name="relativeBeamWidth" value="${relativeWordBeamWidth}">
    </property></property></component>

    <component name="trivialPruner" type="edu.cmu.sphinx.decoder.pruner.SimplePruner"></component>

    <component name="trivialScorer" type="edu.cmu.sphinx.decoder.scorer.SimpleAcousticScorer">
    <property name="frontend" value="liveFrontEnd">
    </property></component>

    <component name="alignerLinguist" type="edu.cmu.sphinx.linguist.flat.FlatLinguist">
    <property name="grammar" value="alignerGrammar">
    <property name="acousticModel" value="acousticModel">
    <property name="wordInsertionProbability" value="${wordInsertionProbability}">
    <property name="silenceInsertionProbability" value="${silenceInsertionProbability}">
    <property name="languageWeight" value="${languageWeight}">
    <property name="unitManager" value="unitManager">
    </property></property></property></property></property></property></component>

    <component name="allphoneLinguist" type="edu.cmu.sphinx.linguist.allphone.AllphoneLinguist">
    <property name="acousticModel" value="acousticModel">
    <property name="useContextDependentPhones" value="false">
    <property name="phoneInsertionProbability" value="0.05">
    </property></property></property></component>

    <component name="flatLinguist" type="edu.cmu.sphinx.linguist.flat.FlatLinguist">
    <property name="grammar" value="jsgfGrammar">
    <property name="acousticModel" value="acousticModel">
    <property name="wordInsertionProbability" value="${wordInsertionProbability}">
    <property name="silenceInsertionProbability" value="${silenceInsertionProbability}">
    <property name="languageWeight" value="${languageWeight}">
    <property name="unitManager" value="unitManager"></property></property></property></property></property></property></component>

    <property name="addOutOfGrammarBranch" value="true"/>
    <property name="outOfGrammarProbability" value="${oogProbability}"/>
    <property name="phoneInsertionProbability" value="${oogLoopProbability}"/>
    <property name="phoneLoopAcousticModel" value="acousticModel"/>
    

    <component name="lexTreeLinguist" type="edu.cmu.sphinx.linguist.lextree.LexTreeLinguist">
    <property name="acousticModel" value="acousticModel">
    <property name="languageModel" value="simpleNGramModel">
    <property name="dictionary" value="dictionary">
    <property name="addFillerWords" value="true">
    <property name="generateUnitStates" value="false">
    <property name="wantUnigramSmear" value="true">
    <property name="unigramSmearWeight" value="1">
    <property name="wordInsertionProbability" value="${wordInsertionProbability}">
    <property name="silenceInsertionProbability" value="${silenceInsertionProbability}">
    <property name="fillerInsertionProbability" value="${fillerInsertionProbability}">
    <property name="languageWeight" value="${languageWeight}">
    <property name="unitManager" value="unitManager">
    </property></property></property></property></property></property></property></property></property></property></property></property></component>

    <component name="trieNgramModel" type="edu.cmu.sphinx.linguist.language.ngram.trie.NgramTrieModel">
    <property name="location" value="">
    <property name="dictionary" value="dictionary">
    <property name="maxDepth" value="3">
    </property></property></property></component>

    <component name="simpleNGramModel" type="edu.cmu.sphinx.linguist.language.ngram.SimpleNGramModel">
    <property name="location" value="">
    <property name="dictionary" value="dictionary">
    <property name="maxDepth" value="3">
    <property name="unigramWeight" value=".7">
    </property></property></property></property></component>

    <component name="largeTrigramModel" type="edu.cmu.sphinx.linguist.language.ngram.large.LargeTrigramModel">
    <property name="location" value="">
    <property name="unigramWeight" value=".5">
    <property name="maxDepth" value="3">
    <property name="dictionary" value="dictionary">
    </property></property></property></property></component>

    <component name="dynamicTrigramModel" type="edu.cmu.sphinx.linguist.language.ngram.DynamicTrigramModel">
    <property name="dictionary" value="dictionary">
    <property name="maxDepth" value="3">
    <property name="unigramWeight" value=".7">
    </property></property></property></component>

    <component name="alignerGrammar" type="edu.cmu.sphinx.linguist.language.grammar.AlignerGrammar">
    <property name="dictionary" value="dictionary">
    <property name="addSilenceWords" value="true">
    </property></property></component>

    <component name="jsgfGrammar" type="edu.cmu.sphinx.jsgf.JSGFGrammar">
    <property name="dictionary" value="dictionary">
    <property name="grammarLocation" value="">
    <property name="grammarName" value="">
    <property name="addSilenceWords" value="true">
    </property></property></property></property></component>

    <component name="grXmlGrammar" type="edu.cmu.sphinx.jsgf.GrXMLGrammar">
    <property name="dictionary" value="dictionary">
    <property name="grammarLocation" value="">
    <property name="grammarName" value="">
    <property name="addSilenceWords" value="true">
    </property></property></property></property></component>

    <component name="dictionary" type="edu.cmu.sphinx.linguist.dictionary.TextDictionary">
    <property name="dictionaryPath" value="resource:/WSJ_8gau_13dCep_16k_40mel_130Hz_6800Hz/dict/cmudict.0.6d">
    <property name="fillerPath" value="resource:/WSJ_8gau_13dCep_16k_40mel_130Hz_6800Hz/noisedict">
    <property name="unitManager" value="unitManager">
    </property></property></property></component>

    <component name="acousticModel" type="edu.cmu.sphinx.linguist.acoustic.tiedstate.TiedStateAcousticModel">
    <property name="loader" value="acousticModelLoader">
    <property name="unitManager" value="unitManager">
    </property></property></component>

    <component name="acousticModelLoader" type="edu.cmu.sphinx.linguist.acoustic.tiedstate.Sphinx3Loader">
    <property name="unitManager" value="unitManager">
    <property name="location" value="">
    <property name="topGaussiansNum" value="4">
    </property></property></property></component>

    <component name="unitManager" type="edu.cmu.sphinx.linguist.acoustic.UnitManager"></component>

    <component name="liveFrontEnd" type="edu.cmu.sphinx.frontend.FrontEnd">
    <propertylist name="pipeline">
    <item>dataSource </item>
    <item>dataBlocker </item>
    <item>speechClassifier </item>
    <item>speechMarker </item>
    <item>preemphasizer </item>
    <item>windower </item>
    <item>fft </item>
    <item>autoCepstrum </item>
    <item>liveCMN </item>
    <item>featureExtraction </item>
    <item>featureTransform </item>
    </propertylist>
    </component>

    <component name="dataSource" type="edu.cmu.sphinx.frontend.util.StreamDataSource"></component>

    <component name="dataBlocker" type="edu.cmu.sphinx.frontend.DataBlocker"></component>

    <component name="dataDumper" type="edu.cmu.sphinx.frontend.util.DataDumper"></component>

    <component name="speechClassifier" type="edu.cmu.sphinx.frontend.endpoint.SpeechClassifier">
    <property name="threshold" value="13">
    </property></component>

    <component name="speechMarker" type="edu.cmu.sphinx.frontend.endpoint.SpeechMarker">
    </component>

    <component name="preemphasizer" type="edu.cmu.sphinx.frontend.filter.Preemphasizer"></component>

    <component name="windower" type="edu.cmu.sphinx.frontend.window.RaisedCosineWindower">
    </component>

    <component name="fft" type="edu.cmu.sphinx.frontend.transform.DiscreteFourierTransform">
    </component>

    <component name="autoCepstrum" type="edu.cmu.sphinx.frontend.AutoCepstrum">
    <property name="loader" value="acousticModelLoader">
    </property></component>

    <component name="batchCMN" type="edu.cmu.sphinx.frontend.feature.BatchCMN"></component>

    <component name="liveCMN" type="edu.cmu.sphinx.frontend.feature.LiveCMN"></component>

    <component name="featureExtraction" type="edu.cmu.sphinx.frontend.feature.DeltasFeatureExtractor"></component>

    <component name="featureTransform" type="edu.cmu.sphinx.frontend.feature.FeatureTransform">
    <property name="loader" value="acousticModelLoader">
    </property></component>

    <component name="speedTracker" type="edu.cmu.sphinx.instrumentation.SpeedTracker">
    <property name="showTimers" value="true">
    <property name="frontend" value="liveFrontEnd">
    <property name="recognizer" value="${recognizer}">
    </property></property></property></component>

    <component name="memoryTracker" type="edu.cmu.sphinx.instrumentation.MemoryTracker">
    <property name="recognizer" value="${recognizer}">
    </property></component>

    <component name="accuracyTracker" type="edu.cmu.sphinx.instrumentation.BestPathAccuracyTracker">
    <property name="recognizer" value="${recognizer}">
    <property name="showAlignedResults" value="true">
    <property name="showRawResults" value="true">
    <property name="logLevel" value="INFO">
    </property></property></property></property></component>

    <component name="recognizerMonitor" type="edu.cmu.sphinx.instrumentation.RecognizerMonitor">
    <property name="recognizer" value="${recognizer}">
    <propertylist name="allocatedMonitors">
    <item>configMonitor </item>
    </propertylist>
    </property></component>

    <component name="configMonitor" <br=""> type="edu.cmu.sphinx.instrumentation.ConfigMonitor">
    <property name="showConfig" value="true">
    </property></component>


    If I am not showing the wrong thing or miss some needed information just tell me. I am still quite new to working with Spinx, and thanks in advance for any help.

     
    • Nickolay V. Shmyrev

      Accuracy tracker only works together with Batch manager during batch accuracy testing, it is not supposed to be used in normal sphinx4 course.

      You can find examples of batch accuracy testing in sphinx4/tests/performance folder.

       
  • Cory J Edwards

    Cory J Edwards - 2015-08-04

    ok so i have to run the class for the batch manager to run and print results. Is there any way I could force it to run in a program just to see the accuracy then for when I go to make the .jar I can comment out the code? Also, I tried running an4 in sphinx4/tests/performance and it said it couldn't find the batchModeRecognizer, and I couldn't find the buld.xml for Sphinx4.

     
    • Nickolay V. Shmyrev

      to run in a program just to see the accuracy

      Accuracy estimation requires reference transcription so that the result is compared to ground truth, so you can not "force it run in a program", you need to prepare reference file first, then run batch recognizer with required arguments.

      Also, I tried running an4 in sphinx4/tests/performance and it said it couldn't find the batchModeRecognizer

      Maybe you need to adjust classpath.

       
      • Cory J Edwards

        Cory J Edwards - 2015-08-04

        In the folder there is no classes at all though just .java's so I am not sure what the path should be. Is there a way to retrieve and add to a reference file while the code is running. My goal is to retrieve the accuracy % and us that as referece and have a program change the settings automatically while it is running to get at and keep 100% or as close as I could get to is.

         
        • Nickolay V. Shmyrev

          Sorry, it is hard to understand what are you looking for, but most likely you confuse accuracy and confidence.

           
          • Cory J Edwards

            Cory J Edwards - 2015-09-02

            From my understanding the confidence is the system saying how confident it is in the words it hears. Where as the accuracy is finding those words in the first place. Now I have looked in both accuracy java docs for what actually makes the accuracy and I can not find it unless I have to look in the recognizer itself which I have not yet. Whether it is confidence or accruacy I need I just want to make the recognizer learn from its surroundings pretty much by automaitcally adjusting the settings in the deafult.config.xml so that it could maybe have to focus more and drown out or whether anything else. This is why I want that number because I want to get it to 100% or as close as I can get by having it adjust those settings. I just need to find that number and extract it to use it.

             
  • Cory J Edwards

    Cory J Edwards - 2015-08-08

    Sorry to be writing back late. I had a busy last few days. Yeah I have not thought about confidence. What is the difference between the two? What I am exactly looking for is a variable that I can check after every recognition. Then after the check I want to max out that variable such as 100% and if it is not or not close to it then have a program that changes the global variables in the default.config.xml file to try and get the original variable back up. This is because the program I am making with Sphinx will be in a lot of differenct environments, and I want it to be able to adjust itself to compensate for the differences in the environments.

     

Log in to post a comment.