[Cmusphinx-devel] Re: Energy End point viewer.

[Philip K. <Philip.Kwok@Sun.COM>]

Paul,

This is a problem I realized but delayed fixing. Thanks for bringing it 
up. The problem is as follows:

The front end has an EnergyEndpointer and a NonSpeechFilter. I want to 
insert an EnergyViewer in between the two processors. The EnergyViewer 
plots the energy levels and SPEECH_START/END and UTTERANCE_START/END 
signals to the GUI.

The EnergyViewer is not part of the SimpleFrontEnd, therefore, it has to 
be added externally. To do that I need to be able to obtain the 
EnergyEndpointer and NonSpeechFilter from the front end, and do 
reconfiguration. However, the FrontEnd interface has no way to obtain 
processors, so I need use SimpleFrontEnd, which has a getProcessor() 
method. Maybe the FrontEnd interface should have a getProcessor() method.

It looks like the better way to do it would be to extend the 
SimpleFrontEnd, to say LiveFrontEnd, and add the EnergyViewer there. 
Then we use the property "edu.cmu.sphinx.decoder.Recognizer.frontend" 
property to set the frontend to LiveFrontEnd.

For now, since we're not using EnergyViewer any more (it takes CPU 
cycles), the entire "insertEnergyEndpointViewer()" method can just be 
removed, and the problem goes away.

philip

Paul Lamere wrote:
> Good Morning Phil:
> 
> I've implemented the listeners that we talked about last week that will 
> allow us to get better timing of the utterances when in live mode.    If 
> you recall, this change allows the application to add a 'SignalListener' 
> that will be invoked when a non-content signal is generated by the 
> frontend.
> 
> In order to isolate these changes to the recognizer, to avoid having to 
> add listeners to the front end or the acoustic scorer to look for these 
> signals, I had the recognizer wrap the front end in its own front end 
> wrapper. This is a very easy way of doing things, and keeps the code 
> clean. Below is  the wrapper as it is defined in the Recognizer class.
> 
> This works fine for all of the batch mode stuff. But unfortunately it 
> doesn't work with the live mode stuff. The problem is that the live mode 
> stuff assumes that the front end is an implementation of the 
> SimpleFrontEnd.  In the LiveDecoder.insertEnergyEndpointViewer there is 
> this code:
> 
>       SimpleFrontEnd frontend = (SimpleFrontEnd) 
> getRecognizer().getFrontEnd();
> 
> 
> Since the frontend is no longer a SimpleFrontEnd, this code breaks.
> I think that we want to make sure that our demo/application code is 
> clean of such assumptions about the  underlying implementation. The 
> LiveDemo shouldn't assume that the FrontEnd is a SimpleFrontEnd unless 
> it created the front end itself and gave it to the decoder to use.
> 
> Also, in the same method there is a snippet of code:
> 
>        // are we filtering out the non-speech regions?
>        boolean filterNonSpeech = SphinxProperties.getSphinxProperties
>            (getContext()).getBoolean(FrontEnd.PROP_FILTER_NON_SPEECH, 
> true);
> 
>        if (filterNonSpeech) {
>            NonSpeechFilter filter = (NonSpeechFilter) frontend.getProcessor
>                ("NonSpeechFilter");
>            filter.setPredecessor(predecessor);
>        }
> 
> I may be just confused but this code appears to be unrelated to 
> inserting the energy endpoint viewer, so I wonder why it is in this 
> method at all.  This snippet also seems to be relying on knowledge of 
> underlying type and configuration of the FrontEnd.    I wonder, also, if 
> it is appropriate for this code to be here, it seems that code that 
> tests a FrontEnd property and modifies the FrontEnd based upon that 
> property perhaps actually belongs in the FrontEnd.    And finally, is 
> seems that this code should use the PROP_FILTER_NON_SPEECH_DEFAULT for 
> the default instead of 'true'.
> 
> Do you have any ideas how we can change the live mode stuff to not 
> depend on the type of the FrontEnd?
> 
> Paul
> 
> 
> Here's the front end wrapper.
> 
> /**
>     * A wrapper class for the front end that looks for specific
>     * feature signals and throws events based upon those
>     */
>    class FrontEndWrapper implements FrontEnd {
> 
>        FrontEnd fe;
> 
>        /**
>         * Creates a front end wrapped around the given front end
>         *
>         * @param fe the front end to wrap
>         */
>        FrontEndWrapper(FrontEnd fe) {
>            this.fe = fe;
>        }
> 
>        // wrapper for FrontEnd.drain
> 
>        public void drain() {
>            fe.drain();
>        }
> 
>        // wrapper for getFeatureFrame
>        public FeatureFrame getFeatureFrame(int numberFeatures, String
>                acousticModelName) throws IOException {
>            FeatureFrame ff =
>                  fe.getFeatureFrame(numberFeatures, acousticModelName);
>            monitorSignals(ff);
>            return ff;
>        }
> 
> 
>        // wrapper for initialize
>        public void initialize(String name, String context,
>                DataSource dataSource) throws IOException {
>            fe.initialize(name, context, dataSource);
>        }
> 
>        // wrapper for setDataSource
>        public void setDataSource(DataSource dataSource) {
>            fe.setDataSource(dataSource);
>        }
> 
> 
>        // wrapper for toString
>        public String toString() {
>            return fe.toString();
>        }
> 
>        /**
>         * Looks for non-content frames and invokes the appropriate
>         * listeners
>         *
>         * @param ff the feature frame to monitor
>         */
>        private void monitorSignals(FeatureFrame ff) {
>            Feature[] features = ff.getFeatures();
> 
>            for (int i = 0; i < features.length; i++) {
>                if (features[i].getSignal() != Signal.CONTENT) {
>                    fireSignalListeners(features[i].getSignal());
>                }
>            }
>        }
>    }
> 
>