Pls help.. Frontend Extraction in Netbeans

[Kurt Junshean Espinosa]

Hi,

I am new to Sphinx4. I searched on how to setup frontend extraction (i.e. mfcc) in netbeans and I only found this broken link: http://cmusphinx.sourceforge.net/sphinx4/doc/NetBeans.html .

I would like to ask if anyone knows how to setup sphinx4 frontend extraction in netbeans and use FeatureFileDumper to get the mfcc values given an audio file?

Many thanks!

Kurt

[Nickolay V. Shmyrev]

> how to setup sphinx4 in netbeans

http://cmusphinx.sourceforge.net/sphinx4/#setupide

> use FeatureFileDumper to get the mfcc

Check test/frontend/FrontEndTest.java and config test/frontend/cepstra_dump.xml

[Kurt Junshean Espinosa]

Hi,

I downloaded sphinx4-1.0beta2 src and bin folders but I couldn't find the files you were referring. Am I getting the right folders? Hope you could help another time.

Thank you.

[Kurt Junshean Espinosa]

Thank you very much! The first one I tried and it works.

But pls bear with me I need to ask again. I couldn't find the FrontEndTest.java and cepstra_dump.xml for in the bin and src folders of sphinx4.

I know that in this folder:

....\sphinx4-1.0beta-src\sphinx4-1.0beta\edu\cmu\sphinx\tools\feature\

is where I could find the FeatureFileDumper.java together with frontend.config.xml. I tried placing the whole folder into the src directory in netbeans project but there were several errors when I tried to build it. Can you pls guide me a bit?

Thanks.

[Nickolay V. Shmyrev]

download beta2, you aren't using the latest release.

[Kurt Junshean Espinosa]

Oh..Thank you very much for your patience.I appreciate it.

[Nickolay V. Shmyrev]

Ups, sorry, please check this out:

http://cmusphinx.org/download/nightly/sphinx4.nightly.tar.gz

[Kurt Junshean Espinosa]

Hi Nickolay,

Thank you for your quick reply! I greatly appreciate your help. I just downloaded the folder from the link you sent me and I already found the 2 files.

Actually, I would like to ask one more thing if you don't mind because honestly this is my first time in using Ant in Netbeans and I don't understand yet how to configure it. I know that this may not be the proper place to ask this question but here is what I did: I used "Create Freeform project" option but there were many errors when I tried to build it. I saw in the FrontEndTest.java the format of the usage but since I am using Netbeans I cannot figure out how to specify the parameters. I also don't know how to use cepstra_dump.xml because when I created the project, it only allows me to use the build.xml. If you please, I would like to ask for steps on how to set it up in Netbeans? I have my gmail account- kjespinosa@gmail.com - if it'll be inappropriate to send it here.

Thank you.

Best regards,

Kurt

[Nickolay V. Shmyrev]

> I used "Create Freeform project" option but there were many errors when I tried to build it.

Did you add sphinx4.jar to the project and into the project classpath?

> I also don't know how to use cepstra_dump.xml because when I created the project, it only allows me to use the build.xml.

Did you try just to add it to the project you created?

[Kurt Junshean Espinosa]

Hi Nickolay,

Would you be able to guide me a bit? Here is what I did in Netbeans:

1. Choose a project
   I choose Java Free-Form Project

2. Name and Location
   Location: D:\sphinx-netbeans (I created this folder)
   Build Script: D:\sphinx4\build.xml (The build.xml of the sphinx4 I downloaded)
   Project Name: SphinxNetbeans
   Project Folder: D:\sphinx-netbeans

3. Build and Run Actions (I used the default parameters)

4. Source Package Folders
   Under Source Package Folders, I added the three directories of the SRC folder in sphinx4 namely: apps, research and sphinx4.
   I did not add anything under Test Package Folders

5. Java Sources Classpath
   I added the Jar files under sphinx4/lib namely:
   js.jar
   junit-4.4.jar
   sphinx4.jar
   tags.jar
   jsapi.jar

6. Project Output (I did not add anything and I clicked 'Finish'.)

7. Then I got the following directory structure in Netbeans:

SphinxNetbeans
->sphinx4/src/apps (collapsable containing its corresponding subfolders)
->sphinx4/src/research (collapsable containing its corresponding subfolders)
->sphinx4/src/sphinx4 (collapsable containing its corresponding subfolders)
->build.xml

Since I couldn't add cepstra_dump.xml under root directory, I added it to:
->sphinx4/src/sphinx4
-->edu.cmu.sphinx.tools.feature (This package contains the FileFeatureDumper.java, frontend.config.xml, build.xml among others.)

1. When I build the project, here's what I got:

all:
create_all_model_classes:
create_my_model_classes:
Created dir: D:\sphinx4\src\sphinx4\edu\cmu\sphinx\model\acoustic\WSJ_8gau_13dCep_16k_40mel_130Hz_6800Hz
Copying 1 file to D:\sphinx4\src\sphinx4\edu\cmu\sphinx\model\acoustic\WSJ_8gau_13dCep_16k_40mel_130Hz_6800Hz
Copying 1 file to D:\sphinx4\src\sphinx4\edu\cmu\sphinx\model\acoustic\WSJ_8gau_13dCep_16k_40mel_130Hz_6800Hz
Copying 1 file to D:\sphinx4\src\sphinx4\edu\cmu\sphinx\model\acoustic\WSJ_8gau_13dCep_16k_40mel_130Hz_6800Hz
create_my_model_classes:
Created dir: D:\sphinx4\src\sphinx4\edu\cmu\sphinx\model\acoustic\WSJ_8gau_13dCep_8kHz_31mel_200Hz_3500Hz
Copying 1 file to D:\sphinx4\src\sphinx4\edu\cmu\sphinx\model\acoustic\WSJ_8gau_13dCep_8kHz_31mel_200Hz_3500Hz
Copying 1 file to D:\sphinx4\src\sphinx4\edu\cmu\sphinx\model\acoustic\WSJ_8gau_13dCep_8kHz_31mel_200Hz_3500Hz
Copying 1 file to D:\sphinx4\src\sphinx4\edu\cmu\sphinx\model\acoustic\WSJ_8gau_13dCep_8kHz_31mel_200Hz_3500Hz
create_my_model_classes:
Created dir: D:\sphinx4\src\sphinx4\edu\cmu\sphinx\model\acoustic\TIDIGITS_8gau_13dCep_16k_40mel_130Hz_6800Hz
Copying 1 file to D:\sphinx4\src\sphinx4\edu\cmu\sphinx\model\acoustic\TIDIGITS_8gau_13dCep_16k_40mel_130Hz_6800Hz
Copying 1 file to D:\sphinx4\src\sphinx4\edu\cmu\sphinx\model\acoustic\TIDIGITS_8gau_13dCep_16k_40mel_130Hz_6800Hz
Copying 1 file to D:\sphinx4\src\sphinx4\edu\cmu\sphinx\model\acoustic\TIDIGITS_8gau_13dCep_16k_40mel_130Hz_6800Hz
create_my_model_classes:
Created dir: D:\sphinx4\src\sphinx4\edu\cmu\sphinx\model\acoustic\RM1_8gau_13dCep_16k_40mel_130Hz_6800Hz
Copying 1 file to D:\sphinx4\src\sphinx4\edu\cmu\sphinx\model\acoustic\RM1_8gau_13dCep_16k_40mel_130Hz_6800Hz
Copying 1 file to D:\sphinx4\src\sphinx4\edu\cmu\sphinx\model\acoustic\RM1_8gau_13dCep_16k_40mel_130Hz_6800Hz
Copying 1 file to D:\sphinx4\src\sphinx4\edu\cmu\sphinx\model\acoustic\RM1_8gau_13dCep_16k_40mel_130Hz_6800Hz
check_jsapi_default.isPresent:
copy_jsapi_jar:
check_jsapi_jar.isPresent:
issue_jsapi_warning:
set_patternset_to_compile:
set_patternset_to_doc:
handle_jsapi:
compile_src:
Compiling 12 source files to D:\sphinx4\bld
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
Compiling 47 source files to D:\sphinx4\bld
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
create_all_models:
check_jsapi_default.isPresent:
copy_jsapi_jar:
check_jsapi_jar.isPresent:
issue_jsapi_warning:
set_patternset_to_compile:
set_patternset_to_doc:
handle_jsapi:
compile_src:
create_my_model:
Building jar: D:\sphinx4\lib\WSJ_8gau_13dCep_16k_40mel_130Hz_6800Hz.jar
check_jsapi_default.isPresent:
copy_jsapi_jar:
check_jsapi_jar.isPresent:
issue_jsapi_warning:
set_patternset_to_compile:
set_patternset_to_doc:
handle_jsapi:
compile_src:
create_my_model:
Building jar: D:\sphinx4\lib\WSJ_8gau_13dCep_8kHz_31mel_200Hz_3500Hz.jar
check_jsapi_default.isPresent:
copy_jsapi_jar:
check_jsapi_jar.isPresent:
issue_jsapi_warning:
set_patternset_to_compile:
set_patternset_to_doc:
handle_jsapi:
compile_src:
create_my_model:
Building jar: D:\sphinx4\lib\TIDIGITS_8gau_13dCep_16k_40mel_130Hz_6800Hz.jar
check_jsapi_default.isPresent:
copy_jsapi_jar:
check_jsapi_jar.isPresent:
issue_jsapi_warning:
set_patternset_to_compile:
set_patternset_to_doc:
handle_jsapi:
compile_src:
create_my_model:
Building jar: D:\sphinx4\lib\RM1_8gau_13dCep_16k_40mel_130Hz_6800Hz.jar
check_jsapi_default.isPresent:
copy_jsapi_jar:
check_jsapi_jar.isPresent:
issue_jsapi_warning:
set_patternset_to_compile:
set_patternset_to_doc:
handle_jsapi:
compile_src:
sphinx4_jar:
Building jar: D:\sphinx4\lib\sphinx4.jar
batch_jar:
Building jar: D:\sphinx4\lib\batch.jar
tags_jar:
Building jar: D:\sphinx4\lib\tags.jar
audiotool_jar:
jars:
check_jsapi_default.isPresent:
copy_jsapi_jar:
check_jsapi_jar.isPresent:
issue_jsapi_warning:
set_patternset_to_compile:
set_patternset_to_doc:
handle_jsapi:
compile_src:
demos:
make_lib_dir:
check_jsapi_jar.isPresent:
issue_jsapi_warning:
compile_sphinx_helloworld:
compile_sphinx_hellodigits:
compile_sphinx_hellongram:
compile_sphinx_lattice:
compile_sphinx_confidence:
compile_jsapi_dialog:
Created dir: D:\sphinx4\bld\edu\cmu\sphinx\demo\jsapi\dialog
Compiling 2 source files to D:\sphinx4\bld
D:\sphinx4\src\apps\edu\cmu\sphinx\demo\jsapi\dialog\Dialog.java
D:\sphinx4\src\apps\edu\cmu\sphinx\demo\jsapi\dialog\DialogManager.java
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
Copying 11 files to D:\sphinx4\bld\edu\cmu\sphinx\demo\jsapi\dialog
Copying 1 file to D:\sphinx4\bld\edu\cmu\sphinx\demo\jsapi\dialog
Building jar: D:\sphinx4\bin\Dialog.jar
compile_jsapi_jsgf:
Created dir: D:\sphinx4\bld\edu\cmu\sphinx\demo\jsapi\jsgf
Compiling 1 source file to D:\sphinx4\bld
D:\sphinx4\src\apps\edu\cmu\sphinx\demo\jsapi\jsgf\JSGFDemo.java
Copying 5 files to D:\sphinx4\bld\edu\cmu\sphinx\demo\jsapi\jsgf
Copying 1 file to D:\sphinx4\bld\edu\cmu\sphinx\demo\jsapi\jsgf
Building jar: D:\sphinx4\bin\JSGFDemo.jar
init:
keystore:
compile_zipcity:
compile_sphinx_wavfile:
compile_sphinx_transcriber:
compile_tags:
Created dir: D:\sphinx4\bld\edu\cmu\sphinx\demo\jsapi\tags
Compiling 5 source files to D:\sphinx4\bld
D:\sphinx4\src\apps\edu\cmu\sphinx\demo\jsapi\tags\Burger.java
D:\sphinx4\src\apps\edu\cmu\sphinx\demo\jsapi\tags\FeatureValueDemo.java
D:\sphinx4\src\apps\edu\cmu\sphinx\demo\jsapi\tags\LiveConnectDemo.java
D:\sphinx4\src\apps\edu\cmu\sphinx\demo\jsapi\tags\OrderItem.java
D:\sphinx4\src\apps\edu\cmu\sphinx\demo\jsapi\tags\Pizza.java
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
Copying 2 files to D:\sphinx4\bld\edu\cmu\sphinx\demo\jsapi\tags
Building jar: D:\sphinx4\bin\FeatureValueDemo.jar
Building jar: D:\sphinx4\bin\LiveConnectDemo.jar
compile_sphinx_demos:
all:
Build complete.
BUILD SUCCESSFUL (total time: 20 seconds)

It says it's successful, though, I still notice some red marks in each of the src folders.

1. Then I tried this. I right-click 'cepstra_producer' under build.xml in this folder: edu.cmu.sphinx.tools.feature( This is the same folder I added cepstra_dump.xml.) and I click Run Target. It produced many errors like this:

D:\sphinx4\src\sphinx4\edu\cmu\sphinx\tools\feature\FeatureFileDumper.java:15: package edu.cmu.sphinx.frontend does not exist
import edu.cmu.sphinx.frontend.*;
D:\sphinx4\src\sphinx4\edu\cmu\sphinx\tools\feature\FeatureFileDumper.java:16: package edu.cmu.sphinx.frontend.util does not exist
import edu.cmu.sphinx.frontend.util.StreamDataSource;
D:\sphinx4\src\sphinx4\edu\cmu\sphinx\tools\feature\FeatureFileDumper.java:17: package edu.cmu.sphinx.util.props does not exist
import edu.cmu.sphinx.util.props.ConfigurationManager;
D:\sphinx4\src\sphinx4\edu\cmu\sphinx\tools\feature\FeatureFileDumper.java:18: package edu.cmu.sphinx.util.props does not exist
........
.......

I'm sorry for this long post. Clearly there is something wrong but I do not know how to fix it. Any help pls? Thanks for your patience.

Best regards,

Kurt

[Nickolay V. Shmyrev]

I suggest you to try to understand what your are doing before doing something. here is the sequence:

1. Install Netbeans 6.5
2. Create a new Java Application Project, name it featureextractor
3. Right-Click Libraries->Add JAR/Folder
4. Add sphinx4.jar

5. Add imports in the main file:

   import edu.cmu.sphinx.frontend.Data;
   import edu.cmu.sphinx.frontend.FrontEnd;
   import edu.cmu.sphinx.frontend.util.StreamDataSource;
   import edu.cmu.sphinx.frontend.util.DataDumper;
   import edu.cmu.sphinx.util.props.ConfigurationManager;
   import java.io.FileInputStream;

6. Add the code of the function main, dont't forget to change wav file path:

   try {
   String configFile = "src/featureextractor/config.xml";
   String audioFile = "C:/test.wav"; // Put your file name here

       ConfigurationManager cm = new ConfigurationManager(configFile);
   
       FrontEnd frontend = (FrontEnd) cm.lookup ("mfcFrontEnd");
   
       StreamDataSource source = (StreamDataSource) cm.lookup ("streamDataSource");
       source.setInputStream(new FileInputStream(audioFile), audioFile);
       DataDumper dumper = (DataDumper)cm.lookup("dataDumper");
   
       Data data = null;
       do {
           data = dumper.getData();
       } while (data != null);
   
   } catch (Exception e) {
       e.printStackTrace();
   }
   

7. Create config.xml. Click featureextractor->new->other. Create empty file.

<?xml version="1.0" encoding="UTF-8"?>
<config>
<property name="logLevel" value="WARNING"/>
<component name="mfcFrontEnd" type="edu.cmu.sphinx.frontend.FrontEnd">
<propertylist name="pipeline">
<item>streamDataSource</item>
<item>preemphasizer</item>
<item>dither</item>
<item>windower</item>
<item>fft</item>
<item>melFilterBank</item>
<item>dct</item>
<item>dataDumper</item>
</propertylist>
</component>
<component name="streamDataSource"
type="edu.cmu.sphinx.frontend.util.StreamDataSource">
<property name="sampleRate" value="16000"/>
</component>
<component name="preemphasizer"
type="edu.cmu.sphinx.frontend.filter.Preemphasizer"/>
<component name="dither"
type="edu.cmu.sphinx.frontend.filter.Dither"/>
<component name="windower"
type="edu.cmu.sphinx.frontend.window.RaisedCosineWindower"/>
<component name="fft"
type="edu.cmu.sphinx.frontend.transform.DiscreteFourierTransform">
<property name="numberFftPoints" value="512"/>
</component>
<component name="melFilterBank"
type="edu.cmu.sphinx.frontend.frequencywarp.MelFrequencyFilterBank">
<property name="numberFilters" value="40"/>
<property name="minimumFrequency" value="130"/>
<property name="maximumFrequency" value="6800"/>
</component>
<component name="dct"
type="edu.cmu.sphinx.frontend.transform.DiscreteCosineTransform"/>
<component name="dataDumper" type="edu.cmu.sphinx.frontend.util.DataDumper"/>
</config>

1. Add test.wav

2. Run and enjoy.

You can download ready-to-try netbeans project here:

http://www.mediafire.com/?lw2lbzgkybf

[Kurt Junshean Espinosa]

Thank you very much for your help Nickolay! FeatureExtractor is already running. And thanks for your suggestion. I will remember that:)