Here are some comments on the tinydoc as revised and posted to this forum. Some of the things worked slightly differently for me. Just thought I'd 'share my experiences
** SPHINXTRAINDIR environment variable and 'time' directory:
> Got to a new directory where you have the space
> mkdir time
> cd time
>
> Set the SPHINXTRAINDIR environment variable to point to this directory.
This makes it seem like SPHINXTRAINDIR should point to the 'time' directory, but when you use it ...
> Set up the directory structure, scripts and config files
> $SPHINXTRAINDIR/scripts_pl/setup_SphinxTrain time
> The main config file is put into etc/sphinx_train.cfg
... it's pointing to the directory where SphinxTrain is installed - not necessarily the same. My setup is like this:
PATH/sphinxTrainStuff/ - contains docs, odds and ends
PATH/sphinxTrainStuff/SphinxTrain/ - the downloaded, unpacked tarball
PATH/sphinxTrainStuff/time/ - as above (and below)
SPHINXTRAINDIR is set to PATH/sphinxTrainStuff/SphinxTrain/
** make_dict post-processing:
> Create a dictionary with (only works for US English)
>
> bin/make_dict etc/time.transcription
>
> Will create etc/word.known etc/word.unknown files ...
I found that neither of these were sorted and etc/word.unknown hadn't been uniq'd, so I did:
> Make the melcep feature files
>
> bin/make_feats etc/time.fileids
>
> I found that I had to change the bin/make_feats script, line 72, from:
>
> bin/wave2feat -verbose -c $1 -nist -di wav -ei wav -do feat -eo feat
>
> to:
>
> bin/wave2feat -verbose -c $1 -raw -di wav -ei wav -do feat -eo feat
make_feats seems to be a simple wrapper for wave2feat. Running bin/wave2feat with no arguments gets a help message explaining all these command-line options. The options you need maybe slightly different. E.g. I used:
as my audio files were in nist sphere format and had the extension '.sph'
** $CFG_N_TIED_STATES:
> I had to edit my generated config file (etc/sphinx_train.cfg):
>
> #$CFG_N_TIED_STATES = 6000;
> $CFG_N_TIED_STATES = "untied";
>
> Without these changes, I was unable to make SphinxTrain run.
I wrote a couple of extra scripts based on 03.makeuntiedmdef/make_untied_mdef.pl, and was able to have $CFG_N_TIED_STATES as I liked:
*** 03.make_mdef/make_mdef.pl
This is basically just make_untied_mdef.pl with 'untied' replaced with ${CFG_N_TIED_STATES} and a change of output format:
77c77
< $logdir = "$CFG_LOG_DIR/03.makeuntiedmdef";
---
> $logdir = "$CFG_LOG_DIR/03.make_mdef";
82c82
< &ST_Log ("MODULE: 03 Make Untied mdef\n");
---
> &ST_Log ("MODULE: 03 Make mdef\n");
86,90c86,87
< $untiedmdef = "${CFG_BASE_DIR}/model_architecture/${CFG_EXPTNAME}.untied.mdef";
<
< ## awb: replace with
< ## mk_mdef_gen -phnlstfn $phonelist TRANSCRIPTFILE DICTIONARY -ountiedmdef .. n_sta
tes ..
< # -minocc 1 Min occurances of a triphone must occur for inclusion in md
ef file
---
> $output_format_arg = "-oalltphnmdef";
> $out_mdef = "${CFG_BASE_DIR}/model_architecture/${CFG_EXPTNAME}.alltriphones.mdef";
93c90
< system ("$MAKE_MDEF -phnlstfn $CFG_RAWPHONEFILE -dictfn $CFG_DICTIONARY -fdictfn $CFG
_FILLERDICT -lsnfn $CFG_TRANSCRIPTFILE -ountiedmdef $untiedmdef -n_state_pm $CFG_STAT
ESPERHMM -maxtriphones 10000 2>$logfile");
---
> system ("$MAKE_MDEF -phnlstfn $CFG_RAWPHONEFILE -dictfn $CFG_DICTIONARY -fdictfn $CFG
_FILLERDICT -lsnfn $CFG_TRANSCRIPTFILE $output_format_arg $out_mdef -n_state_pm $CFG_
STATESPERHMM -maxtriphones 10000 2>$logfile");
ivan@debian:~/jura/speech/res/cmu/strain/time/scripts_pl$
** the basic perl scripts:
> Now we can start on the basic perl scripts, ...
SPHINXTRAINDIR/scripts_pl/RunAll.pl (not in time/scripts_pl/) runs all these scripts in order, adding the config file as an argument. Note some of the scripts also take a second argument '1', i.e.:
Here are some comments on the tinydoc as revised and posted to this forum. Some of the things worked slightly differently for me. Just thought I'd 'share my experiences
** SPHINXTRAINDIR environment variable and 'time' directory:
> Got to a new directory where you have the space
> mkdir time
> cd time
>
> Set the SPHINXTRAINDIR environment variable to point to this directory.
This makes it seem like SPHINXTRAINDIR should point to the 'time' directory, but when you use it ...
> Set up the directory structure, scripts and config files
> $SPHINXTRAINDIR/scripts_pl/setup_SphinxTrain time
> The main config file is put into etc/sphinx_train.cfg
... it's pointing to the directory where SphinxTrain is installed - not necessarily the same. My setup is like this:
PATH/sphinxTrainStuff/ - contains docs, odds and ends
PATH/sphinxTrainStuff/SphinxTrain/ - the downloaded, unpacked tarball
PATH/sphinxTrainStuff/time/ - as above (and below)
SPHINXTRAINDIR is set to PATH/sphinxTrainStuff/SphinxTrain/
** make_dict post-processing:
> Create a dictionary with (only works for US English)
>
> bin/make_dict etc/time.transcription
>
> Will create etc/word.known etc/word.unknown files ...
I found that neither of these were sorted and etc/word.unknown hadn't been uniq'd, so I did:
cat word.known word.unknown | sort | uniq > time.dic
** make_feats and wave2feat:
> Make the melcep feature files
>
> bin/make_feats etc/time.fileids
>
> I found that I had to change the bin/make_feats script, line 72, from:
>
> bin/wave2feat -verbose -c $1 -nist -di wav -ei wav -do feat -eo feat
>
> to:
>
> bin/wave2feat -verbose -c $1 -raw -di wav -ei wav -do feat -eo feat
make_feats seems to be a simple wrapper for wave2feat. Running bin/wave2feat with no arguments gets a help message explaining all these command-line options. The options you need maybe slightly different. E.g. I used:
wave2feat -verbose -c $1 -nist -di wav -ei sph -do feat -eo feat
as my audio files were in nist sphere format and had the extension '.sph'
** $CFG_N_TIED_STATES:
> I had to edit my generated config file (etc/sphinx_train.cfg):
>
> #$CFG_N_TIED_STATES = 6000;
> $CFG_N_TIED_STATES = "untied";
>
> Without these changes, I was unable to make SphinxTrain run.
I wrote a couple of extra scripts based on 03.makeuntiedmdef/make_untied_mdef.pl, and was able to have $CFG_N_TIED_STATES as I liked:
*** 03.make_mdef/make_mdef.pl
This is basically just make_untied_mdef.pl with 'untied' replaced with ${CFG_N_TIED_STATES} and a change of output format:
diff 03.makeuntiedmdef/make_untied_mdef.pl 03.make_mdef/make_mdef.pl
77c77
< $logdir = "$CFG_LOG_DIR/03.makeuntiedmdef";
---
> $logdir = "$CFG_LOG_DIR/03.make_mdef";
82c82
< &ST_Log ("MODULE: 03 Make Untied mdef\n");
---
> &ST_Log ("MODULE: 03 Make mdef\n");
86c86,87
< $untiedmdef = "${CFG_BASE_DIR}/model_architecture/${CFG_EXPTNAME}.untied.mdef"
;
---
> $output_format_arg = "-ocimdef";
> $out_mdef = "${CFG_BASE_DIR}/model_architecture/${CFG_EXPTNAME}.${CFG_N_TIED_S
TATES}.mdef";
93c94
< system ("$MAKE_MDEF -phnlstfn $CFG_RAWPHONEFILE -dictfn $CFG_DICTIONARY -fdict
fn $CFG_FILLERDICT -lsnfn $CFG_TRANSCRIPTFILE -ountiedmdef $untiedmdef -n_state
_pm $CFG_STATESPERHMM -maxtriphones 10000 2>$logfile");
---
> system ("$MAKE_MDEF -phnlstfn $CFG_RAWPHONEFILE -dictfn $CFG_DICTIONARY -fdict
fn $CFG_FILLERDICT -lsnfn $CFG_TRANSCRIPTFILE $output_format_arg $out_mdef -n_s
tate_pm $CFG_STATESPERHMM -maxtriphones 10000 2>$logfile");
*** 03.make_mdef/make_alltri_mdef.pl
I added this as because a later script (06.prunetree/prunetree.pl) expected an alltriphones.mdef file that no-one else seemed to be making.
diff 03.makeuntiedmdef/make_untied_mdef.pl 03.make_mdef/make_alltri_mdef.pl
77c77
< $logdir = "$CFG_LOG_DIR/03.makeuntiedmdef";
---
> $logdir = "$CFG_LOG_DIR/03.make_mdef";
82c82
< &ST_Log ("MODULE: 03 Make Untied mdef\n");
---
> &ST_Log ("MODULE: 03 Make mdef\n");
86,90c86,87
< $untiedmdef = "${CFG_BASE_DIR}/model_architecture/${CFG_EXPTNAME}.untied.mdef";
<
< ## awb: replace with
< ## mk_mdef_gen -phnlstfn $phonelist TRANSCRIPTFILE DICTIONARY -ountiedmdef .. n_sta
tes ..
< # -minocc 1 Min occurances of a triphone must occur for inclusion in md
ef file
---
> $output_format_arg = "-oalltphnmdef";
> $out_mdef = "${CFG_BASE_DIR}/model_architecture/${CFG_EXPTNAME}.alltriphones.mdef";
93c90
< system ("$MAKE_MDEF -phnlstfn $CFG_RAWPHONEFILE -dictfn $CFG_DICTIONARY -fdictfn $CFG
_FILLERDICT -lsnfn $CFG_TRANSCRIPTFILE -ountiedmdef $untiedmdef -n_state_pm $CFG_STAT
ESPERHMM -maxtriphones 10000 2>$logfile");
---
> system ("$MAKE_MDEF -phnlstfn $CFG_RAWPHONEFILE -dictfn $CFG_DICTIONARY -fdictfn $CFG
_FILLERDICT -lsnfn $CFG_TRANSCRIPTFILE $output_format_arg $out_mdef -n_state_pm $CFG_
STATESPERHMM -maxtriphones 10000 2>$logfile");
ivan@debian:~/jura/speech/res/cmu/strain/time/scripts_pl$
** the basic perl scripts:
> Now we can start on the basic perl scripts, ...
SPHINXTRAINDIR/scripts_pl/RunAll.pl (not in time/scripts_pl/) runs all these scripts in order, adding the config file as an argument. Note some of the scripts also take a second argument '1', i.e.:
"$CFG_SCRIPT_DIR/02.ci_schmm/slave_convg.pl -cfg $cfg_file 1",
"$CFG_SCRIPT_DIR/04.cd_schmm_untied/slave_convg.pl -cfg $cfg_file 1",
"$CFG_SCRIPT_DIR/07.cd-schmm/slave_convg.pl -cfg $cfg_file 1",
Looking at the scripts this argument seems to be catered for (i.e. it's optional unless you want it > 1: it's the number of iterations).
Note also that scripts 04-08 are commented out in RunAll and should be uncommented - all (especially 08) need to be run for 09 to work.
** > Sphinx2 output will be in ./model_parameters/time.s2models/:
See separate note on using Sphinx2 on the output.
Hope some of this is of use.
Ivan