Re: [Kaldi-users] reproducing numbers in wsj/s5

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

OK, everything there looks pretty normal.  Let's check the features.  Try
doing

 path.sh; copy-feats scp:data/train_si284/feats.scp ark,t:- | head
copy-feats scp:data/train_si284/feats.scp ark,t:-
011c0201  [
  46.88603 -32.66241 -11.9018 -14.99819 -4.834829 -2.596024 -5.20842
2.553219 3.935668 10.76861 9.013732 4.780495 -3.267961
  46.34384 -33.52037 -10.28797 -11.47937 -5.184441 -6.867837 -12.17628
-6.93225 -7.423645 -1.106866 3.337547 -0.5728683 0.6970833
  46.81014 -31.05773 -7.440367 -11.46406 -5.926404 -2.568073 -3.351702
-5.60004 -8.281028 -9.460709 6.799894 6.513657 -7.970572
  46.8651 -30.11357 -5.052508 -11.06986 -6.155907 -2.664125 -10.907
-8.487434 -3.194182 -6.367419 6.390572 0.01187897 -0.1876552
  46.92208 -33.49314 -9.207782 -12.05287 -5.4948 -4.269254 -9.913167
-1.722165 -0.7680123 -2.616493 9.212611 7.324253 12.41928
  46.8317 -32.19164 -8.810924 -12.31202 -3.736232 -1.335111 -8.361964
-2.103669 -3.199154 -6.140819 -8.036694 -8.49004 2.942402
  47.34614 -33.04692 -7.571927 -8.986545 -3.523783 1.130052 -3.151051
3.572651 3.381188 -3.117314 -2.670754 2.165966 0.3512328
  46.63126 -33.57331 -11.13035 -7.506401 2.113541 3.268804 -1.814186
2.770107 -4.381127 -3.264618 -3.549418 2.463501 -0.7675107
  46.31366 -33.00954 -6.964884 -6.109342 -7.121596 -4.326709 -10.80855
-6.96948 6.597637 4.109591 1.001096 2.592361 6.154356

and show me the otuput
Dan

On Mon, Jun 3, 2013 at 5:59 PM, Mailing list used for User Communication
and Updates <kal...@li...> wrote:

> Here is the result for
>
>     ls -l data/lang  data/lang/phones exp/mono0a exp/mono0a/log
> exp/mono0a/decode/log
>
> Hao
>
> On Mon, Jun 3, 2013 at 4:49 PM, Mailing list used for User
> Communication and Updates <kal...@li...> wrote:
> > My corresponding numbers are -108.535 ->  -95.116, and the difference is
> > about the same.  [It's strange that the absolute numbers differ slightly,
> > though.]
> > Could you do
> > ls -l data/lang  data/lang/phones exp/mono0a exp/mono0a/log
> > exp/mono0a/decode/log
> > and send the output?
> >
> > Dan
> >
> >
> >
> > On Mon, Jun 3, 2013 at 5:42 PM, Mailing list used for User Communication
> and
> > Updates <kal...@li...> wrote:
> >>
> >> Copied from exp/mono0a/log/update.39.log
> >>
> >>     LOG (gmm-sum-accs:main():gmm-sum-accs.cc:60) Summed 2 stats, total
> >> count 849568, avg like/frame -95.5657
> >>
> >> Hao
> >>
> >> On Mon, Jun 3, 2013 at 4:29 PM, Mailing list used for User
> >> Communication and Updates <kal...@li...> wrote:
> >> > OK, that is quite weird.  What is the avg like/frame on the final
> >> > update.??.log?
> >> > Dan
> >> >
> >> >
> >> >
> >> > On Mon, Jun 3, 2013 at 5:00 PM, Mailing list used for User
> Communication
> >> > and
> >> > Updates <kal...@li...> wrote:
> >> >>
> >> >> Unfortunately those are absolute differences. Here is what I get for
> >> >> the
> >> >> mono0a.
> >> >>
> >> >>     %WER 40.44 [ 2282 / 5643, 118 ins, 537 del, 1627 sub ]
> >> >> exp/mono0a/decode_tgpr_eval92/wer_9
> >> >>
> >> >> And below is the reported one in RESULTS.
> >> >>
> >> >>     exp/mono0a/decode_tgpr_eval92/wer_10:%WER 25.54 [ 1441 / 5643,
> 149
> >> >> ins, 191 del, 1101 sub ]
> >> >>
> >> >> I will try to checkout the same version as the RESULTS file and run
> >> >> the experiments again.
> >> >>
> >> >> Here is the total data count from exp/mono0a/log/update.0.log.
> >> >>
> >> >>     LOG (gmm-sum-accs:main():gmm-sum-accs.cc:60) Summed 2 stats,
> total
> >> >> count 849568, avg like/frame -109.265
> >> >>
> >> >> Thanks for your help.
> >> >>
> >> >> Hao
> >> >>
> >> >> On Mon, Jun 3, 2013 at 3:43 PM, Mailing list used for User
> >> >> Communication and Updates <kal...@li...> wrote:
> >> >> > Assuming those are relative numbers, they're probably within the
> >> >> > margin
> >> >> > of
> >> >> > error.  See if it's the same on other test sets.  Because various
> >> >> > algorithms
> >> >> > call rand(), and different machines implement this differently,
> >> >> > results
> >> >> > aren't fully reproducible.
> >> >> >
> >> >> > But to check that it's not some code change that hurts the results,
> >> >> > you
> >> >> > could try checking out a copy with the same revision number as the
> >> >> > RESULTS
> >> >> > file, and running again and seeing what results you get.  If there
> is
> >> >> > a
> >> >> > difference, I'd like to know.
> >> >> >
> >> >> > It's also possible some files were missing in your WSJ
> distribution--
> >> >> > let me
> >> >> > know the data count reported in one the */log/update.log files and
> >> >> > I'll
> >> >> > compare with a local copy.
> >> >> >
> >> >> > Dan
> >> >> >
> >> >> >
> >> >> >
> >> >> > On Mon, Jun 3, 2013 at 4:34 PM, Mailing list used for User
> >> >> > Communication
> >> >> > and
> >> >> > Updates <kal...@li...> wrote:
> >> >> >>
> >> >> >> Hi,
> >> >> >>
> >> >> >> I'm following the WSJ s5 recipe, and I wasn't able to reproduce
> the
> >> >> >> results stated in the RESULTS file. Are we supposed to get exactly
> >> >> >> the
> >> >> >> same numbers with the (hyper-)parameters given in the recipe?
> >> >> >>
> >> >> >> Just to give a sense of how far I'm off. My monophone models
> perform
> >> >> >> 10%
> >> >> >> worse than the reported results on eval92 and the resulting
> triphone
> >> >> >> models perform 4% worse.
> >> >> >>
> >> >> >> Thanks,
> >> >> >> Hao
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >>
> ------------------------------------------------------------------------------
> >> >> >> How ServiceNow helps IT people transform IT departments:
> >> >> >> 1. A cloud service to automate IT design, transition and
> operations
> >> >> >> 2. Dashboards that offer high-level views of enterprise services
> >> >> >> 3. A single system of record for all IT processes
> >> >> >> http://p.sf.net/sfu/servicenow-d2d-j
> >> >> >> _______________________________________________
> >> >> >> Kaldi-users mailing list
> >> >> >> Kal...@li...
> >> >> >> https://lists.sourceforge.net/lists/listinfo/kaldi-users
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> ------------------------------------------------------------------------------
> >> >> > How ServiceNow helps IT people transform IT departments:
> >> >> > 1. A cloud service to automate IT design, transition and operations
> >> >> > 2. Dashboards that offer high-level views of enterprise services
> >> >> > 3. A single system of record for all IT processes
> >> >> > http://p.sf.net/sfu/servicenow-d2d-j
> >> >> > _______________________________________________
> >> >> > Kaldi-users mailing list
> >> >> > Kal...@li...
> >> >> > https://lists.sourceforge.net/lists/listinfo/kaldi-users
> >> >> >
> >> >>
> >> >>
> >> >>
> >> >>
> ------------------------------------------------------------------------------
> >> >> How ServiceNow helps IT people transform IT departments:
> >> >> 1. A cloud service to automate IT design, transition and operations
> >> >> 2. Dashboards that offer high-level views of enterprise services
> >> >> 3. A single system of record for all IT processes
> >> >> http://p.sf.net/sfu/servicenow-d2d-j
> >> >> _______________________________________________
> >> >> Kaldi-users mailing list
> >> >> Kal...@li...
> >> >> https://lists.sourceforge.net/lists/listinfo/kaldi-users
> >> >
> >> >
> >> >
> >> >
> >> >
> ------------------------------------------------------------------------------
> >> > How ServiceNow helps IT people transform IT departments:
> >> > 1. A cloud service to automate IT design, transition and operations
> >> > 2. Dashboards that offer high-level views of enterprise services
> >> > 3. A single system of record for all IT processes
> >> > http://p.sf.net/sfu/servicenow-d2d-j
> >> > _______________________________________________
> >> > Kaldi-users mailing list
> >> > Kal...@li...
> >> > https://lists.sourceforge.net/lists/listinfo/kaldi-users
> >> >
> >>
> >>
> >>
> ------------------------------------------------------------------------------
> >> How ServiceNow helps IT people transform IT departments:
> >> 1. A cloud service to automate IT design, transition and operations
> >> 2. Dashboards that offer high-level views of enterprise services
> >> 3. A single system of record for all IT processes
> >> http://p.sf.net/sfu/servicenow-d2d-j
> >> _______________________________________________
> >> Kaldi-users mailing list
> >> Kal...@li...
> >> https://lists.sourceforge.net/lists/listinfo/kaldi-users
> >
> >
> >
> >
> ------------------------------------------------------------------------------
> > How ServiceNow helps IT people transform IT departments:
> > 1. A cloud service to automate IT design, transition and operations
> > 2. Dashboards that offer high-level views of enterprise services
> > 3. A single system of record for all IT processes
> > http://p.sf.net/sfu/servicenow-d2d-j
> > _______________________________________________
> > Kaldi-users mailing list
> > Kal...@li...
> > https://lists.sourceforge.net/lists/listinfo/kaldi-users
> >
>
>
> ------------------------------------------------------------------------------
> How ServiceNow helps IT people transform IT departments:
> 1. A cloud service to automate IT design, transition and operations
> 2. Dashboards that offer high-level views of enterprise services
> 3. A single system of record for all IT processes
> http://p.sf.net/sfu/servicenow-d2d-j
> _______________________________________________
> Kaldi-users mailing list
> Kal...@li...
> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>
>