Menu
▾
▴
Re: [Kaldi-developers] Problems Running Kaldi Tests
|
From: Joaquin A. R. <ja...@co...> - 2014-12-30 04:29:03
|
There is a "data/lang" folder, and there is a "data/lang_test" folder. The
"data/lang" folder does have the "phones" subfolder, but the
"data/lang_test" folder only has the "tmp" folder inside it. All the files
that were expected to be inside "phones" are directly inside
"data/lang_test".
On Mon, Dec 29, 2014 at 11:22 PM, Daniel Povey <dp...@gm...> wrote:
> That file should be in the phones/ subdirectory. let me look into it.
> Dan
>
>
> On Mon, Dec 29, 2014 at 8:21 PM, Joaquin Antonio Ruales
> <ja...@co...> wrote:
> > Hi Daniel,
> >
> > All tests passed as expected.
> > I am now trying to run the Voxforge recipe, and after train_mono.sh
> > succeeds, I get the error:
> >
> > mkgraph.sh: expected data/lang_test/phones/silence.csl to exist
> >
> > I have checked the directories and that file is in fact missing.
> However, I
> > have found a "silence.csl" file as well as other expected files inside
> > data/lang_test/silence.csl (one level above from the expected
> directory). I
> > have thus modified mkgraph.sh to reference the expected file with the
> line:
> >
> > required="$lang/L.fst $lang/G.fst $lang/phones.txt $lang/words.txt
> > $lang/silence.csl $lang/disambig.int $model $tree"
> >
> > which now references the file one directory higher. Is this the correct
> > thing to do? or am I referencing the wrong file?
> >
> > After I made this change I reran ./run.sh, and the recipe didn't throw
> the
> > error mentioned above anymore. However, I obtain more errors later on:
> >
> > fstcomposecontext --context-size=1 --central-position=0
> > --read-disambig-syms=data/lang_test/phones/disambig.int
> > --write-disambig-syms=data/lang_test/tmp/disambig_ilabels_1_0.int
> > data/lang_test/tmp/ilabels_1_0
> >
> > ERROR (fstcomposecontext:main():fstcomposecontext.cc:126)
> fstcomposecontext:
> > Could not read disambiguation symbols from
> > data/lang_test/phones/disambig.int
> >
> > ERROR (fstcomposecontext:main():fstcomposecontext.cc:126)
> fstcomposecontext:
> > Could not read disambiguation symbols from
> > data/lang_test/phones/disambig.int
> >
> > Again, the file data/lang_test/phones/disambig.int does not exist, but
> the
> > file data/lang_test/disambig.int (one folder above) does exist. Is it
> > correct to change the referenced disambig.int to be the one on the
> parent
> > folder?
> > I want to make sure that I am not making changes that don't make sense
> with
> > mkgraph.sh and disambig.int as mentioned above.
> >
> > Thanks,
> > Joaquín
> >
> > On Thu, Dec 4, 2014 at 3:05 PM, Daniel Povey <dp...@gm...> wrote:
> >>
> >> I have modified that test program and it should pass now.
> >>
> >> We normally don't get so many issues discovered by one person- it only
> >> happens when we encounter a new platform that hasn't been properly
> >> tested before.
> >>
> >> Dan
> >>
> >>
> >> On Thu, Dec 4, 2014 at 4:34 AM, Joaquin Antonio Ruales
> >> <ja...@co...> wrote:
> >> > Yep! now the matrix test works. Now I'm getting an error on the
> >> > mle-full-gmm-test. could it be another problem only relevant to OSX
> >> > Mavericks and Yosemite? Here are the last few lines of output of
> >> > ./mle-full-gmm-test:
> >> >
> >> > Condition number of random matrix large 1.558446e+02, trying again
> (this
> >> > is
> >> > normal)
> >> >
> >> > Avg log-like per frame [full-cov, 1-mix] should be: -2.535968e+01
> >> >
> >> > Total log-like [full-cov, 1-mix] should be: -3.550355e+04
> >> >
> >> > Cov eigs are [ 1.639155e+01 1.397001e+01 1.149592e+01 1.090052e+01
> >> > 1.049452e+01 8.523492e+00 7.486929e+00 7.382205e+00 6.812490e+00
> >> > 5.214448e+00 ]
> >> >
> >> > LOG (UnitTestEstimateFullGmm():mle-full-gmm-test.cc:361) Testing
> >> > natural<>normal conversion
> >> >
> >> > KALDI_ASSERT: at UnitTestEstimateFullGmm:mle-full-gmm-test.cc:374,
> >> > failed:
> >> > std::abs(gmm->means_invcovars().Row(0)(d) -
> >> > rgmm.means_invcovars().Row(0)(d)) < prec_v
> >> >
> >> > Stack trace is:
> >> >
> >> > 0 mle-full-gmm-test 0x000000010b873bb7
> >> > _ZN5kaldi18KaldiGetStackTraceEv + 71
> >> >
> >> > 1 mle-full-gmm-test 0x000000010b873f86
> >> > _ZN5kaldi19KaldiAssertFailure_EPKcS1_iS1_ + 230
> >> >
> >> > 2 mle-full-gmm-test 0x000000010b78f8c6
> >> > _Z23UnitTestEstimateFullGmmv + 4774
> >> >
> >> > 3 mle-full-gmm-test 0x000000010b790a48 main + 40
> >> >
> >> > 4 libdyld.dylib 0x000000010bb725fd start + 1
> >> >
> >> > 5 ??? 0x0000000000000001 0x0 + 1
> >> >
> >> > Abort trap: 6
> >> >
> >> >
> >> > On Wed, Dec 3, 2014 at 8:10 PM, Dogan Can <dog...@us...> wrote:
> >> >>
> >> >> Hi Dan,
> >> >>
> >> >> It turns out this was due to a bug in our RandGauss2 function that
> >> >> accepts
> >> >> double arguments. I’m committing the following fix to kaldi trunk
> right
> >> >> away.
> >> >>
> >> >> Cheers,
> >> >> Dogan
> >> >>
> >> >> diff --git a/src/base/kaldi-math.cc b/src/base/kaldi-math.cc
> >> >> index 5b30320..0de73bf 100644
> >> >> --- a/src/base/kaldi-math.cc
> >> >> +++ b/src/base/kaldi-math.cc
> >> >> @@ -147,7 +147,7 @@ void RandGauss2(double *a, double *b, RandomState
> >> >> *state)
> >> >> float a_float, b_float;
> >> >> // Just because we're using doubles doesn't mean we need
> >> >> super-high-quality
> >> >> // random numbers, so we just use the floating-point version
> >> >> internally.
> >> >> - RandGauss2(&a_float, &b_float);
> >> >> + RandGauss2(&a_float, &b_float, state);
> >> >> *a = a_float; *b = b_float;
> >> >> }
> >> >>
> >> >> On Dec 3, 2014, at 2:23 AM, Dogan Can <dog...@us...> wrote:
> >> >>
> >> >> Hi Dan
> >> >>
> >> >> I don’t have a Mavericks setup but I can replicate the failure on
> >> >> Yosemite
> >> >> with clang. When everything is compiled with gcc, matrix-lib-test
> >> >> finishes
> >> >> successfully.
> >> >>
> >> >> I looked a bit into the root cause of why the test was going into an
> >> >> infinite loop. The random 3x3 matrices generated during the failing
> >> >> test end
> >> >> up having the same first and third columns and fail the condition
> >> >> number
> >> >> test, hence the generation loop never terminates. I did some
> sleuthing
> >> >> to
> >> >> figure out how that was happening and it turns out the value of
> >> >> rstate.seed
> >> >> (see the snippet below from matrix/kaldi-matrix.cc) is the same
> before
> >> >> and
> >> >> after the call to RandGauss2 (note that inner for loop executes only
> >> >> once).
> >> >> It seems like we are hitting a race condition in the system provided
> >> >> rand_r
> >> >> implementation or a compiler bug.
> >> >>
> >> >>
> >> >> 1093 template<typename Real>
> >> >> 1094 void MatrixBase<Real>::SetRandn() {
> >> >> 1095 kaldi::RandomState rstate;
> >> >> 1096 for (MatrixIndexT row = 0; row < num_rows_; row++) {
> >> >> 1097 Real *row_data = this->RowData(row);
> >> >> 1098 MatrixIndexT nc = (num_cols_ % 2 == 1) ? num_cols_ - 1 :
> >> >> num_cols_;
> >> >> 1099 for (MatrixIndexT col = 0; col < nc; col += 2) {
> >> >> 1100 kaldi::RandGauss2(row_data + col, row_data + col + 1,
> >> >> &rstate);
> >> >> 1101 }
> >> >> 1102 if (nc != num_cols_) row_data[nc] =
> >> >> static_cast<Real>(kaldi::RandGauss(&rstate));
> >> >> 1103 }
> >> >> 1104 }
> >> >>
> >> >>
> >> >> Cheers,
> >> >> Dogan
> >> >>
> >> >> On Dec 2, 2014, at 10:13 PM, Daniel Povey <dp...@gm...> wrote:
> >> >>
> >> >> According to
> >> >>
> >> >>
> http://stackoverflow.com/questions/19554439/gdb-missing-in-os-x-mavericks
> >> >> gdb is no longer supported in mavericks and you have to use lldb.
> I've
> >> >> never used that so I don't know what to tell you to do.
> >> >> Dogan, do you have a mavericks setup, and can you see if you get the
> >> >> infinite loop that he gets when you test?
> >> >> Joaqin: I would just continue through the tutorial for now. Probably
> >> >> it's not going to be a problem.
> >> >> Dan
> >> >>
> >> >>
> >> >> On Wed, Dec 3, 2014 at 1:09 AM, Daniel Povey <dp...@gm...>
> wrote:
> >> >>
> >> >> Hm. That makes me think there may be a problem with your gcc/gdb
> >> >> installation- maybe you could try uninstalling it and installing it
> >> >> again. Did you get it from MacPorts?
> >> >> Dan
> >> >>
> >> >>
> >> >> On Wed, Dec 3, 2014 at 1:05 AM, Joaquin Antonio Ruales
> >> >> <ja...@co...> wrote:
> >> >>
> >> >> Now I get:
> >> >>
> >> >>
> >> >> Program received signal SIGINT, Interrupt.
> >> >>
> >> >> 0x00007fff8c755e9a in ?? ()
> >> >>
> >> >> (gdb) b matrix-lib-test.cc:39
> >> >>
> >> >> Cannot access memory at address 0xab670
> >> >>
> >> >>
> >> >> On Wed, Dec 3, 2014 at 12:57 AM, Daniel Povey <dp...@gm...>
> wrote:
> >> >>
> >> >>
> >> >> Probably when you did ctrl-c it was in the Atlas library, sometimes
> it
> >> >> can't get a proper backtrace. Instead, after doing ctrl-c do:
> >> >> (gdb) b matrix-lib-test.cc:39
> >> >> (gdb) c
> >> >>
> >> >> to continue, and when it breaks, show me a backtrace.
> >> >> Dan
> >> >>
> >> >>
> >> >> On Wed, Dec 3, 2014 at 12:48 AM, Joaquin Antonio Ruales
> >> >> <ja...@co...> wrote:
> >> >>
> >> >> This is what I get:
> >> >>
> >> >>
> >> >> Program received signal SIGINT, Interrupt.
> >> >>
> >> >> 0x00007fff8b2c6342 in ?? ()
> >> >>
> >> >> (gdb) bt
> >> >>
> >> >> #0 0x00007fff8b2c6342 in ?? ()
> >> >>
> >> >> #1 0x00007fff5fbfdbc8 in ?? ()
> >> >>
> >> >> #2 0x0000000000000001 in ?? ()
> >> >>
> >> >> #3 0x00007fff5fbfdfdc in ?? ()
> >> >>
> >> >> #4 0x00007fff5fbfdfe0 in ?? ()
> >> >>
> >> >> #5 0x00007fff5fbfdbbe in ?? ()
> >> >>
> >> >> #6 0x0000000100102672 in ?? ()
> >> >>
> >> >> #7 0x00007fff5fbfdc10 in ?? ()
> >> >>
> >> >> #8 0x00007fff8b00eac4 in ?? ()
> >> >>
> >> >> #9 0x00007fff5fbfda90 in ?? ()
> >> >>
> >> >> #10 0x00007fff8e2bf272 in ?? ()
> >> >>
> >> >> #11 0x00000002001a0a00 in ?? ()
> >> >>
> >> >> #12 0x00000001001a0a00 in ?? ()
> >> >>
> >> >> #13 0x000000010019d000 in ?? ()
> >> >>
> >> >> #14 0x00000001001a0a00 in ?? ()
> >> >>
> >> >> #15 0x00000001003002ca in ?? ()
> >> >>
> >> >> #16 0x0000000000000000 in ?? ()
> >> >>
> >> >>
> >> >> On Wed, Dec 3, 2014 at 12:42 AM, Daniel Povey <dp...@gm...>
> wrote:
> >> >>
> >> >>
> >> >> That isn't normal and it doesn't happen when I compile on a Mac.
> >> >> Can you run it in gdb by doing
> >> >> gdb ./matrix-lib-test
> >> >> (gdb) r
> >> >>
> >> >> and then do ctrl-c when it gets in the loop and type "bt" and show me
> >> >> the backtrace?
> >> >> Dan
> >> >>
> >> >>
> >> >> On Wed, Dec 3, 2014 at 12:36 AM, Joaquin Antonio Ruales
> >> >> <ja...@co...> wrote:
> >> >>
> >> >> Hi Dan,
> >> >>
> >> >> Thanks for the quick reply. I have "svn up"ed and recompiled
> >> >> everything,
> >> >> but
> >> >> now the same matrix test runs into an infinite loop. Would it be safe
> >> >> to
> >> >> ignore the test results and continue with the tutorial? Here are the
> >> >> few
> >> >> lines of output from the test:
> >> >>
> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
> >> >> random
> >> >> matrix large 7.50727e+16, trying again (this is normal)
> >> >>
> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
> >> >> random
> >> >> matrix large 1.91265e+16, trying again (this is normal)
> >> >>
> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
> >> >> random
> >> >> matrix large 7.70738e+16, trying again (this is normal)
> >> >>
> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
> >> >> random
> >> >> matrix large 2.32488e+17, trying again (this is normal)
> >> >>
> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
> >> >> random
> >> >> matrix large 8.44565e+17, trying again (this is normal)
> >> >>
> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
> >> >> random
> >> >> matrix large 1.83557e+17, trying again (this is normal)
> >> >>
> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
> >> >> random
> >> >> matrix large 2.61736e+17, trying again (this is normal)
> >> >>
> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
> >> >> random
> >> >> matrix large 8.67803e+16, trying again (this is normal)
> >> >>
> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
> >> >> random
> >> >> matrix large 1.80053e+16, trying again (this is normal)
> >> >>
> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
> >> >> random
> >> >> matrix large 1.67101e+16, trying again (this is normal)
> >> >>
> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
> >> >> random
> >> >> matrix large 1.49485e+16, trying again (this is normal)
> >> >>
> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
> >> >> random
> >> >> matrix large 8.31813e+16, trying again (this is normal)
> >> >>
> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
> >> >> random
> >> >> matrix large 1.6285e+16, trying again (this is normal)
> >> >>
> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
> >> >> random
> >> >> matrix large 2.48172e+16, trying again (this is normal)
> >> >>
> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
> >> >> random
> >> >> matrix large 2.1569e+16, trying again (this is normal)
> >> >>
> >> >>
> >> >> On Tue, Dec 2, 2014 at 4:35 PM, Daniel Povey <dp...@gm...>
> >> >> wrote:
> >> >>
> >> >>
> >> >> I think I had noticed this before, and I fixed the test by changing
> >> >> the threshold.
> >> >> If you do "svn up" and recompile, it should pass.
> >> >> Dan
> >> >>
> >> >>
> >> >> On Tue, Dec 2, 2014 at 4:20 PM, Joaquín Ruales
> >> >> <ja...@co...>
> >> >> wrote:
> >> >>
> >> >> Hi Kaldi Team,
> >> >>
> >> >> I'm running into trouble when running the Kaldi tests (make test)
> >> >> described
> >> >> in the Kaldi tutorial. I'm running it on a Mac (Mavericks) and the
> >> >> error
> >> >> is
> >> >> in matrix-lib-test: KALDI_ASSERT: at
> >> >> UnitTestLinearCgd:matrix-lib-test.cc:3118, failed: error < 1.0e-05
> >> >> *
> >> >> b.Norm(2.0)
> >> >>
> >> >> Has anyone faced this problem before or have any suggestions?
> >> >>
> >> >> Thanks,
> >> >> Joaquín
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> ------------------------------------------------------------------------------
> >> >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> >> >> from Actuate! Instantly Supercharge Your Business Reports and
> >> >> Dashboards
> >> >> with Interactivity, Sharing, Native Excel Exports, App Integration
> >> >> &
> >> >> more
> >> >> Get technology previously reserved for billion-dollar
> >> >> corporations,
> >> >> FREE
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
> >> >> _______________________________________________
> >> >> Kaldi-developers mailing list
> >> >> Kal...@li...
> >> >> https://lists.sourceforge.net/lists/listinfo/kaldi-developers
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >
> >
> >
>
|
