Menu
▾
▴
kaldi-developers — Kaldi Developers
You can subscribe to this list here.
| 2011 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
(4) |
Jul
|
Aug
|
Sep
(1) |
Oct
(4) |
Nov
(1) |
Dec
(14) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2012 |
Jan
(1) |
Feb
(8) |
Mar
|
Apr
(1) |
May
(3) |
Jun
(13) |
Jul
(7) |
Aug
(11) |
Sep
(6) |
Oct
(14) |
Nov
(16) |
Dec
(1) |
| 2013 |
Jan
(3) |
Feb
(8) |
Mar
(17) |
Apr
(21) |
May
(27) |
Jun
(11) |
Jul
(11) |
Aug
(21) |
Sep
(39) |
Oct
(17) |
Nov
(39) |
Dec
(28) |
| 2014 |
Jan
(36) |
Feb
(30) |
Mar
(35) |
Apr
(17) |
May
(22) |
Jun
(28) |
Jul
(23) |
Aug
(41) |
Sep
(17) |
Oct
(10) |
Nov
(22) |
Dec
(56) |
| 2015 |
Jan
(30) |
Feb
(32) |
Mar
(37) |
Apr
(28) |
May
(79) |
Jun
(18) |
Jul
(35) |
Aug
|
Sep
(1) |
Oct
|
Nov
|
Dec
|
|
From: Daniel P. <dp...@gm...> - 2015-01-07 00:37:07
|
Interestingly, Hydra is what I wanted to call the Kaldi project (I was outvoted). It's not really possible to compare the two. Hydra is a closed-source decoder, and it's only a decoder, it doesn't have a system for building models like Kaldi does. I would imagine that the Hydra decoder is faster, since they've obviously put a lot of effort into it, but the online-nnet2 decoder is sufficiently fast, in that you can get it to decode in real-time fairly easily, without much loss in accuracy by using suitable beams and a large enough chunk-size (e.g. 20 frames), and by configuring your matrix library (ATLAS, OpenBlas, MKL) to use, say, 2 threads. Although it would be very easy to use GPUs for the neural net part of the computation, there hasn't been much demand for it because if you can decode in real-time using a couple of cores of CPU, it'll generally be more efficient in terms of hardware cost than using one core of CPU, plus a GPU. Note that the online-nnet2 decoder is not really a decoder per se, it just calls the standard decoding code in lattice-faster-decoder.h, which isn't that complicated; but the online-nnet2 code takes care of various online feature estimation issues and of batching up the features into suitable size chunks so that matrix operations in the neural net code will be fast. Dan On Tue, Jan 6, 2015 at 4:02 PM, <Dan...@pa...> wrote: > CMU’s Hydra ASR decoder made a splash out here. From the references below > (or any other info you can find), does anyone have a feeling for how this > compares with the Kaldi nnet2 online decoder in speed and accuracy? > > > > http://www.cs.cmu.edu/~ianlane/publications/2012_Kim_Interspeech.pdf > > http://on-demand.gputechconf.com/gtc/2013/presentations/S3406-HYDRA-Hybrid-CPU-GPU-Speech-Recognition-Engine.pdf > > http://www.cs.cmu.edu/~ianlane/publications/SLT_JungsukKim.pdf > > http://www.nvidia.com/content/cuda/spotlights/ian-lane-cmu.html > > > > Dan > > > > > ------------------------------------------------------------------------------ > Dive into the World of Parallel Programming! The Go Parallel Website, > sponsored by Intel and developed in partnership with Slashdot Media, is your > hub for all things parallel software development, from weekly thought > leadership blogs to news, videos, case studies, tutorials and more. Take a > look and join the conversation now. http://goparallel.sourceforge.net > _______________________________________________ > Kaldi-developers mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-developers > |
|
From: <Dan...@pa...> - 2015-01-07 00:25:10
|
CMU's Hydra ASR decoder made a splash out here. From the references below (or any other info you can find), does anyone have a feeling for how this compares with the Kaldi nnet2 online decoder in speed and accuracy? http://www.cs.cmu.edu/~ianlane/publications/2012_Kim_Interspeech.pdf http://on-demand.gputechconf.com/gtc/2013/presentations/S3406-HYDRA-Hybrid-CPU-GPU-Speech-Recognition-Engine.pdf http://www.cs.cmu.edu/~ianlane/publications/SLT_JungsukKim.pdf http://www.nvidia.com/content/cuda/spotlights/ian-lane-cmu.html Dan |
|
From: Daniel P. <dp...@gm...> - 2015-01-06 19:04:46
|
I think the most likely difference relates to the acoustic scales you used. The output from decode-faster should be the same as the other pipeline, but only if the --acoustic-scale option was identical in all the stages (gmm-decode-faster, gmm-latgen-faster, lattice-1best)... note, the --lm-scale option, if provided, is the inverse of --acoustic-scale, it's an alternative way to set it. The beams should also be the same for the output to be (almost) exactly identical. Dan On Tue, Jan 6, 2015 at 10:18 AM, Xavier Anguera <xan...@gm...> wrote: > Dan, all, > while yesterday's solution worked perfectly for my needs, I later found a > "simpler" way to obtain the time codes of a phoneme decoding that seems to > also work, but is giving me a slightly different output. > I your proposal the set of steps is: steps/decode_si.sh -> lattice-1best -> > nbest-to-linear -> ali-to-phones > This generates lattices that then converts into a 1-best decoding. Instead, > I found the following to avoid outputing lattices, but just the alignments: > steps/decode_nolats.sh -> ali-to-phones > > While this second solution is faster (less steps) it is not returning the > exact same output.. I see that internally it is based on gmm-decode-faster > instead of gmm-latgen-faster. Should I worry? which one is the best solution > (if any) > > thanks, > > X. Anguera > > On Tue, Jan 6, 2015 at 12:41 AM, Xavier Anguera <xan...@gm...> wrote: >> >> Thanks Dan, >> it worked perfectly! >> >> X. >> >> >> On Mon, Jan 5, 2015 at 9:21 PM, Daniel Povey <dp...@gm...> wrote: >>> >>> Your whole pipeline is based on using the words in the lattices, not >>> the phones. In your case the words *are* the phones, because you're >>> using a phone bigram LM. So you need to do lattice-align-words, not >>> lattice-align-phones. The confidence algorithm only works on words so >>> you need to use words. >>> Alternatively, if you don't need the confidences, a more efficient way >>> to do it without lattice-align-words is to simply do lattice-1best | >>> nbest-to-linear [only keeping the alignment output] | ali-to-phones >>> (--write-lengths=true). You'll have to write a script to convert the >>> output of ali-to-phones to ctm format. >>> >>> Wei, if you have time, could you please work on adding a boolean >>> option --ctm-output to the program ali-to-phones (and an option >>> --frame-shift, default 0.01, to control the times of the ctm output)? >>> The confidences can just be 1. This issue seems to come up >>> repeatedly. >>> >>> >>> Dan >>> >>> >>> On Mon, Jan 5, 2015 at 12:10 PM, Xavier Anguera <xan...@gm...> >>> wrote: >>> > Hi, >>> > I am trying to perform phonetic decoding in Kaldi where I would like to >>> > obtain a final ctm file with a time-aligned 1-best phone sequence given >>> > my >>> > input audio. I must be missing something, as the decoded phones look >>> > good >>> > but their timings are not accurate at all. Here is what I am doing: >>> > >>> > 1) I create a phone bigram LM with utils/make_phone_bigram_lang.sh >>> > 2) I combine LM and acoustic models into a recognition graph with >>> > utils/mkgraph.sh >>> > 3) I perform the decoding of the input audio with steps/decode_si.sh >>> > 4) Obtain the 1-best CTM using the following command: >>> > lattice-align-phones --output-error-lats=true $hmm/final.mdl >>> > "ark:gunzip >>> > -c $decodedir/lat.*.gz |" ark:- | \ >>> > lattice-to-ctm-conf --decode-mbr=true --acoustic-scale=$acwt ark:- >>> > - | >>> > \ >>> > utils/int2sym.pl -f 5 $graph_or_lang/words.txt > $odir/$name.ctm >>> > || >>> > exit 1; >>> > >>> > Note that when using the same acoustic models for word decoding I get >>> > very >>> > good word-starting times. In this case I am using, in step 4, >>> > lattice-align-words instead, could this be the problem? >>> > >>> > Thanks, >>> > >>> > X. Anguera >>> > >>> > >>> > ------------------------------------------------------------------------------ >>> > Dive into the World of Parallel Programming! The Go Parallel Website, >>> > sponsored by Intel and developed in partnership with Slashdot Media, is >>> > your >>> > hub for all things parallel software development, from weekly thought >>> > leadership blogs to news, videos, case studies, tutorials and more. >>> > Take a >>> > look and join the conversation now. http://goparallel.sourceforge.net >>> > _______________________________________________ >>> > Kaldi-developers mailing list >>> > Kal...@li... >>> > https://lists.sourceforge.net/lists/listinfo/kaldi-developers >>> > >> >> > |
|
From: Xavier A. <xan...@gm...> - 2015-01-06 18:18:23
|
Dan, all, while yesterday's solution worked perfectly for my needs, I later found a "simpler" way to obtain the time codes of a phoneme decoding that seems to also work, but is giving me a slightly different output. I your proposal the set of steps is: steps/decode_si.sh -> lattice-1best -> nbest-to-linear -> ali-to-phones This generates lattices that then converts into a 1-best decoding. Instead, I found the following to avoid outputing lattices, but just the alignments: steps/decode_nolats.sh -> ali-to-phones While this second solution is faster (less steps) it is not returning the exact same output.. I see that internally it is based on gmm-decode-faster instead of gmm-latgen-faster. Should I worry? which one is the best solution (if any) thanks, X. Anguera On Tue, Jan 6, 2015 at 12:41 AM, Xavier Anguera <xan...@gm...> wrote: > Thanks Dan, > it worked perfectly! > > X. > > > On Mon, Jan 5, 2015 at 9:21 PM, Daniel Povey <dp...@gm...> wrote: > >> Your whole pipeline is based on using the words in the lattices, not >> the phones. In your case the words *are* the phones, because you're >> using a phone bigram LM. So you need to do lattice-align-words, not >> lattice-align-phones. The confidence algorithm only works on words so >> you need to use words. >> Alternatively, if you don't need the confidences, a more efficient way >> to do it without lattice-align-words is to simply do lattice-1best | >> nbest-to-linear [only keeping the alignment output] | ali-to-phones >> (--write-lengths=true). You'll have to write a script to convert the >> output of ali-to-phones to ctm format. >> >> Wei, if you have time, could you please work on adding a boolean >> option --ctm-output to the program ali-to-phones (and an option >> --frame-shift, default 0.01, to control the times of the ctm output)? >> The confidences can just be 1. This issue seems to come up >> repeatedly. >> >> >> Dan >> >> >> On Mon, Jan 5, 2015 at 12:10 PM, Xavier Anguera <xan...@gm...> >> wrote: >> > Hi, >> > I am trying to perform phonetic decoding in Kaldi where I would like to >> > obtain a final ctm file with a time-aligned 1-best phone sequence given >> my >> > input audio. I must be missing something, as the decoded phones look >> good >> > but their timings are not accurate at all. Here is what I am doing: >> > >> > 1) I create a phone bigram LM with utils/make_phone_bigram_lang.sh >> > 2) I combine LM and acoustic models into a recognition graph with >> > utils/mkgraph.sh >> > 3) I perform the decoding of the input audio with steps/decode_si.sh >> > 4) Obtain the 1-best CTM using the following command: >> > lattice-align-phones --output-error-lats=true $hmm/final.mdl >> "ark:gunzip >> > -c $decodedir/lat.*.gz |" ark:- | \ >> > lattice-to-ctm-conf --decode-mbr=true --acoustic-scale=$acwt ark:- >> - | >> > \ >> > utils/int2sym.pl -f 5 $graph_or_lang/words.txt > $odir/$name.ctm >> || >> > exit 1; >> > >> > Note that when using the same acoustic models for word decoding I get >> very >> > good word-starting times. In this case I am using, in step 4, >> > lattice-align-words instead, could this be the problem? >> > >> > Thanks, >> > >> > X. Anguera >> > >> > >> ------------------------------------------------------------------------------ >> > Dive into the World of Parallel Programming! The Go Parallel Website, >> > sponsored by Intel and developed in partnership with Slashdot Media, is >> your >> > hub for all things parallel software development, from weekly thought >> > leadership blogs to news, videos, case studies, tutorials and more. >> Take a >> > look and join the conversation now. http://goparallel.sourceforge.net >> > _______________________________________________ >> > Kaldi-developers mailing list >> > Kal...@li... >> > https://lists.sourceforge.net/lists/listinfo/kaldi-developers >> > >> > > |
|
From: Xavier A. <xan...@gm...> - 2015-01-05 23:42:05
|
Thanks Dan, it worked perfectly! X. On Mon, Jan 5, 2015 at 9:21 PM, Daniel Povey <dp...@gm...> wrote: > Your whole pipeline is based on using the words in the lattices, not > the phones. In your case the words *are* the phones, because you're > using a phone bigram LM. So you need to do lattice-align-words, not > lattice-align-phones. The confidence algorithm only works on words so > you need to use words. > Alternatively, if you don't need the confidences, a more efficient way > to do it without lattice-align-words is to simply do lattice-1best | > nbest-to-linear [only keeping the alignment output] | ali-to-phones > (--write-lengths=true). You'll have to write a script to convert the > output of ali-to-phones to ctm format. > > Wei, if you have time, could you please work on adding a boolean > option --ctm-output to the program ali-to-phones (and an option > --frame-shift, default 0.01, to control the times of the ctm output)? > The confidences can just be 1. This issue seems to come up > repeatedly. > > > Dan > > > On Mon, Jan 5, 2015 at 12:10 PM, Xavier Anguera <xan...@gm...> > wrote: > > Hi, > > I am trying to perform phonetic decoding in Kaldi where I would like to > > obtain a final ctm file with a time-aligned 1-best phone sequence given > my > > input audio. I must be missing something, as the decoded phones look good > > but their timings are not accurate at all. Here is what I am doing: > > > > 1) I create a phone bigram LM with utils/make_phone_bigram_lang.sh > > 2) I combine LM and acoustic models into a recognition graph with > > utils/mkgraph.sh > > 3) I perform the decoding of the input audio with steps/decode_si.sh > > 4) Obtain the 1-best CTM using the following command: > > lattice-align-phones --output-error-lats=true $hmm/final.mdl > "ark:gunzip > > -c $decodedir/lat.*.gz |" ark:- | \ > > lattice-to-ctm-conf --decode-mbr=true --acoustic-scale=$acwt ark:- > - | > > \ > > utils/int2sym.pl -f 5 $graph_or_lang/words.txt > $odir/$name.ctm || > > exit 1; > > > > Note that when using the same acoustic models for word decoding I get > very > > good word-starting times. In this case I am using, in step 4, > > lattice-align-words instead, could this be the problem? > > > > Thanks, > > > > X. Anguera > > > > > ------------------------------------------------------------------------------ > > Dive into the World of Parallel Programming! The Go Parallel Website, > > sponsored by Intel and developed in partnership with Slashdot Media, is > your > > hub for all things parallel software development, from weekly thought > > leadership blogs to news, videos, case studies, tutorials and more. Take > a > > look and join the conversation now. http://goparallel.sourceforge.net > > _______________________________________________ > > Kaldi-developers mailing list > > Kal...@li... > > https://lists.sourceforge.net/lists/listinfo/kaldi-developers > > > |
|
From: Daniel P. <dp...@gm...> - 2015-01-05 20:22:00
|
Your whole pipeline is based on using the words in the lattices, not the phones. In your case the words *are* the phones, because you're using a phone bigram LM. So you need to do lattice-align-words, not lattice-align-phones. The confidence algorithm only works on words so you need to use words. Alternatively, if you don't need the confidences, a more efficient way to do it without lattice-align-words is to simply do lattice-1best | nbest-to-linear [only keeping the alignment output] | ali-to-phones (--write-lengths=true). You'll have to write a script to convert the output of ali-to-phones to ctm format. Wei, if you have time, could you please work on adding a boolean option --ctm-output to the program ali-to-phones (and an option --frame-shift, default 0.01, to control the times of the ctm output)? The confidences can just be 1. This issue seems to come up repeatedly. Dan On Mon, Jan 5, 2015 at 12:10 PM, Xavier Anguera <xan...@gm...> wrote: > Hi, > I am trying to perform phonetic decoding in Kaldi where I would like to > obtain a final ctm file with a time-aligned 1-best phone sequence given my > input audio. I must be missing something, as the decoded phones look good > but their timings are not accurate at all. Here is what I am doing: > > 1) I create a phone bigram LM with utils/make_phone_bigram_lang.sh > 2) I combine LM and acoustic models into a recognition graph with > utils/mkgraph.sh > 3) I perform the decoding of the input audio with steps/decode_si.sh > 4) Obtain the 1-best CTM using the following command: > lattice-align-phones --output-error-lats=true $hmm/final.mdl "ark:gunzip > -c $decodedir/lat.*.gz |" ark:- | \ > lattice-to-ctm-conf --decode-mbr=true --acoustic-scale=$acwt ark:- - | > \ > utils/int2sym.pl -f 5 $graph_or_lang/words.txt > $odir/$name.ctm || > exit 1; > > Note that when using the same acoustic models for word decoding I get very > good word-starting times. In this case I am using, in step 4, > lattice-align-words instead, could this be the problem? > > Thanks, > > X. Anguera > > ------------------------------------------------------------------------------ > Dive into the World of Parallel Programming! The Go Parallel Website, > sponsored by Intel and developed in partnership with Slashdot Media, is your > hub for all things parallel software development, from weekly thought > leadership blogs to news, videos, case studies, tutorials and more. Take a > look and join the conversation now. http://goparallel.sourceforge.net > _______________________________________________ > Kaldi-developers mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-developers > |
|
From: Xavier A. <xan...@gm...> - 2015-01-05 20:10:21
|
Hi,
I am trying to perform phonetic decoding in Kaldi where I would like to
obtain a final ctm file with a time-aligned 1-best phone sequence given my
input audio. I must be missing something, as the decoded phones look good
but their timings are not accurate at all. Here is what I am doing:
1) I create a phone bigram LM with utils/make_phone_bigram_lang.sh
2) I combine LM and acoustic models into a recognition graph
with utils/mkgraph.sh
3) I perform the decoding of the input audio with steps/decode_si.sh
4) Obtain the 1-best CTM using the following command:
lattice-align-phones --output-error-lats=true $hmm/final.mdl
"ark:gunzip -c $decodedir/lat.*.gz |" ark:- | \
lattice-to-ctm-conf --decode-mbr=true --acoustic-scale=$acwt ark:- - |
\
utils/int2sym.pl -f 5 $graph_or_lang/words.txt > $odir/$name.ctm ||
exit 1;
Note that when using the same acoustic models for word decoding I get very
good word-starting times. In this case I am using, in step
4, lattice-align-words instead, could this be the problem?
Thanks,
X. Anguera
|
|
From: Joaquin A. R. <ja...@co...> - 2014-12-30 05:15:07
|
Thanks! On Mon, Dec 29, 2014 at 11:50 PM, Daniel Povey <dp...@gm...> wrote: > OK, I just committed a fix (by removing a slash in a script) so this > will work on a mac. > Dan > > > On Mon, Dec 29, 2014 at 8:46 PM, Joaquin Antonio Ruales > <ja...@co...> wrote: > > Hmm, it seems to be another Mac problem: > > > http://serverfault.com/questions/11518/recursive-cp-copies-folder-contents-instead-of-folder-on-os-x > > > > I believe that changing > > > > for f in phones.txt words.txt phones.txt L.fst L_disambig.fst phones/; do > > cp -r data/lang/$f $test > > done > > > > for > > > > for f in phones.txt words.txt phones.txt L.fst L_disambig.fst phones; do > > cp -r data/lang/$f $test > > done > > > > should do the trick (if "phones" has a slash after it, Mac computers copy > > the contents of the folder rather than the folder itself, so this case > > requires ommiting the /, for OSX). > > > > Thanks, > > Joaquín > > > > On Mon, Dec 29, 2014 at 11:30 PM, Daniel Povey <dp...@gm...> wrote: > >> > >> local/voxforge_format_data.sh creates that directory. See if you can > >> figure out what happened. > >> Dan > >> > >> > >> On Mon, Dec 29, 2014 at 8:28 PM, Joaquin Antonio Ruales > >> <ja...@co...> wrote: > >> > There is a "data/lang" folder, and there is a "data/lang_test" folder. > >> > The > >> > "data/lang" folder does have the "phones" subfolder, but the > >> > "data/lang_test" folder only has the "tmp" folder inside it. All the > >> > files > >> > that were expected to be inside "phones" are directly inside > >> > "data/lang_test". > >> > > >> > On Mon, Dec 29, 2014 at 11:22 PM, Daniel Povey <dp...@gm...> > wrote: > >> >> > >> >> That file should be in the phones/ subdirectory. let me look into > it. > >> >> Dan > >> >> > >> >> > >> >> On Mon, Dec 29, 2014 at 8:21 PM, Joaquin Antonio Ruales > >> >> <ja...@co...> wrote: > >> >> > Hi Daniel, > >> >> > > >> >> > All tests passed as expected. > >> >> > I am now trying to run the Voxforge recipe, and after train_mono.sh > >> >> > succeeds, I get the error: > >> >> > > >> >> > mkgraph.sh: expected data/lang_test/phones/silence.csl to exist > >> >> > > >> >> > I have checked the directories and that file is in fact missing. > >> >> > However, I > >> >> > have found a "silence.csl" file as well as other expected files > >> >> > inside > >> >> > data/lang_test/silence.csl (one level above from the expected > >> >> > directory). I > >> >> > have thus modified mkgraph.sh to reference the expected file with > the > >> >> > line: > >> >> > > >> >> > required="$lang/L.fst $lang/G.fst $lang/phones.txt $lang/words.txt > >> >> > $lang/silence.csl $lang/disambig.int $model $tree" > >> >> > > >> >> > which now references the file one directory higher. Is this the > >> >> > correct > >> >> > thing to do? or am I referencing the wrong file? > >> >> > > >> >> > After I made this change I reran ./run.sh, and the recipe didn't > >> >> > throw > >> >> > the > >> >> > error mentioned above anymore. However, I obtain more errors later > >> >> > on: > >> >> > > >> >> > fstcomposecontext --context-size=1 --central-position=0 > >> >> > --read-disambig-syms=data/lang_test/phones/disambig.int > >> >> > --write-disambig-syms=data/lang_test/tmp/disambig_ilabels_1_0.int > >> >> > data/lang_test/tmp/ilabels_1_0 > >> >> > > >> >> > ERROR (fstcomposecontext:main():fstcomposecontext.cc:126) > >> >> > fstcomposecontext: > >> >> > Could not read disambiguation symbols from > >> >> > data/lang_test/phones/disambig.int > >> >> > > >> >> > ERROR (fstcomposecontext:main():fstcomposecontext.cc:126) > >> >> > fstcomposecontext: > >> >> > Could not read disambiguation symbols from > >> >> > data/lang_test/phones/disambig.int > >> >> > > >> >> > Again, the file data/lang_test/phones/disambig.int does not exist, > >> >> > but > >> >> > the > >> >> > file data/lang_test/disambig.int (one folder above) does exist. > Is it > >> >> > correct to change the referenced disambig.int to be the one on the > >> >> > parent > >> >> > folder? > >> >> > I want to make sure that I am not making changes that don't make > >> >> > sense > >> >> > with > >> >> > mkgraph.sh and disambig.int as mentioned above. > >> >> > > >> >> > Thanks, > >> >> > Joaquín > >> >> > > >> >> > On Thu, Dec 4, 2014 at 3:05 PM, Daniel Povey <dp...@gm...> > >> >> > wrote: > >> >> >> > >> >> >> I have modified that test program and it should pass now. > >> >> >> > >> >> >> We normally don't get so many issues discovered by one person- it > >> >> >> only > >> >> >> happens when we encounter a new platform that hasn't been properly > >> >> >> tested before. > >> >> >> > >> >> >> Dan > >> >> >> > >> >> >> > >> >> >> On Thu, Dec 4, 2014 at 4:34 AM, Joaquin Antonio Ruales > >> >> >> <ja...@co...> wrote: > >> >> >> > Yep! now the matrix test works. Now I'm getting an error on the > >> >> >> > mle-full-gmm-test. could it be another problem only relevant to > >> >> >> > OSX > >> >> >> > Mavericks and Yosemite? Here are the last few lines of output of > >> >> >> > ./mle-full-gmm-test: > >> >> >> > > >> >> >> > Condition number of random matrix large 1.558446e+02, trying > again > >> >> >> > (this > >> >> >> > is > >> >> >> > normal) > >> >> >> > > >> >> >> > Avg log-like per frame [full-cov, 1-mix] should be: > -2.535968e+01 > >> >> >> > > >> >> >> > Total log-like [full-cov, 1-mix] should be: -3.550355e+04 > >> >> >> > > >> >> >> > Cov eigs are [ 1.639155e+01 1.397001e+01 1.149592e+01 > >> >> >> > 1.090052e+01 > >> >> >> > 1.049452e+01 8.523492e+00 7.486929e+00 7.382205e+00 6.812490e+00 > >> >> >> > 5.214448e+00 ] > >> >> >> > > >> >> >> > LOG (UnitTestEstimateFullGmm():mle-full-gmm-test.cc:361) Testing > >> >> >> > natural<>normal conversion > >> >> >> > > >> >> >> > KALDI_ASSERT: at > UnitTestEstimateFullGmm:mle-full-gmm-test.cc:374, > >> >> >> > failed: > >> >> >> > std::abs(gmm->means_invcovars().Row(0)(d) - > >> >> >> > rgmm.means_invcovars().Row(0)(d)) < prec_v > >> >> >> > > >> >> >> > Stack trace is: > >> >> >> > > >> >> >> > 0 mle-full-gmm-test 0x000000010b873bb7 > >> >> >> > _ZN5kaldi18KaldiGetStackTraceEv + 71 > >> >> >> > > >> >> >> > 1 mle-full-gmm-test 0x000000010b873f86 > >> >> >> > _ZN5kaldi19KaldiAssertFailure_EPKcS1_iS1_ + 230 > >> >> >> > > >> >> >> > 2 mle-full-gmm-test 0x000000010b78f8c6 > >> >> >> > _Z23UnitTestEstimateFullGmmv + 4774 > >> >> >> > > >> >> >> > 3 mle-full-gmm-test 0x000000010b790a48 main > + > >> >> >> > 40 > >> >> >> > > >> >> >> > 4 libdyld.dylib 0x000000010bb725fd > start + > >> >> >> > 1 > >> >> >> > > >> >> >> > 5 ??? 0x0000000000000001 0x0 > + 1 > >> >> >> > > >> >> >> > Abort trap: 6 > >> >> >> > > >> >> >> > > >> >> >> > On Wed, Dec 3, 2014 at 8:10 PM, Dogan Can <dog...@us...> > >> >> >> > wrote: > >> >> >> >> > >> >> >> >> Hi Dan, > >> >> >> >> > >> >> >> >> It turns out this was due to a bug in our RandGauss2 function > >> >> >> >> that > >> >> >> >> accepts > >> >> >> >> double arguments. I’m committing the following fix to kaldi > trunk > >> >> >> >> right > >> >> >> >> away. > >> >> >> >> > >> >> >> >> Cheers, > >> >> >> >> Dogan > >> >> >> >> > >> >> >> >> diff --git a/src/base/kaldi-math.cc b/src/base/kaldi-math.cc > >> >> >> >> index 5b30320..0de73bf 100644 > >> >> >> >> --- a/src/base/kaldi-math.cc > >> >> >> >> +++ b/src/base/kaldi-math.cc > >> >> >> >> @@ -147,7 +147,7 @@ void RandGauss2(double *a, double *b, > >> >> >> >> RandomState > >> >> >> >> *state) > >> >> >> >> float a_float, b_float; > >> >> >> >> // Just because we're using doubles doesn't mean we need > >> >> >> >> super-high-quality > >> >> >> >> // random numbers, so we just use the floating-point version > >> >> >> >> internally. > >> >> >> >> - RandGauss2(&a_float, &b_float); > >> >> >> >> + RandGauss2(&a_float, &b_float, state); > >> >> >> >> *a = a_float; *b = b_float; > >> >> >> >> } > >> >> >> >> > >> >> >> >> On Dec 3, 2014, at 2:23 AM, Dogan Can <dog...@us...> > wrote: > >> >> >> >> > >> >> >> >> Hi Dan > >> >> >> >> > >> >> >> >> I don’t have a Mavericks setup but I can replicate the failure > on > >> >> >> >> Yosemite > >> >> >> >> with clang. When everything is compiled with gcc, > matrix-lib-test > >> >> >> >> finishes > >> >> >> >> successfully. > >> >> >> >> > >> >> >> >> I looked a bit into the root cause of why the test was going > into > >> >> >> >> an > >> >> >> >> infinite loop. The random 3x3 matrices generated during the > >> >> >> >> failing > >> >> >> >> test end > >> >> >> >> up having the same first and third columns and fail the > condition > >> >> >> >> number > >> >> >> >> test, hence the generation loop never terminates. I did some > >> >> >> >> sleuthing > >> >> >> >> to > >> >> >> >> figure out how that was happening and it turns out the value of > >> >> >> >> rstate.seed > >> >> >> >> (see the snippet below from matrix/kaldi-matrix.cc) is the same > >> >> >> >> before > >> >> >> >> and > >> >> >> >> after the call to RandGauss2 (note that inner for loop executes > >> >> >> >> only > >> >> >> >> once). > >> >> >> >> It seems like we are hitting a race condition in the system > >> >> >> >> provided > >> >> >> >> rand_r > >> >> >> >> implementation or a compiler bug. > >> >> >> >> > >> >> >> >> > >> >> >> >> 1093 template<typename Real> > >> >> >> >> 1094 void MatrixBase<Real>::SetRandn() { > >> >> >> >> 1095 kaldi::RandomState rstate; > >> >> >> >> 1096 for (MatrixIndexT row = 0; row < num_rows_; row++) { > >> >> >> >> 1097 Real *row_data = this->RowData(row); > >> >> >> >> 1098 MatrixIndexT nc = (num_cols_ % 2 == 1) ? num_cols_ - > 1 : > >> >> >> >> num_cols_; > >> >> >> >> 1099 for (MatrixIndexT col = 0; col < nc; col += 2) { > >> >> >> >> 1100 kaldi::RandGauss2(row_data + col, row_data + col + > 1, > >> >> >> >> &rstate); > >> >> >> >> 1101 } > >> >> >> >> 1102 if (nc != num_cols_) row_data[nc] = > >> >> >> >> static_cast<Real>(kaldi::RandGauss(&rstate)); > >> >> >> >> 1103 } > >> >> >> >> 1104 } > >> >> >> >> > >> >> >> >> > >> >> >> >> Cheers, > >> >> >> >> Dogan > >> >> >> >> > >> >> >> >> On Dec 2, 2014, at 10:13 PM, Daniel Povey <dp...@gm...> > >> >> >> >> wrote: > >> >> >> >> > >> >> >> >> According to > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > http://stackoverflow.com/questions/19554439/gdb-missing-in-os-x-mavericks > >> >> >> >> gdb is no longer supported in mavericks and you have to use > lldb. > >> >> >> >> I've > >> >> >> >> never used that so I don't know what to tell you to do. > >> >> >> >> Dogan, do you have a mavericks setup, and can you see if you > get > >> >> >> >> the > >> >> >> >> infinite loop that he gets when you test? > >> >> >> >> Joaqin: I would just continue through the tutorial for now. > >> >> >> >> Probably > >> >> >> >> it's not going to be a problem. > >> >> >> >> Dan > >> >> >> >> > >> >> >> >> > >> >> >> >> On Wed, Dec 3, 2014 at 1:09 AM, Daniel Povey <dp...@gm... > > > >> >> >> >> wrote: > >> >> >> >> > >> >> >> >> Hm. That makes me think there may be a problem with your > gcc/gdb > >> >> >> >> installation- maybe you could try uninstalling it and > installing > >> >> >> >> it > >> >> >> >> again. Did you get it from MacPorts? > >> >> >> >> Dan > >> >> >> >> > >> >> >> >> > >> >> >> >> On Wed, Dec 3, 2014 at 1:05 AM, Joaquin Antonio Ruales > >> >> >> >> <ja...@co...> wrote: > >> >> >> >> > >> >> >> >> Now I get: > >> >> >> >> > >> >> >> >> > >> >> >> >> Program received signal SIGINT, Interrupt. > >> >> >> >> > >> >> >> >> 0x00007fff8c755e9a in ?? () > >> >> >> >> > >> >> >> >> (gdb) b matrix-lib-test.cc:39 > >> >> >> >> > >> >> >> >> Cannot access memory at address 0xab670 > >> >> >> >> > >> >> >> >> > >> >> >> >> On Wed, Dec 3, 2014 at 12:57 AM, Daniel Povey < > dp...@gm...> > >> >> >> >> wrote: > >> >> >> >> > >> >> >> >> > >> >> >> >> Probably when you did ctrl-c it was in the Atlas library, > >> >> >> >> sometimes > >> >> >> >> it > >> >> >> >> can't get a proper backtrace. Instead, after doing ctrl-c do: > >> >> >> >> (gdb) b matrix-lib-test.cc:39 > >> >> >> >> (gdb) c > >> >> >> >> > >> >> >> >> to continue, and when it breaks, show me a backtrace. > >> >> >> >> Dan > >> >> >> >> > >> >> >> >> > >> >> >> >> On Wed, Dec 3, 2014 at 12:48 AM, Joaquin Antonio Ruales > >> >> >> >> <ja...@co...> wrote: > >> >> >> >> > >> >> >> >> This is what I get: > >> >> >> >> > >> >> >> >> > >> >> >> >> Program received signal SIGINT, Interrupt. > >> >> >> >> > >> >> >> >> 0x00007fff8b2c6342 in ?? () > >> >> >> >> > >> >> >> >> (gdb) bt > >> >> >> >> > >> >> >> >> #0 0x00007fff8b2c6342 in ?? () > >> >> >> >> > >> >> >> >> #1 0x00007fff5fbfdbc8 in ?? () > >> >> >> >> > >> >> >> >> #2 0x0000000000000001 in ?? () > >> >> >> >> > >> >> >> >> #3 0x00007fff5fbfdfdc in ?? () > >> >> >> >> > >> >> >> >> #4 0x00007fff5fbfdfe0 in ?? () > >> >> >> >> > >> >> >> >> #5 0x00007fff5fbfdbbe in ?? () > >> >> >> >> > >> >> >> >> #6 0x0000000100102672 in ?? () > >> >> >> >> > >> >> >> >> #7 0x00007fff5fbfdc10 in ?? () > >> >> >> >> > >> >> >> >> #8 0x00007fff8b00eac4 in ?? () > >> >> >> >> > >> >> >> >> #9 0x00007fff5fbfda90 in ?? () > >> >> >> >> > >> >> >> >> #10 0x00007fff8e2bf272 in ?? () > >> >> >> >> > >> >> >> >> #11 0x00000002001a0a00 in ?? () > >> >> >> >> > >> >> >> >> #12 0x00000001001a0a00 in ?? () > >> >> >> >> > >> >> >> >> #13 0x000000010019d000 in ?? () > >> >> >> >> > >> >> >> >> #14 0x00000001001a0a00 in ?? () > >> >> >> >> > >> >> >> >> #15 0x00000001003002ca in ?? () > >> >> >> >> > >> >> >> >> #16 0x0000000000000000 in ?? () > >> >> >> >> > >> >> >> >> > >> >> >> >> On Wed, Dec 3, 2014 at 12:42 AM, Daniel Povey < > dp...@gm...> > >> >> >> >> wrote: > >> >> >> >> > >> >> >> >> > >> >> >> >> That isn't normal and it doesn't happen when I compile on a > Mac. > >> >> >> >> Can you run it in gdb by doing > >> >> >> >> gdb ./matrix-lib-test > >> >> >> >> (gdb) r > >> >> >> >> > >> >> >> >> and then do ctrl-c when it gets in the loop and type "bt" and > >> >> >> >> show > >> >> >> >> me > >> >> >> >> the backtrace? > >> >> >> >> Dan > >> >> >> >> > >> >> >> >> > >> >> >> >> On Wed, Dec 3, 2014 at 12:36 AM, Joaquin Antonio Ruales > >> >> >> >> <ja...@co...> wrote: > >> >> >> >> > >> >> >> >> Hi Dan, > >> >> >> >> > >> >> >> >> Thanks for the quick reply. I have "svn up"ed and recompiled > >> >> >> >> everything, > >> >> >> >> but > >> >> >> >> now the same matrix test runs into an infinite loop. Would it > be > >> >> >> >> safe > >> >> >> >> to > >> >> >> >> ignore the test results and continue with the tutorial? Here > are > >> >> >> >> the > >> >> >> >> few > >> >> >> >> lines of output from the test: > >> >> >> >> > >> >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition > number > >> >> >> >> of > >> >> >> >> random > >> >> >> >> matrix large 7.50727e+16, trying again (this is normal) > >> >> >> >> > >> >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition > number > >> >> >> >> of > >> >> >> >> random > >> >> >> >> matrix large 1.91265e+16, trying again (this is normal) > >> >> >> >> > >> >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition > number > >> >> >> >> of > >> >> >> >> random > >> >> >> >> matrix large 7.70738e+16, trying again (this is normal) > >> >> >> >> > >> >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition > number > >> >> >> >> of > >> >> >> >> random > >> >> >> >> matrix large 2.32488e+17, trying again (this is normal) > >> >> >> >> > >> >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition > number > >> >> >> >> of > >> >> >> >> random > >> >> >> >> matrix large 8.44565e+17, trying again (this is normal) > >> >> >> >> > >> >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition > number > >> >> >> >> of > >> >> >> >> random > >> >> >> >> matrix large 1.83557e+17, trying again (this is normal) > >> >> >> >> > >> >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition > number > >> >> >> >> of > >> >> >> >> random > >> >> >> >> matrix large 2.61736e+17, trying again (this is normal) > >> >> >> >> > >> >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition > number > >> >> >> >> of > >> >> >> >> random > >> >> >> >> matrix large 8.67803e+16, trying again (this is normal) > >> >> >> >> > >> >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition > number > >> >> >> >> of > >> >> >> >> random > >> >> >> >> matrix large 1.80053e+16, trying again (this is normal) > >> >> >> >> > >> >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition > number > >> >> >> >> of > >> >> >> >> random > >> >> >> >> matrix large 1.67101e+16, trying again (this is normal) > >> >> >> >> > >> >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition > number > >> >> >> >> of > >> >> >> >> random > >> >> >> >> matrix large 1.49485e+16, trying again (this is normal) > >> >> >> >> > >> >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition > number > >> >> >> >> of > >> >> >> >> random > >> >> >> >> matrix large 8.31813e+16, trying again (this is normal) > >> >> >> >> > >> >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition > number > >> >> >> >> of > >> >> >> >> random > >> >> >> >> matrix large 1.6285e+16, trying again (this is normal) > >> >> >> >> > >> >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition > number > >> >> >> >> of > >> >> >> >> random > >> >> >> >> matrix large 2.48172e+16, trying again (this is normal) > >> >> >> >> > >> >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition > number > >> >> >> >> of > >> >> >> >> random > >> >> >> >> matrix large 2.1569e+16, trying again (this is normal) > >> >> >> >> > >> >> >> >> > >> >> >> >> On Tue, Dec 2, 2014 at 4:35 PM, Daniel Povey <dp...@gm... > > > >> >> >> >> wrote: > >> >> >> >> > >> >> >> >> > >> >> >> >> I think I had noticed this before, and I fixed the test by > >> >> >> >> changing > >> >> >> >> the threshold. > >> >> >> >> If you do "svn up" and recompile, it should pass. > >> >> >> >> Dan > >> >> >> >> > >> >> >> >> > >> >> >> >> On Tue, Dec 2, 2014 at 4:20 PM, Joaquín Ruales > >> >> >> >> <ja...@co...> > >> >> >> >> wrote: > >> >> >> >> > >> >> >> >> Hi Kaldi Team, > >> >> >> >> > >> >> >> >> I'm running into trouble when running the Kaldi tests (make > test) > >> >> >> >> described > >> >> >> >> in the Kaldi tutorial. I'm running it on a Mac (Mavericks) and > >> >> >> >> the > >> >> >> >> error > >> >> >> >> is > >> >> >> >> in matrix-lib-test: KALDI_ASSERT: at > >> >> >> >> UnitTestLinearCgd:matrix-lib-test.cc:3118, failed: error < > >> >> >> >> 1.0e-05 > >> >> >> >> * > >> >> >> >> b.Norm(2.0) > >> >> >> >> > >> >> >> >> Has anyone faced this problem before or have any suggestions? > >> >> >> >> > >> >> >> >> Thanks, > >> >> >> >> Joaquín > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > ------------------------------------------------------------------------------ > >> >> >> >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT > Server > >> >> >> >> from Actuate! Instantly Supercharge Your Business Reports and > >> >> >> >> Dashboards > >> >> >> >> with Interactivity, Sharing, Native Excel Exports, App > >> >> >> >> Integration > >> >> >> >> & > >> >> >> >> more > >> >> >> >> Get technology previously reserved for billion-dollar > >> >> >> >> corporations, > >> >> >> >> FREE > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > >> >> >> >> _______________________________________________ > >> >> >> >> Kaldi-developers mailing list > >> >> >> >> Kal...@li... > >> >> >> >> https://lists.sourceforge.net/lists/listinfo/kaldi-developers > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> > > >> >> > > >> >> > > >> > > >> > > > > > > |
|
From: Daniel P. <dp...@gm...> - 2014-12-30 04:50:37
|
OK, I just committed a fix (by removing a slash in a script) so this will work on a mac. Dan On Mon, Dec 29, 2014 at 8:46 PM, Joaquin Antonio Ruales <ja...@co...> wrote: > Hmm, it seems to be another Mac problem: > http://serverfault.com/questions/11518/recursive-cp-copies-folder-contents-instead-of-folder-on-os-x > > I believe that changing > > for f in phones.txt words.txt phones.txt L.fst L_disambig.fst phones/; do > cp -r data/lang/$f $test > done > > for > > for f in phones.txt words.txt phones.txt L.fst L_disambig.fst phones; do > cp -r data/lang/$f $test > done > > should do the trick (if "phones" has a slash after it, Mac computers copy > the contents of the folder rather than the folder itself, so this case > requires ommiting the /, for OSX). > > Thanks, > Joaquín > > On Mon, Dec 29, 2014 at 11:30 PM, Daniel Povey <dp...@gm...> wrote: >> >> local/voxforge_format_data.sh creates that directory. See if you can >> figure out what happened. >> Dan >> >> >> On Mon, Dec 29, 2014 at 8:28 PM, Joaquin Antonio Ruales >> <ja...@co...> wrote: >> > There is a "data/lang" folder, and there is a "data/lang_test" folder. >> > The >> > "data/lang" folder does have the "phones" subfolder, but the >> > "data/lang_test" folder only has the "tmp" folder inside it. All the >> > files >> > that were expected to be inside "phones" are directly inside >> > "data/lang_test". >> > >> > On Mon, Dec 29, 2014 at 11:22 PM, Daniel Povey <dp...@gm...> wrote: >> >> >> >> That file should be in the phones/ subdirectory. let me look into it. >> >> Dan >> >> >> >> >> >> On Mon, Dec 29, 2014 at 8:21 PM, Joaquin Antonio Ruales >> >> <ja...@co...> wrote: >> >> > Hi Daniel, >> >> > >> >> > All tests passed as expected. >> >> > I am now trying to run the Voxforge recipe, and after train_mono.sh >> >> > succeeds, I get the error: >> >> > >> >> > mkgraph.sh: expected data/lang_test/phones/silence.csl to exist >> >> > >> >> > I have checked the directories and that file is in fact missing. >> >> > However, I >> >> > have found a "silence.csl" file as well as other expected files >> >> > inside >> >> > data/lang_test/silence.csl (one level above from the expected >> >> > directory). I >> >> > have thus modified mkgraph.sh to reference the expected file with the >> >> > line: >> >> > >> >> > required="$lang/L.fst $lang/G.fst $lang/phones.txt $lang/words.txt >> >> > $lang/silence.csl $lang/disambig.int $model $tree" >> >> > >> >> > which now references the file one directory higher. Is this the >> >> > correct >> >> > thing to do? or am I referencing the wrong file? >> >> > >> >> > After I made this change I reran ./run.sh, and the recipe didn't >> >> > throw >> >> > the >> >> > error mentioned above anymore. However, I obtain more errors later >> >> > on: >> >> > >> >> > fstcomposecontext --context-size=1 --central-position=0 >> >> > --read-disambig-syms=data/lang_test/phones/disambig.int >> >> > --write-disambig-syms=data/lang_test/tmp/disambig_ilabels_1_0.int >> >> > data/lang_test/tmp/ilabels_1_0 >> >> > >> >> > ERROR (fstcomposecontext:main():fstcomposecontext.cc:126) >> >> > fstcomposecontext: >> >> > Could not read disambiguation symbols from >> >> > data/lang_test/phones/disambig.int >> >> > >> >> > ERROR (fstcomposecontext:main():fstcomposecontext.cc:126) >> >> > fstcomposecontext: >> >> > Could not read disambiguation symbols from >> >> > data/lang_test/phones/disambig.int >> >> > >> >> > Again, the file data/lang_test/phones/disambig.int does not exist, >> >> > but >> >> > the >> >> > file data/lang_test/disambig.int (one folder above) does exist. Is it >> >> > correct to change the referenced disambig.int to be the one on the >> >> > parent >> >> > folder? >> >> > I want to make sure that I am not making changes that don't make >> >> > sense >> >> > with >> >> > mkgraph.sh and disambig.int as mentioned above. >> >> > >> >> > Thanks, >> >> > Joaquín >> >> > >> >> > On Thu, Dec 4, 2014 at 3:05 PM, Daniel Povey <dp...@gm...> >> >> > wrote: >> >> >> >> >> >> I have modified that test program and it should pass now. >> >> >> >> >> >> We normally don't get so many issues discovered by one person- it >> >> >> only >> >> >> happens when we encounter a new platform that hasn't been properly >> >> >> tested before. >> >> >> >> >> >> Dan >> >> >> >> >> >> >> >> >> On Thu, Dec 4, 2014 at 4:34 AM, Joaquin Antonio Ruales >> >> >> <ja...@co...> wrote: >> >> >> > Yep! now the matrix test works. Now I'm getting an error on the >> >> >> > mle-full-gmm-test. could it be another problem only relevant to >> >> >> > OSX >> >> >> > Mavericks and Yosemite? Here are the last few lines of output of >> >> >> > ./mle-full-gmm-test: >> >> >> > >> >> >> > Condition number of random matrix large 1.558446e+02, trying again >> >> >> > (this >> >> >> > is >> >> >> > normal) >> >> >> > >> >> >> > Avg log-like per frame [full-cov, 1-mix] should be: -2.535968e+01 >> >> >> > >> >> >> > Total log-like [full-cov, 1-mix] should be: -3.550355e+04 >> >> >> > >> >> >> > Cov eigs are [ 1.639155e+01 1.397001e+01 1.149592e+01 >> >> >> > 1.090052e+01 >> >> >> > 1.049452e+01 8.523492e+00 7.486929e+00 7.382205e+00 6.812490e+00 >> >> >> > 5.214448e+00 ] >> >> >> > >> >> >> > LOG (UnitTestEstimateFullGmm():mle-full-gmm-test.cc:361) Testing >> >> >> > natural<>normal conversion >> >> >> > >> >> >> > KALDI_ASSERT: at UnitTestEstimateFullGmm:mle-full-gmm-test.cc:374, >> >> >> > failed: >> >> >> > std::abs(gmm->means_invcovars().Row(0)(d) - >> >> >> > rgmm.means_invcovars().Row(0)(d)) < prec_v >> >> >> > >> >> >> > Stack trace is: >> >> >> > >> >> >> > 0 mle-full-gmm-test 0x000000010b873bb7 >> >> >> > _ZN5kaldi18KaldiGetStackTraceEv + 71 >> >> >> > >> >> >> > 1 mle-full-gmm-test 0x000000010b873f86 >> >> >> > _ZN5kaldi19KaldiAssertFailure_EPKcS1_iS1_ + 230 >> >> >> > >> >> >> > 2 mle-full-gmm-test 0x000000010b78f8c6 >> >> >> > _Z23UnitTestEstimateFullGmmv + 4774 >> >> >> > >> >> >> > 3 mle-full-gmm-test 0x000000010b790a48 main + >> >> >> > 40 >> >> >> > >> >> >> > 4 libdyld.dylib 0x000000010bb725fd start + >> >> >> > 1 >> >> >> > >> >> >> > 5 ??? 0x0000000000000001 0x0 + 1 >> >> >> > >> >> >> > Abort trap: 6 >> >> >> > >> >> >> > >> >> >> > On Wed, Dec 3, 2014 at 8:10 PM, Dogan Can <dog...@us...> >> >> >> > wrote: >> >> >> >> >> >> >> >> Hi Dan, >> >> >> >> >> >> >> >> It turns out this was due to a bug in our RandGauss2 function >> >> >> >> that >> >> >> >> accepts >> >> >> >> double arguments. I’m committing the following fix to kaldi trunk >> >> >> >> right >> >> >> >> away. >> >> >> >> >> >> >> >> Cheers, >> >> >> >> Dogan >> >> >> >> >> >> >> >> diff --git a/src/base/kaldi-math.cc b/src/base/kaldi-math.cc >> >> >> >> index 5b30320..0de73bf 100644 >> >> >> >> --- a/src/base/kaldi-math.cc >> >> >> >> +++ b/src/base/kaldi-math.cc >> >> >> >> @@ -147,7 +147,7 @@ void RandGauss2(double *a, double *b, >> >> >> >> RandomState >> >> >> >> *state) >> >> >> >> float a_float, b_float; >> >> >> >> // Just because we're using doubles doesn't mean we need >> >> >> >> super-high-quality >> >> >> >> // random numbers, so we just use the floating-point version >> >> >> >> internally. >> >> >> >> - RandGauss2(&a_float, &b_float); >> >> >> >> + RandGauss2(&a_float, &b_float, state); >> >> >> >> *a = a_float; *b = b_float; >> >> >> >> } >> >> >> >> >> >> >> >> On Dec 3, 2014, at 2:23 AM, Dogan Can <dog...@us...> wrote: >> >> >> >> >> >> >> >> Hi Dan >> >> >> >> >> >> >> >> I don’t have a Mavericks setup but I can replicate the failure on >> >> >> >> Yosemite >> >> >> >> with clang. When everything is compiled with gcc, matrix-lib-test >> >> >> >> finishes >> >> >> >> successfully. >> >> >> >> >> >> >> >> I looked a bit into the root cause of why the test was going into >> >> >> >> an >> >> >> >> infinite loop. The random 3x3 matrices generated during the >> >> >> >> failing >> >> >> >> test end >> >> >> >> up having the same first and third columns and fail the condition >> >> >> >> number >> >> >> >> test, hence the generation loop never terminates. I did some >> >> >> >> sleuthing >> >> >> >> to >> >> >> >> figure out how that was happening and it turns out the value of >> >> >> >> rstate.seed >> >> >> >> (see the snippet below from matrix/kaldi-matrix.cc) is the same >> >> >> >> before >> >> >> >> and >> >> >> >> after the call to RandGauss2 (note that inner for loop executes >> >> >> >> only >> >> >> >> once). >> >> >> >> It seems like we are hitting a race condition in the system >> >> >> >> provided >> >> >> >> rand_r >> >> >> >> implementation or a compiler bug. >> >> >> >> >> >> >> >> >> >> >> >> 1093 template<typename Real> >> >> >> >> 1094 void MatrixBase<Real>::SetRandn() { >> >> >> >> 1095 kaldi::RandomState rstate; >> >> >> >> 1096 for (MatrixIndexT row = 0; row < num_rows_; row++) { >> >> >> >> 1097 Real *row_data = this->RowData(row); >> >> >> >> 1098 MatrixIndexT nc = (num_cols_ % 2 == 1) ? num_cols_ - 1 : >> >> >> >> num_cols_; >> >> >> >> 1099 for (MatrixIndexT col = 0; col < nc; col += 2) { >> >> >> >> 1100 kaldi::RandGauss2(row_data + col, row_data + col + 1, >> >> >> >> &rstate); >> >> >> >> 1101 } >> >> >> >> 1102 if (nc != num_cols_) row_data[nc] = >> >> >> >> static_cast<Real>(kaldi::RandGauss(&rstate)); >> >> >> >> 1103 } >> >> >> >> 1104 } >> >> >> >> >> >> >> >> >> >> >> >> Cheers, >> >> >> >> Dogan >> >> >> >> >> >> >> >> On Dec 2, 2014, at 10:13 PM, Daniel Povey <dp...@gm...> >> >> >> >> wrote: >> >> >> >> >> >> >> >> According to >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> http://stackoverflow.com/questions/19554439/gdb-missing-in-os-x-mavericks >> >> >> >> gdb is no longer supported in mavericks and you have to use lldb. >> >> >> >> I've >> >> >> >> never used that so I don't know what to tell you to do. >> >> >> >> Dogan, do you have a mavericks setup, and can you see if you get >> >> >> >> the >> >> >> >> infinite loop that he gets when you test? >> >> >> >> Joaqin: I would just continue through the tutorial for now. >> >> >> >> Probably >> >> >> >> it's not going to be a problem. >> >> >> >> Dan >> >> >> >> >> >> >> >> >> >> >> >> On Wed, Dec 3, 2014 at 1:09 AM, Daniel Povey <dp...@gm...> >> >> >> >> wrote: >> >> >> >> >> >> >> >> Hm. That makes me think there may be a problem with your gcc/gdb >> >> >> >> installation- maybe you could try uninstalling it and installing >> >> >> >> it >> >> >> >> again. Did you get it from MacPorts? >> >> >> >> Dan >> >> >> >> >> >> >> >> >> >> >> >> On Wed, Dec 3, 2014 at 1:05 AM, Joaquin Antonio Ruales >> >> >> >> <ja...@co...> wrote: >> >> >> >> >> >> >> >> Now I get: >> >> >> >> >> >> >> >> >> >> >> >> Program received signal SIGINT, Interrupt. >> >> >> >> >> >> >> >> 0x00007fff8c755e9a in ?? () >> >> >> >> >> >> >> >> (gdb) b matrix-lib-test.cc:39 >> >> >> >> >> >> >> >> Cannot access memory at address 0xab670 >> >> >> >> >> >> >> >> >> >> >> >> On Wed, Dec 3, 2014 at 12:57 AM, Daniel Povey <dp...@gm...> >> >> >> >> wrote: >> >> >> >> >> >> >> >> >> >> >> >> Probably when you did ctrl-c it was in the Atlas library, >> >> >> >> sometimes >> >> >> >> it >> >> >> >> can't get a proper backtrace. Instead, after doing ctrl-c do: >> >> >> >> (gdb) b matrix-lib-test.cc:39 >> >> >> >> (gdb) c >> >> >> >> >> >> >> >> to continue, and when it breaks, show me a backtrace. >> >> >> >> Dan >> >> >> >> >> >> >> >> >> >> >> >> On Wed, Dec 3, 2014 at 12:48 AM, Joaquin Antonio Ruales >> >> >> >> <ja...@co...> wrote: >> >> >> >> >> >> >> >> This is what I get: >> >> >> >> >> >> >> >> >> >> >> >> Program received signal SIGINT, Interrupt. >> >> >> >> >> >> >> >> 0x00007fff8b2c6342 in ?? () >> >> >> >> >> >> >> >> (gdb) bt >> >> >> >> >> >> >> >> #0 0x00007fff8b2c6342 in ?? () >> >> >> >> >> >> >> >> #1 0x00007fff5fbfdbc8 in ?? () >> >> >> >> >> >> >> >> #2 0x0000000000000001 in ?? () >> >> >> >> >> >> >> >> #3 0x00007fff5fbfdfdc in ?? () >> >> >> >> >> >> >> >> #4 0x00007fff5fbfdfe0 in ?? () >> >> >> >> >> >> >> >> #5 0x00007fff5fbfdbbe in ?? () >> >> >> >> >> >> >> >> #6 0x0000000100102672 in ?? () >> >> >> >> >> >> >> >> #7 0x00007fff5fbfdc10 in ?? () >> >> >> >> >> >> >> >> #8 0x00007fff8b00eac4 in ?? () >> >> >> >> >> >> >> >> #9 0x00007fff5fbfda90 in ?? () >> >> >> >> >> >> >> >> #10 0x00007fff8e2bf272 in ?? () >> >> >> >> >> >> >> >> #11 0x00000002001a0a00 in ?? () >> >> >> >> >> >> >> >> #12 0x00000001001a0a00 in ?? () >> >> >> >> >> >> >> >> #13 0x000000010019d000 in ?? () >> >> >> >> >> >> >> >> #14 0x00000001001a0a00 in ?? () >> >> >> >> >> >> >> >> #15 0x00000001003002ca in ?? () >> >> >> >> >> >> >> >> #16 0x0000000000000000 in ?? () >> >> >> >> >> >> >> >> >> >> >> >> On Wed, Dec 3, 2014 at 12:42 AM, Daniel Povey <dp...@gm...> >> >> >> >> wrote: >> >> >> >> >> >> >> >> >> >> >> >> That isn't normal and it doesn't happen when I compile on a Mac. >> >> >> >> Can you run it in gdb by doing >> >> >> >> gdb ./matrix-lib-test >> >> >> >> (gdb) r >> >> >> >> >> >> >> >> and then do ctrl-c when it gets in the loop and type "bt" and >> >> >> >> show >> >> >> >> me >> >> >> >> the backtrace? >> >> >> >> Dan >> >> >> >> >> >> >> >> >> >> >> >> On Wed, Dec 3, 2014 at 12:36 AM, Joaquin Antonio Ruales >> >> >> >> <ja...@co...> wrote: >> >> >> >> >> >> >> >> Hi Dan, >> >> >> >> >> >> >> >> Thanks for the quick reply. I have "svn up"ed and recompiled >> >> >> >> everything, >> >> >> >> but >> >> >> >> now the same matrix test runs into an infinite loop. Would it be >> >> >> >> safe >> >> >> >> to >> >> >> >> ignore the test results and continue with the tutorial? Here are >> >> >> >> the >> >> >> >> few >> >> >> >> lines of output from the test: >> >> >> >> >> >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number >> >> >> >> of >> >> >> >> random >> >> >> >> matrix large 7.50727e+16, trying again (this is normal) >> >> >> >> >> >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number >> >> >> >> of >> >> >> >> random >> >> >> >> matrix large 1.91265e+16, trying again (this is normal) >> >> >> >> >> >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number >> >> >> >> of >> >> >> >> random >> >> >> >> matrix large 7.70738e+16, trying again (this is normal) >> >> >> >> >> >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number >> >> >> >> of >> >> >> >> random >> >> >> >> matrix large 2.32488e+17, trying again (this is normal) >> >> >> >> >> >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number >> >> >> >> of >> >> >> >> random >> >> >> >> matrix large 8.44565e+17, trying again (this is normal) >> >> >> >> >> >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number >> >> >> >> of >> >> >> >> random >> >> >> >> matrix large 1.83557e+17, trying again (this is normal) >> >> >> >> >> >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number >> >> >> >> of >> >> >> >> random >> >> >> >> matrix large 2.61736e+17, trying again (this is normal) >> >> >> >> >> >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number >> >> >> >> of >> >> >> >> random >> >> >> >> matrix large 8.67803e+16, trying again (this is normal) >> >> >> >> >> >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number >> >> >> >> of >> >> >> >> random >> >> >> >> matrix large 1.80053e+16, trying again (this is normal) >> >> >> >> >> >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number >> >> >> >> of >> >> >> >> random >> >> >> >> matrix large 1.67101e+16, trying again (this is normal) >> >> >> >> >> >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number >> >> >> >> of >> >> >> >> random >> >> >> >> matrix large 1.49485e+16, trying again (this is normal) >> >> >> >> >> >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number >> >> >> >> of >> >> >> >> random >> >> >> >> matrix large 8.31813e+16, trying again (this is normal) >> >> >> >> >> >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number >> >> >> >> of >> >> >> >> random >> >> >> >> matrix large 1.6285e+16, trying again (this is normal) >> >> >> >> >> >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number >> >> >> >> of >> >> >> >> random >> >> >> >> matrix large 2.48172e+16, trying again (this is normal) >> >> >> >> >> >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number >> >> >> >> of >> >> >> >> random >> >> >> >> matrix large 2.1569e+16, trying again (this is normal) >> >> >> >> >> >> >> >> >> >> >> >> On Tue, Dec 2, 2014 at 4:35 PM, Daniel Povey <dp...@gm...> >> >> >> >> wrote: >> >> >> >> >> >> >> >> >> >> >> >> I think I had noticed this before, and I fixed the test by >> >> >> >> changing >> >> >> >> the threshold. >> >> >> >> If you do "svn up" and recompile, it should pass. >> >> >> >> Dan >> >> >> >> >> >> >> >> >> >> >> >> On Tue, Dec 2, 2014 at 4:20 PM, Joaquín Ruales >> >> >> >> <ja...@co...> >> >> >> >> wrote: >> >> >> >> >> >> >> >> Hi Kaldi Team, >> >> >> >> >> >> >> >> I'm running into trouble when running the Kaldi tests (make test) >> >> >> >> described >> >> >> >> in the Kaldi tutorial. I'm running it on a Mac (Mavericks) and >> >> >> >> the >> >> >> >> error >> >> >> >> is >> >> >> >> in matrix-lib-test: KALDI_ASSERT: at >> >> >> >> UnitTestLinearCgd:matrix-lib-test.cc:3118, failed: error < >> >> >> >> 1.0e-05 >> >> >> >> * >> >> >> >> b.Norm(2.0) >> >> >> >> >> >> >> >> Has anyone faced this problem before or have any suggestions? >> >> >> >> >> >> >> >> Thanks, >> >> >> >> Joaquín >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> ------------------------------------------------------------------------------ >> >> >> >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >> >> >> >> from Actuate! Instantly Supercharge Your Business Reports and >> >> >> >> Dashboards >> >> >> >> with Interactivity, Sharing, Native Excel Exports, App >> >> >> >> Integration >> >> >> >> & >> >> >> >> more >> >> >> >> Get technology previously reserved for billion-dollar >> >> >> >> corporations, >> >> >> >> FREE >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >> >> >> >> _______________________________________________ >> >> >> >> Kaldi-developers mailing list >> >> >> >> Kal...@li... >> >> >> >> https://lists.sourceforge.net/lists/listinfo/kaldi-developers >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> > >> >> > >> >> > >> > >> > > > |
|
From: Joaquin A. R. <ja...@co...> - 2014-12-30 04:46:38
|
Hmm, it seems to be another Mac problem: http://serverfault.com/questions/11518/recursive-cp-copies-folder-contents-instead-of-folder-on-os-x I believe that changing for f in phones.txt words.txt phones.txt L.fst L_disambig.fst phones/; do cp -r data/lang/$f $test done for for f in phones.txt words.txt phones.txt L.fst L_disambig.fst phones; do cp -r data/lang/$f $test done should do the trick (if "phones" has a slash after it, Mac computers copy the contents of the folder rather than the folder itself, so this case requires ommiting the /, for OSX). Thanks, Joaquín On Mon, Dec 29, 2014 at 11:30 PM, Daniel Povey <dp...@gm...> wrote: > local/voxforge_format_data.sh creates that directory. See if you can > figure out what happened. > Dan > > > On Mon, Dec 29, 2014 at 8:28 PM, Joaquin Antonio Ruales > <ja...@co...> wrote: > > There is a "data/lang" folder, and there is a "data/lang_test" folder. > The > > "data/lang" folder does have the "phones" subfolder, but the > > "data/lang_test" folder only has the "tmp" folder inside it. All the > files > > that were expected to be inside "phones" are directly inside > > "data/lang_test". > > > > On Mon, Dec 29, 2014 at 11:22 PM, Daniel Povey <dp...@gm...> wrote: > >> > >> That file should be in the phones/ subdirectory. let me look into it. > >> Dan > >> > >> > >> On Mon, Dec 29, 2014 at 8:21 PM, Joaquin Antonio Ruales > >> <ja...@co...> wrote: > >> > Hi Daniel, > >> > > >> > All tests passed as expected. > >> > I am now trying to run the Voxforge recipe, and after train_mono.sh > >> > succeeds, I get the error: > >> > > >> > mkgraph.sh: expected data/lang_test/phones/silence.csl to exist > >> > > >> > I have checked the directories and that file is in fact missing. > >> > However, I > >> > have found a "silence.csl" file as well as other expected files inside > >> > data/lang_test/silence.csl (one level above from the expected > >> > directory). I > >> > have thus modified mkgraph.sh to reference the expected file with the > >> > line: > >> > > >> > required="$lang/L.fst $lang/G.fst $lang/phones.txt $lang/words.txt > >> > $lang/silence.csl $lang/disambig.int $model $tree" > >> > > >> > which now references the file one directory higher. Is this the > correct > >> > thing to do? or am I referencing the wrong file? > >> > > >> > After I made this change I reran ./run.sh, and the recipe didn't throw > >> > the > >> > error mentioned above anymore. However, I obtain more errors later on: > >> > > >> > fstcomposecontext --context-size=1 --central-position=0 > >> > --read-disambig-syms=data/lang_test/phones/disambig.int > >> > --write-disambig-syms=data/lang_test/tmp/disambig_ilabels_1_0.int > >> > data/lang_test/tmp/ilabels_1_0 > >> > > >> > ERROR (fstcomposecontext:main():fstcomposecontext.cc:126) > >> > fstcomposecontext: > >> > Could not read disambiguation symbols from > >> > data/lang_test/phones/disambig.int > >> > > >> > ERROR (fstcomposecontext:main():fstcomposecontext.cc:126) > >> > fstcomposecontext: > >> > Could not read disambiguation symbols from > >> > data/lang_test/phones/disambig.int > >> > > >> > Again, the file data/lang_test/phones/disambig.int does not exist, > but > >> > the > >> > file data/lang_test/disambig.int (one folder above) does exist. Is it > >> > correct to change the referenced disambig.int to be the one on the > >> > parent > >> > folder? > >> > I want to make sure that I am not making changes that don't make sense > >> > with > >> > mkgraph.sh and disambig.int as mentioned above. > >> > > >> > Thanks, > >> > Joaquín > >> > > >> > On Thu, Dec 4, 2014 at 3:05 PM, Daniel Povey <dp...@gm...> > wrote: > >> >> > >> >> I have modified that test program and it should pass now. > >> >> > >> >> We normally don't get so many issues discovered by one person- it > only > >> >> happens when we encounter a new platform that hasn't been properly > >> >> tested before. > >> >> > >> >> Dan > >> >> > >> >> > >> >> On Thu, Dec 4, 2014 at 4:34 AM, Joaquin Antonio Ruales > >> >> <ja...@co...> wrote: > >> >> > Yep! now the matrix test works. Now I'm getting an error on the > >> >> > mle-full-gmm-test. could it be another problem only relevant to OSX > >> >> > Mavericks and Yosemite? Here are the last few lines of output of > >> >> > ./mle-full-gmm-test: > >> >> > > >> >> > Condition number of random matrix large 1.558446e+02, trying again > >> >> > (this > >> >> > is > >> >> > normal) > >> >> > > >> >> > Avg log-like per frame [full-cov, 1-mix] should be: -2.535968e+01 > >> >> > > >> >> > Total log-like [full-cov, 1-mix] should be: -3.550355e+04 > >> >> > > >> >> > Cov eigs are [ 1.639155e+01 1.397001e+01 1.149592e+01 1.090052e+01 > >> >> > 1.049452e+01 8.523492e+00 7.486929e+00 7.382205e+00 6.812490e+00 > >> >> > 5.214448e+00 ] > >> >> > > >> >> > LOG (UnitTestEstimateFullGmm():mle-full-gmm-test.cc:361) Testing > >> >> > natural<>normal conversion > >> >> > > >> >> > KALDI_ASSERT: at UnitTestEstimateFullGmm:mle-full-gmm-test.cc:374, > >> >> > failed: > >> >> > std::abs(gmm->means_invcovars().Row(0)(d) - > >> >> > rgmm.means_invcovars().Row(0)(d)) < prec_v > >> >> > > >> >> > Stack trace is: > >> >> > > >> >> > 0 mle-full-gmm-test 0x000000010b873bb7 > >> >> > _ZN5kaldi18KaldiGetStackTraceEv + 71 > >> >> > > >> >> > 1 mle-full-gmm-test 0x000000010b873f86 > >> >> > _ZN5kaldi19KaldiAssertFailure_EPKcS1_iS1_ + 230 > >> >> > > >> >> > 2 mle-full-gmm-test 0x000000010b78f8c6 > >> >> > _Z23UnitTestEstimateFullGmmv + 4774 > >> >> > > >> >> > 3 mle-full-gmm-test 0x000000010b790a48 main + > 40 > >> >> > > >> >> > 4 libdyld.dylib 0x000000010bb725fd start + > 1 > >> >> > > >> >> > 5 ??? 0x0000000000000001 0x0 + 1 > >> >> > > >> >> > Abort trap: 6 > >> >> > > >> >> > > >> >> > On Wed, Dec 3, 2014 at 8:10 PM, Dogan Can <dog...@us...> > wrote: > >> >> >> > >> >> >> Hi Dan, > >> >> >> > >> >> >> It turns out this was due to a bug in our RandGauss2 function that > >> >> >> accepts > >> >> >> double arguments. I’m committing the following fix to kaldi trunk > >> >> >> right > >> >> >> away. > >> >> >> > >> >> >> Cheers, > >> >> >> Dogan > >> >> >> > >> >> >> diff --git a/src/base/kaldi-math.cc b/src/base/kaldi-math.cc > >> >> >> index 5b30320..0de73bf 100644 > >> >> >> --- a/src/base/kaldi-math.cc > >> >> >> +++ b/src/base/kaldi-math.cc > >> >> >> @@ -147,7 +147,7 @@ void RandGauss2(double *a, double *b, > >> >> >> RandomState > >> >> >> *state) > >> >> >> float a_float, b_float; > >> >> >> // Just because we're using doubles doesn't mean we need > >> >> >> super-high-quality > >> >> >> // random numbers, so we just use the floating-point version > >> >> >> internally. > >> >> >> - RandGauss2(&a_float, &b_float); > >> >> >> + RandGauss2(&a_float, &b_float, state); > >> >> >> *a = a_float; *b = b_float; > >> >> >> } > >> >> >> > >> >> >> On Dec 3, 2014, at 2:23 AM, Dogan Can <dog...@us...> wrote: > >> >> >> > >> >> >> Hi Dan > >> >> >> > >> >> >> I don’t have a Mavericks setup but I can replicate the failure on > >> >> >> Yosemite > >> >> >> with clang. When everything is compiled with gcc, matrix-lib-test > >> >> >> finishes > >> >> >> successfully. > >> >> >> > >> >> >> I looked a bit into the root cause of why the test was going into > an > >> >> >> infinite loop. The random 3x3 matrices generated during the > failing > >> >> >> test end > >> >> >> up having the same first and third columns and fail the condition > >> >> >> number > >> >> >> test, hence the generation loop never terminates. I did some > >> >> >> sleuthing > >> >> >> to > >> >> >> figure out how that was happening and it turns out the value of > >> >> >> rstate.seed > >> >> >> (see the snippet below from matrix/kaldi-matrix.cc) is the same > >> >> >> before > >> >> >> and > >> >> >> after the call to RandGauss2 (note that inner for loop executes > only > >> >> >> once). > >> >> >> It seems like we are hitting a race condition in the system > provided > >> >> >> rand_r > >> >> >> implementation or a compiler bug. > >> >> >> > >> >> >> > >> >> >> 1093 template<typename Real> > >> >> >> 1094 void MatrixBase<Real>::SetRandn() { > >> >> >> 1095 kaldi::RandomState rstate; > >> >> >> 1096 for (MatrixIndexT row = 0; row < num_rows_; row++) { > >> >> >> 1097 Real *row_data = this->RowData(row); > >> >> >> 1098 MatrixIndexT nc = (num_cols_ % 2 == 1) ? num_cols_ - 1 : > >> >> >> num_cols_; > >> >> >> 1099 for (MatrixIndexT col = 0; col < nc; col += 2) { > >> >> >> 1100 kaldi::RandGauss2(row_data + col, row_data + col + 1, > >> >> >> &rstate); > >> >> >> 1101 } > >> >> >> 1102 if (nc != num_cols_) row_data[nc] = > >> >> >> static_cast<Real>(kaldi::RandGauss(&rstate)); > >> >> >> 1103 } > >> >> >> 1104 } > >> >> >> > >> >> >> > >> >> >> Cheers, > >> >> >> Dogan > >> >> >> > >> >> >> On Dec 2, 2014, at 10:13 PM, Daniel Povey <dp...@gm...> > wrote: > >> >> >> > >> >> >> According to > >> >> >> > >> >> >> > >> >> >> > http://stackoverflow.com/questions/19554439/gdb-missing-in-os-x-mavericks > >> >> >> gdb is no longer supported in mavericks and you have to use lldb. > >> >> >> I've > >> >> >> never used that so I don't know what to tell you to do. > >> >> >> Dogan, do you have a mavericks setup, and can you see if you get > the > >> >> >> infinite loop that he gets when you test? > >> >> >> Joaqin: I would just continue through the tutorial for now. > >> >> >> Probably > >> >> >> it's not going to be a problem. > >> >> >> Dan > >> >> >> > >> >> >> > >> >> >> On Wed, Dec 3, 2014 at 1:09 AM, Daniel Povey <dp...@gm...> > >> >> >> wrote: > >> >> >> > >> >> >> Hm. That makes me think there may be a problem with your gcc/gdb > >> >> >> installation- maybe you could try uninstalling it and installing > it > >> >> >> again. Did you get it from MacPorts? > >> >> >> Dan > >> >> >> > >> >> >> > >> >> >> On Wed, Dec 3, 2014 at 1:05 AM, Joaquin Antonio Ruales > >> >> >> <ja...@co...> wrote: > >> >> >> > >> >> >> Now I get: > >> >> >> > >> >> >> > >> >> >> Program received signal SIGINT, Interrupt. > >> >> >> > >> >> >> 0x00007fff8c755e9a in ?? () > >> >> >> > >> >> >> (gdb) b matrix-lib-test.cc:39 > >> >> >> > >> >> >> Cannot access memory at address 0xab670 > >> >> >> > >> >> >> > >> >> >> On Wed, Dec 3, 2014 at 12:57 AM, Daniel Povey <dp...@gm...> > >> >> >> wrote: > >> >> >> > >> >> >> > >> >> >> Probably when you did ctrl-c it was in the Atlas library, > sometimes > >> >> >> it > >> >> >> can't get a proper backtrace. Instead, after doing ctrl-c do: > >> >> >> (gdb) b matrix-lib-test.cc:39 > >> >> >> (gdb) c > >> >> >> > >> >> >> to continue, and when it breaks, show me a backtrace. > >> >> >> Dan > >> >> >> > >> >> >> > >> >> >> On Wed, Dec 3, 2014 at 12:48 AM, Joaquin Antonio Ruales > >> >> >> <ja...@co...> wrote: > >> >> >> > >> >> >> This is what I get: > >> >> >> > >> >> >> > >> >> >> Program received signal SIGINT, Interrupt. > >> >> >> > >> >> >> 0x00007fff8b2c6342 in ?? () > >> >> >> > >> >> >> (gdb) bt > >> >> >> > >> >> >> #0 0x00007fff8b2c6342 in ?? () > >> >> >> > >> >> >> #1 0x00007fff5fbfdbc8 in ?? () > >> >> >> > >> >> >> #2 0x0000000000000001 in ?? () > >> >> >> > >> >> >> #3 0x00007fff5fbfdfdc in ?? () > >> >> >> > >> >> >> #4 0x00007fff5fbfdfe0 in ?? () > >> >> >> > >> >> >> #5 0x00007fff5fbfdbbe in ?? () > >> >> >> > >> >> >> #6 0x0000000100102672 in ?? () > >> >> >> > >> >> >> #7 0x00007fff5fbfdc10 in ?? () > >> >> >> > >> >> >> #8 0x00007fff8b00eac4 in ?? () > >> >> >> > >> >> >> #9 0x00007fff5fbfda90 in ?? () > >> >> >> > >> >> >> #10 0x00007fff8e2bf272 in ?? () > >> >> >> > >> >> >> #11 0x00000002001a0a00 in ?? () > >> >> >> > >> >> >> #12 0x00000001001a0a00 in ?? () > >> >> >> > >> >> >> #13 0x000000010019d000 in ?? () > >> >> >> > >> >> >> #14 0x00000001001a0a00 in ?? () > >> >> >> > >> >> >> #15 0x00000001003002ca in ?? () > >> >> >> > >> >> >> #16 0x0000000000000000 in ?? () > >> >> >> > >> >> >> > >> >> >> On Wed, Dec 3, 2014 at 12:42 AM, Daniel Povey <dp...@gm...> > >> >> >> wrote: > >> >> >> > >> >> >> > >> >> >> That isn't normal and it doesn't happen when I compile on a Mac. > >> >> >> Can you run it in gdb by doing > >> >> >> gdb ./matrix-lib-test > >> >> >> (gdb) r > >> >> >> > >> >> >> and then do ctrl-c when it gets in the loop and type "bt" and show > >> >> >> me > >> >> >> the backtrace? > >> >> >> Dan > >> >> >> > >> >> >> > >> >> >> On Wed, Dec 3, 2014 at 12:36 AM, Joaquin Antonio Ruales > >> >> >> <ja...@co...> wrote: > >> >> >> > >> >> >> Hi Dan, > >> >> >> > >> >> >> Thanks for the quick reply. I have "svn up"ed and recompiled > >> >> >> everything, > >> >> >> but > >> >> >> now the same matrix test runs into an infinite loop. Would it be > >> >> >> safe > >> >> >> to > >> >> >> ignore the test results and continue with the tutorial? Here are > the > >> >> >> few > >> >> >> lines of output from the test: > >> >> >> > >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number > of > >> >> >> random > >> >> >> matrix large 7.50727e+16, trying again (this is normal) > >> >> >> > >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number > of > >> >> >> random > >> >> >> matrix large 1.91265e+16, trying again (this is normal) > >> >> >> > >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number > of > >> >> >> random > >> >> >> matrix large 7.70738e+16, trying again (this is normal) > >> >> >> > >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number > of > >> >> >> random > >> >> >> matrix large 2.32488e+17, trying again (this is normal) > >> >> >> > >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number > of > >> >> >> random > >> >> >> matrix large 8.44565e+17, trying again (this is normal) > >> >> >> > >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number > of > >> >> >> random > >> >> >> matrix large 1.83557e+17, trying again (this is normal) > >> >> >> > >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number > of > >> >> >> random > >> >> >> matrix large 2.61736e+17, trying again (this is normal) > >> >> >> > >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number > of > >> >> >> random > >> >> >> matrix large 8.67803e+16, trying again (this is normal) > >> >> >> > >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number > of > >> >> >> random > >> >> >> matrix large 1.80053e+16, trying again (this is normal) > >> >> >> > >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number > of > >> >> >> random > >> >> >> matrix large 1.67101e+16, trying again (this is normal) > >> >> >> > >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number > of > >> >> >> random > >> >> >> matrix large 1.49485e+16, trying again (this is normal) > >> >> >> > >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number > of > >> >> >> random > >> >> >> matrix large 8.31813e+16, trying again (this is normal) > >> >> >> > >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number > of > >> >> >> random > >> >> >> matrix large 1.6285e+16, trying again (this is normal) > >> >> >> > >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number > of > >> >> >> random > >> >> >> matrix large 2.48172e+16, trying again (this is normal) > >> >> >> > >> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number > of > >> >> >> random > >> >> >> matrix large 2.1569e+16, trying again (this is normal) > >> >> >> > >> >> >> > >> >> >> On Tue, Dec 2, 2014 at 4:35 PM, Daniel Povey <dp...@gm...> > >> >> >> wrote: > >> >> >> > >> >> >> > >> >> >> I think I had noticed this before, and I fixed the test by > changing > >> >> >> the threshold. > >> >> >> If you do "svn up" and recompile, it should pass. > >> >> >> Dan > >> >> >> > >> >> >> > >> >> >> On Tue, Dec 2, 2014 at 4:20 PM, Joaquín Ruales > >> >> >> <ja...@co...> > >> >> >> wrote: > >> >> >> > >> >> >> Hi Kaldi Team, > >> >> >> > >> >> >> I'm running into trouble when running the Kaldi tests (make test) > >> >> >> described > >> >> >> in the Kaldi tutorial. I'm running it on a Mac (Mavericks) and the > >> >> >> error > >> >> >> is > >> >> >> in matrix-lib-test: KALDI_ASSERT: at > >> >> >> UnitTestLinearCgd:matrix-lib-test.cc:3118, failed: error < 1.0e-05 > >> >> >> * > >> >> >> b.Norm(2.0) > >> >> >> > >> >> >> Has anyone faced this problem before or have any suggestions? > >> >> >> > >> >> >> Thanks, > >> >> >> Joaquín > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > ------------------------------------------------------------------------------ > >> >> >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > >> >> >> from Actuate! Instantly Supercharge Your Business Reports and > >> >> >> Dashboards > >> >> >> with Interactivity, Sharing, Native Excel Exports, App Integration > >> >> >> & > >> >> >> more > >> >> >> Get technology previously reserved for billion-dollar > >> >> >> corporations, > >> >> >> FREE > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > >> >> >> _______________________________________________ > >> >> >> Kaldi-developers mailing list > >> >> >> Kal...@li... > >> >> >> https://lists.sourceforge.net/lists/listinfo/kaldi-developers > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> > > >> > > >> > > > > > > |
|
From: Daniel P. <dp...@gm...> - 2014-12-30 04:31:08
|
local/voxforge_format_data.sh creates that directory. See if you can
figure out what happened.
Dan
On Mon, Dec 29, 2014 at 8:28 PM, Joaquin Antonio Ruales
<ja...@co...> wrote:
> There is a "data/lang" folder, and there is a "data/lang_test" folder. The
> "data/lang" folder does have the "phones" subfolder, but the
> "data/lang_test" folder only has the "tmp" folder inside it. All the files
> that were expected to be inside "phones" are directly inside
> "data/lang_test".
>
> On Mon, Dec 29, 2014 at 11:22 PM, Daniel Povey <dp...@gm...> wrote:
>>
>> That file should be in the phones/ subdirectory. let me look into it.
>> Dan
>>
>>
>> On Mon, Dec 29, 2014 at 8:21 PM, Joaquin Antonio Ruales
>> <ja...@co...> wrote:
>> > Hi Daniel,
>> >
>> > All tests passed as expected.
>> > I am now trying to run the Voxforge recipe, and after train_mono.sh
>> > succeeds, I get the error:
>> >
>> > mkgraph.sh: expected data/lang_test/phones/silence.csl to exist
>> >
>> > I have checked the directories and that file is in fact missing.
>> > However, I
>> > have found a "silence.csl" file as well as other expected files inside
>> > data/lang_test/silence.csl (one level above from the expected
>> > directory). I
>> > have thus modified mkgraph.sh to reference the expected file with the
>> > line:
>> >
>> > required="$lang/L.fst $lang/G.fst $lang/phones.txt $lang/words.txt
>> > $lang/silence.csl $lang/disambig.int $model $tree"
>> >
>> > which now references the file one directory higher. Is this the correct
>> > thing to do? or am I referencing the wrong file?
>> >
>> > After I made this change I reran ./run.sh, and the recipe didn't throw
>> > the
>> > error mentioned above anymore. However, I obtain more errors later on:
>> >
>> > fstcomposecontext --context-size=1 --central-position=0
>> > --read-disambig-syms=data/lang_test/phones/disambig.int
>> > --write-disambig-syms=data/lang_test/tmp/disambig_ilabels_1_0.int
>> > data/lang_test/tmp/ilabels_1_0
>> >
>> > ERROR (fstcomposecontext:main():fstcomposecontext.cc:126)
>> > fstcomposecontext:
>> > Could not read disambiguation symbols from
>> > data/lang_test/phones/disambig.int
>> >
>> > ERROR (fstcomposecontext:main():fstcomposecontext.cc:126)
>> > fstcomposecontext:
>> > Could not read disambiguation symbols from
>> > data/lang_test/phones/disambig.int
>> >
>> > Again, the file data/lang_test/phones/disambig.int does not exist, but
>> > the
>> > file data/lang_test/disambig.int (one folder above) does exist. Is it
>> > correct to change the referenced disambig.int to be the one on the
>> > parent
>> > folder?
>> > I want to make sure that I am not making changes that don't make sense
>> > with
>> > mkgraph.sh and disambig.int as mentioned above.
>> >
>> > Thanks,
>> > Joaquín
>> >
>> > On Thu, Dec 4, 2014 at 3:05 PM, Daniel Povey <dp...@gm...> wrote:
>> >>
>> >> I have modified that test program and it should pass now.
>> >>
>> >> We normally don't get so many issues discovered by one person- it only
>> >> happens when we encounter a new platform that hasn't been properly
>> >> tested before.
>> >>
>> >> Dan
>> >>
>> >>
>> >> On Thu, Dec 4, 2014 at 4:34 AM, Joaquin Antonio Ruales
>> >> <ja...@co...> wrote:
>> >> > Yep! now the matrix test works. Now I'm getting an error on the
>> >> > mle-full-gmm-test. could it be another problem only relevant to OSX
>> >> > Mavericks and Yosemite? Here are the last few lines of output of
>> >> > ./mle-full-gmm-test:
>> >> >
>> >> > Condition number of random matrix large 1.558446e+02, trying again
>> >> > (this
>> >> > is
>> >> > normal)
>> >> >
>> >> > Avg log-like per frame [full-cov, 1-mix] should be: -2.535968e+01
>> >> >
>> >> > Total log-like [full-cov, 1-mix] should be: -3.550355e+04
>> >> >
>> >> > Cov eigs are [ 1.639155e+01 1.397001e+01 1.149592e+01 1.090052e+01
>> >> > 1.049452e+01 8.523492e+00 7.486929e+00 7.382205e+00 6.812490e+00
>> >> > 5.214448e+00 ]
>> >> >
>> >> > LOG (UnitTestEstimateFullGmm():mle-full-gmm-test.cc:361) Testing
>> >> > natural<>normal conversion
>> >> >
>> >> > KALDI_ASSERT: at UnitTestEstimateFullGmm:mle-full-gmm-test.cc:374,
>> >> > failed:
>> >> > std::abs(gmm->means_invcovars().Row(0)(d) -
>> >> > rgmm.means_invcovars().Row(0)(d)) < prec_v
>> >> >
>> >> > Stack trace is:
>> >> >
>> >> > 0 mle-full-gmm-test 0x000000010b873bb7
>> >> > _ZN5kaldi18KaldiGetStackTraceEv + 71
>> >> >
>> >> > 1 mle-full-gmm-test 0x000000010b873f86
>> >> > _ZN5kaldi19KaldiAssertFailure_EPKcS1_iS1_ + 230
>> >> >
>> >> > 2 mle-full-gmm-test 0x000000010b78f8c6
>> >> > _Z23UnitTestEstimateFullGmmv + 4774
>> >> >
>> >> > 3 mle-full-gmm-test 0x000000010b790a48 main + 40
>> >> >
>> >> > 4 libdyld.dylib 0x000000010bb725fd start + 1
>> >> >
>> >> > 5 ??? 0x0000000000000001 0x0 + 1
>> >> >
>> >> > Abort trap: 6
>> >> >
>> >> >
>> >> > On Wed, Dec 3, 2014 at 8:10 PM, Dogan Can <dog...@us...> wrote:
>> >> >>
>> >> >> Hi Dan,
>> >> >>
>> >> >> It turns out this was due to a bug in our RandGauss2 function that
>> >> >> accepts
>> >> >> double arguments. I’m committing the following fix to kaldi trunk
>> >> >> right
>> >> >> away.
>> >> >>
>> >> >> Cheers,
>> >> >> Dogan
>> >> >>
>> >> >> diff --git a/src/base/kaldi-math.cc b/src/base/kaldi-math.cc
>> >> >> index 5b30320..0de73bf 100644
>> >> >> --- a/src/base/kaldi-math.cc
>> >> >> +++ b/src/base/kaldi-math.cc
>> >> >> @@ -147,7 +147,7 @@ void RandGauss2(double *a, double *b,
>> >> >> RandomState
>> >> >> *state)
>> >> >> float a_float, b_float;
>> >> >> // Just because we're using doubles doesn't mean we need
>> >> >> super-high-quality
>> >> >> // random numbers, so we just use the floating-point version
>> >> >> internally.
>> >> >> - RandGauss2(&a_float, &b_float);
>> >> >> + RandGauss2(&a_float, &b_float, state);
>> >> >> *a = a_float; *b = b_float;
>> >> >> }
>> >> >>
>> >> >> On Dec 3, 2014, at 2:23 AM, Dogan Can <dog...@us...> wrote:
>> >> >>
>> >> >> Hi Dan
>> >> >>
>> >> >> I don’t have a Mavericks setup but I can replicate the failure on
>> >> >> Yosemite
>> >> >> with clang. When everything is compiled with gcc, matrix-lib-test
>> >> >> finishes
>> >> >> successfully.
>> >> >>
>> >> >> I looked a bit into the root cause of why the test was going into an
>> >> >> infinite loop. The random 3x3 matrices generated during the failing
>> >> >> test end
>> >> >> up having the same first and third columns and fail the condition
>> >> >> number
>> >> >> test, hence the generation loop never terminates. I did some
>> >> >> sleuthing
>> >> >> to
>> >> >> figure out how that was happening and it turns out the value of
>> >> >> rstate.seed
>> >> >> (see the snippet below from matrix/kaldi-matrix.cc) is the same
>> >> >> before
>> >> >> and
>> >> >> after the call to RandGauss2 (note that inner for loop executes only
>> >> >> once).
>> >> >> It seems like we are hitting a race condition in the system provided
>> >> >> rand_r
>> >> >> implementation or a compiler bug.
>> >> >>
>> >> >>
>> >> >> 1093 template<typename Real>
>> >> >> 1094 void MatrixBase<Real>::SetRandn() {
>> >> >> 1095 kaldi::RandomState rstate;
>> >> >> 1096 for (MatrixIndexT row = 0; row < num_rows_; row++) {
>> >> >> 1097 Real *row_data = this->RowData(row);
>> >> >> 1098 MatrixIndexT nc = (num_cols_ % 2 == 1) ? num_cols_ - 1 :
>> >> >> num_cols_;
>> >> >> 1099 for (MatrixIndexT col = 0; col < nc; col += 2) {
>> >> >> 1100 kaldi::RandGauss2(row_data + col, row_data + col + 1,
>> >> >> &rstate);
>> >> >> 1101 }
>> >> >> 1102 if (nc != num_cols_) row_data[nc] =
>> >> >> static_cast<Real>(kaldi::RandGauss(&rstate));
>> >> >> 1103 }
>> >> >> 1104 }
>> >> >>
>> >> >>
>> >> >> Cheers,
>> >> >> Dogan
>> >> >>
>> >> >> On Dec 2, 2014, at 10:13 PM, Daniel Povey <dp...@gm...> wrote:
>> >> >>
>> >> >> According to
>> >> >>
>> >> >>
>> >> >> http://stackoverflow.com/questions/19554439/gdb-missing-in-os-x-mavericks
>> >> >> gdb is no longer supported in mavericks and you have to use lldb.
>> >> >> I've
>> >> >> never used that so I don't know what to tell you to do.
>> >> >> Dogan, do you have a mavericks setup, and can you see if you get the
>> >> >> infinite loop that he gets when you test?
>> >> >> Joaqin: I would just continue through the tutorial for now.
>> >> >> Probably
>> >> >> it's not going to be a problem.
>> >> >> Dan
>> >> >>
>> >> >>
>> >> >> On Wed, Dec 3, 2014 at 1:09 AM, Daniel Povey <dp...@gm...>
>> >> >> wrote:
>> >> >>
>> >> >> Hm. That makes me think there may be a problem with your gcc/gdb
>> >> >> installation- maybe you could try uninstalling it and installing it
>> >> >> again. Did you get it from MacPorts?
>> >> >> Dan
>> >> >>
>> >> >>
>> >> >> On Wed, Dec 3, 2014 at 1:05 AM, Joaquin Antonio Ruales
>> >> >> <ja...@co...> wrote:
>> >> >>
>> >> >> Now I get:
>> >> >>
>> >> >>
>> >> >> Program received signal SIGINT, Interrupt.
>> >> >>
>> >> >> 0x00007fff8c755e9a in ?? ()
>> >> >>
>> >> >> (gdb) b matrix-lib-test.cc:39
>> >> >>
>> >> >> Cannot access memory at address 0xab670
>> >> >>
>> >> >>
>> >> >> On Wed, Dec 3, 2014 at 12:57 AM, Daniel Povey <dp...@gm...>
>> >> >> wrote:
>> >> >>
>> >> >>
>> >> >> Probably when you did ctrl-c it was in the Atlas library, sometimes
>> >> >> it
>> >> >> can't get a proper backtrace. Instead, after doing ctrl-c do:
>> >> >> (gdb) b matrix-lib-test.cc:39
>> >> >> (gdb) c
>> >> >>
>> >> >> to continue, and when it breaks, show me a backtrace.
>> >> >> Dan
>> >> >>
>> >> >>
>> >> >> On Wed, Dec 3, 2014 at 12:48 AM, Joaquin Antonio Ruales
>> >> >> <ja...@co...> wrote:
>> >> >>
>> >> >> This is what I get:
>> >> >>
>> >> >>
>> >> >> Program received signal SIGINT, Interrupt.
>> >> >>
>> >> >> 0x00007fff8b2c6342 in ?? ()
>> >> >>
>> >> >> (gdb) bt
>> >> >>
>> >> >> #0 0x00007fff8b2c6342 in ?? ()
>> >> >>
>> >> >> #1 0x00007fff5fbfdbc8 in ?? ()
>> >> >>
>> >> >> #2 0x0000000000000001 in ?? ()
>> >> >>
>> >> >> #3 0x00007fff5fbfdfdc in ?? ()
>> >> >>
>> >> >> #4 0x00007fff5fbfdfe0 in ?? ()
>> >> >>
>> >> >> #5 0x00007fff5fbfdbbe in ?? ()
>> >> >>
>> >> >> #6 0x0000000100102672 in ?? ()
>> >> >>
>> >> >> #7 0x00007fff5fbfdc10 in ?? ()
>> >> >>
>> >> >> #8 0x00007fff8b00eac4 in ?? ()
>> >> >>
>> >> >> #9 0x00007fff5fbfda90 in ?? ()
>> >> >>
>> >> >> #10 0x00007fff8e2bf272 in ?? ()
>> >> >>
>> >> >> #11 0x00000002001a0a00 in ?? ()
>> >> >>
>> >> >> #12 0x00000001001a0a00 in ?? ()
>> >> >>
>> >> >> #13 0x000000010019d000 in ?? ()
>> >> >>
>> >> >> #14 0x00000001001a0a00 in ?? ()
>> >> >>
>> >> >> #15 0x00000001003002ca in ?? ()
>> >> >>
>> >> >> #16 0x0000000000000000 in ?? ()
>> >> >>
>> >> >>
>> >> >> On Wed, Dec 3, 2014 at 12:42 AM, Daniel Povey <dp...@gm...>
>> >> >> wrote:
>> >> >>
>> >> >>
>> >> >> That isn't normal and it doesn't happen when I compile on a Mac.
>> >> >> Can you run it in gdb by doing
>> >> >> gdb ./matrix-lib-test
>> >> >> (gdb) r
>> >> >>
>> >> >> and then do ctrl-c when it gets in the loop and type "bt" and show
>> >> >> me
>> >> >> the backtrace?
>> >> >> Dan
>> >> >>
>> >> >>
>> >> >> On Wed, Dec 3, 2014 at 12:36 AM, Joaquin Antonio Ruales
>> >> >> <ja...@co...> wrote:
>> >> >>
>> >> >> Hi Dan,
>> >> >>
>> >> >> Thanks for the quick reply. I have "svn up"ed and recompiled
>> >> >> everything,
>> >> >> but
>> >> >> now the same matrix test runs into an infinite loop. Would it be
>> >> >> safe
>> >> >> to
>> >> >> ignore the test results and continue with the tutorial? Here are the
>> >> >> few
>> >> >> lines of output from the test:
>> >> >>
>> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
>> >> >> random
>> >> >> matrix large 7.50727e+16, trying again (this is normal)
>> >> >>
>> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
>> >> >> random
>> >> >> matrix large 1.91265e+16, trying again (this is normal)
>> >> >>
>> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
>> >> >> random
>> >> >> matrix large 7.70738e+16, trying again (this is normal)
>> >> >>
>> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
>> >> >> random
>> >> >> matrix large 2.32488e+17, trying again (this is normal)
>> >> >>
>> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
>> >> >> random
>> >> >> matrix large 8.44565e+17, trying again (this is normal)
>> >> >>
>> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
>> >> >> random
>> >> >> matrix large 1.83557e+17, trying again (this is normal)
>> >> >>
>> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
>> >> >> random
>> >> >> matrix large 2.61736e+17, trying again (this is normal)
>> >> >>
>> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
>> >> >> random
>> >> >> matrix large 8.67803e+16, trying again (this is normal)
>> >> >>
>> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
>> >> >> random
>> >> >> matrix large 1.80053e+16, trying again (this is normal)
>> >> >>
>> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
>> >> >> random
>> >> >> matrix large 1.67101e+16, trying again (this is normal)
>> >> >>
>> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
>> >> >> random
>> >> >> matrix large 1.49485e+16, trying again (this is normal)
>> >> >>
>> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
>> >> >> random
>> >> >> matrix large 8.31813e+16, trying again (this is normal)
>> >> >>
>> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
>> >> >> random
>> >> >> matrix large 1.6285e+16, trying again (this is normal)
>> >> >>
>> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
>> >> >> random
>> >> >> matrix large 2.48172e+16, trying again (this is normal)
>> >> >>
>> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
>> >> >> random
>> >> >> matrix large 2.1569e+16, trying again (this is normal)
>> >> >>
>> >> >>
>> >> >> On Tue, Dec 2, 2014 at 4:35 PM, Daniel Povey <dp...@gm...>
>> >> >> wrote:
>> >> >>
>> >> >>
>> >> >> I think I had noticed this before, and I fixed the test by changing
>> >> >> the threshold.
>> >> >> If you do "svn up" and recompile, it should pass.
>> >> >> Dan
>> >> >>
>> >> >>
>> >> >> On Tue, Dec 2, 2014 at 4:20 PM, Joaquín Ruales
>> >> >> <ja...@co...>
>> >> >> wrote:
>> >> >>
>> >> >> Hi Kaldi Team,
>> >> >>
>> >> >> I'm running into trouble when running the Kaldi tests (make test)
>> >> >> described
>> >> >> in the Kaldi tutorial. I'm running it on a Mac (Mavericks) and the
>> >> >> error
>> >> >> is
>> >> >> in matrix-lib-test: KALDI_ASSERT: at
>> >> >> UnitTestLinearCgd:matrix-lib-test.cc:3118, failed: error < 1.0e-05
>> >> >> *
>> >> >> b.Norm(2.0)
>> >> >>
>> >> >> Has anyone faced this problem before or have any suggestions?
>> >> >>
>> >> >> Thanks,
>> >> >> Joaquín
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> ------------------------------------------------------------------------------
>> >> >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
>> >> >> from Actuate! Instantly Supercharge Your Business Reports and
>> >> >> Dashboards
>> >> >> with Interactivity, Sharing, Native Excel Exports, App Integration
>> >> >> &
>> >> >> more
>> >> >> Get technology previously reserved for billion-dollar
>> >> >> corporations,
>> >> >> FREE
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
>> >> >> _______________________________________________
>> >> >> Kaldi-developers mailing list
>> >> >> Kal...@li...
>> >> >> https://lists.sourceforge.net/lists/listinfo/kaldi-developers
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >
>> >
>> >
>
>
|
|
From: Joaquin A. R. <ja...@co...> - 2014-12-30 04:29:03
|
There is a "data/lang" folder, and there is a "data/lang_test" folder. The
"data/lang" folder does have the "phones" subfolder, but the
"data/lang_test" folder only has the "tmp" folder inside it. All the files
that were expected to be inside "phones" are directly inside
"data/lang_test".
On Mon, Dec 29, 2014 at 11:22 PM, Daniel Povey <dp...@gm...> wrote:
> That file should be in the phones/ subdirectory. let me look into it.
> Dan
>
>
> On Mon, Dec 29, 2014 at 8:21 PM, Joaquin Antonio Ruales
> <ja...@co...> wrote:
> > Hi Daniel,
> >
> > All tests passed as expected.
> > I am now trying to run the Voxforge recipe, and after train_mono.sh
> > succeeds, I get the error:
> >
> > mkgraph.sh: expected data/lang_test/phones/silence.csl to exist
> >
> > I have checked the directories and that file is in fact missing.
> However, I
> > have found a "silence.csl" file as well as other expected files inside
> > data/lang_test/silence.csl (one level above from the expected
> directory). I
> > have thus modified mkgraph.sh to reference the expected file with the
> line:
> >
> > required="$lang/L.fst $lang/G.fst $lang/phones.txt $lang/words.txt
> > $lang/silence.csl $lang/disambig.int $model $tree"
> >
> > which now references the file one directory higher. Is this the correct
> > thing to do? or am I referencing the wrong file?
> >
> > After I made this change I reran ./run.sh, and the recipe didn't throw
> the
> > error mentioned above anymore. However, I obtain more errors later on:
> >
> > fstcomposecontext --context-size=1 --central-position=0
> > --read-disambig-syms=data/lang_test/phones/disambig.int
> > --write-disambig-syms=data/lang_test/tmp/disambig_ilabels_1_0.int
> > data/lang_test/tmp/ilabels_1_0
> >
> > ERROR (fstcomposecontext:main():fstcomposecontext.cc:126)
> fstcomposecontext:
> > Could not read disambiguation symbols from
> > data/lang_test/phones/disambig.int
> >
> > ERROR (fstcomposecontext:main():fstcomposecontext.cc:126)
> fstcomposecontext:
> > Could not read disambiguation symbols from
> > data/lang_test/phones/disambig.int
> >
> > Again, the file data/lang_test/phones/disambig.int does not exist, but
> the
> > file data/lang_test/disambig.int (one folder above) does exist. Is it
> > correct to change the referenced disambig.int to be the one on the
> parent
> > folder?
> > I want to make sure that I am not making changes that don't make sense
> with
> > mkgraph.sh and disambig.int as mentioned above.
> >
> > Thanks,
> > Joaquín
> >
> > On Thu, Dec 4, 2014 at 3:05 PM, Daniel Povey <dp...@gm...> wrote:
> >>
> >> I have modified that test program and it should pass now.
> >>
> >> We normally don't get so many issues discovered by one person- it only
> >> happens when we encounter a new platform that hasn't been properly
> >> tested before.
> >>
> >> Dan
> >>
> >>
> >> On Thu, Dec 4, 2014 at 4:34 AM, Joaquin Antonio Ruales
> >> <ja...@co...> wrote:
> >> > Yep! now the matrix test works. Now I'm getting an error on the
> >> > mle-full-gmm-test. could it be another problem only relevant to OSX
> >> > Mavericks and Yosemite? Here are the last few lines of output of
> >> > ./mle-full-gmm-test:
> >> >
> >> > Condition number of random matrix large 1.558446e+02, trying again
> (this
> >> > is
> >> > normal)
> >> >
> >> > Avg log-like per frame [full-cov, 1-mix] should be: -2.535968e+01
> >> >
> >> > Total log-like [full-cov, 1-mix] should be: -3.550355e+04
> >> >
> >> > Cov eigs are [ 1.639155e+01 1.397001e+01 1.149592e+01 1.090052e+01
> >> > 1.049452e+01 8.523492e+00 7.486929e+00 7.382205e+00 6.812490e+00
> >> > 5.214448e+00 ]
> >> >
> >> > LOG (UnitTestEstimateFullGmm():mle-full-gmm-test.cc:361) Testing
> >> > natural<>normal conversion
> >> >
> >> > KALDI_ASSERT: at UnitTestEstimateFullGmm:mle-full-gmm-test.cc:374,
> >> > failed:
> >> > std::abs(gmm->means_invcovars().Row(0)(d) -
> >> > rgmm.means_invcovars().Row(0)(d)) < prec_v
> >> >
> >> > Stack trace is:
> >> >
> >> > 0 mle-full-gmm-test 0x000000010b873bb7
> >> > _ZN5kaldi18KaldiGetStackTraceEv + 71
> >> >
> >> > 1 mle-full-gmm-test 0x000000010b873f86
> >> > _ZN5kaldi19KaldiAssertFailure_EPKcS1_iS1_ + 230
> >> >
> >> > 2 mle-full-gmm-test 0x000000010b78f8c6
> >> > _Z23UnitTestEstimateFullGmmv + 4774
> >> >
> >> > 3 mle-full-gmm-test 0x000000010b790a48 main + 40
> >> >
> >> > 4 libdyld.dylib 0x000000010bb725fd start + 1
> >> >
> >> > 5 ??? 0x0000000000000001 0x0 + 1
> >> >
> >> > Abort trap: 6
> >> >
> >> >
> >> > On Wed, Dec 3, 2014 at 8:10 PM, Dogan Can <dog...@us...> wrote:
> >> >>
> >> >> Hi Dan,
> >> >>
> >> >> It turns out this was due to a bug in our RandGauss2 function that
> >> >> accepts
> >> >> double arguments. I’m committing the following fix to kaldi trunk
> right
> >> >> away.
> >> >>
> >> >> Cheers,
> >> >> Dogan
> >> >>
> >> >> diff --git a/src/base/kaldi-math.cc b/src/base/kaldi-math.cc
> >> >> index 5b30320..0de73bf 100644
> >> >> --- a/src/base/kaldi-math.cc
> >> >> +++ b/src/base/kaldi-math.cc
> >> >> @@ -147,7 +147,7 @@ void RandGauss2(double *a, double *b, RandomState
> >> >> *state)
> >> >> float a_float, b_float;
> >> >> // Just because we're using doubles doesn't mean we need
> >> >> super-high-quality
> >> >> // random numbers, so we just use the floating-point version
> >> >> internally.
> >> >> - RandGauss2(&a_float, &b_float);
> >> >> + RandGauss2(&a_float, &b_float, state);
> >> >> *a = a_float; *b = b_float;
> >> >> }
> >> >>
> >> >> On Dec 3, 2014, at 2:23 AM, Dogan Can <dog...@us...> wrote:
> >> >>
> >> >> Hi Dan
> >> >>
> >> >> I don’t have a Mavericks setup but I can replicate the failure on
> >> >> Yosemite
> >> >> with clang. When everything is compiled with gcc, matrix-lib-test
> >> >> finishes
> >> >> successfully.
> >> >>
> >> >> I looked a bit into the root cause of why the test was going into an
> >> >> infinite loop. The random 3x3 matrices generated during the failing
> >> >> test end
> >> >> up having the same first and third columns and fail the condition
> >> >> number
> >> >> test, hence the generation loop never terminates. I did some
> sleuthing
> >> >> to
> >> >> figure out how that was happening and it turns out the value of
> >> >> rstate.seed
> >> >> (see the snippet below from matrix/kaldi-matrix.cc) is the same
> before
> >> >> and
> >> >> after the call to RandGauss2 (note that inner for loop executes only
> >> >> once).
> >> >> It seems like we are hitting a race condition in the system provided
> >> >> rand_r
> >> >> implementation or a compiler bug.
> >> >>
> >> >>
> >> >> 1093 template<typename Real>
> >> >> 1094 void MatrixBase<Real>::SetRandn() {
> >> >> 1095 kaldi::RandomState rstate;
> >> >> 1096 for (MatrixIndexT row = 0; row < num_rows_; row++) {
> >> >> 1097 Real *row_data = this->RowData(row);
> >> >> 1098 MatrixIndexT nc = (num_cols_ % 2 == 1) ? num_cols_ - 1 :
> >> >> num_cols_;
> >> >> 1099 for (MatrixIndexT col = 0; col < nc; col += 2) {
> >> >> 1100 kaldi::RandGauss2(row_data + col, row_data + col + 1,
> >> >> &rstate);
> >> >> 1101 }
> >> >> 1102 if (nc != num_cols_) row_data[nc] =
> >> >> static_cast<Real>(kaldi::RandGauss(&rstate));
> >> >> 1103 }
> >> >> 1104 }
> >> >>
> >> >>
> >> >> Cheers,
> >> >> Dogan
> >> >>
> >> >> On Dec 2, 2014, at 10:13 PM, Daniel Povey <dp...@gm...> wrote:
> >> >>
> >> >> According to
> >> >>
> >> >>
> http://stackoverflow.com/questions/19554439/gdb-missing-in-os-x-mavericks
> >> >> gdb is no longer supported in mavericks and you have to use lldb.
> I've
> >> >> never used that so I don't know what to tell you to do.
> >> >> Dogan, do you have a mavericks setup, and can you see if you get the
> >> >> infinite loop that he gets when you test?
> >> >> Joaqin: I would just continue through the tutorial for now. Probably
> >> >> it's not going to be a problem.
> >> >> Dan
> >> >>
> >> >>
> >> >> On Wed, Dec 3, 2014 at 1:09 AM, Daniel Povey <dp...@gm...>
> wrote:
> >> >>
> >> >> Hm. That makes me think there may be a problem with your gcc/gdb
> >> >> installation- maybe you could try uninstalling it and installing it
> >> >> again. Did you get it from MacPorts?
> >> >> Dan
> >> >>
> >> >>
> >> >> On Wed, Dec 3, 2014 at 1:05 AM, Joaquin Antonio Ruales
> >> >> <ja...@co...> wrote:
> >> >>
> >> >> Now I get:
> >> >>
> >> >>
> >> >> Program received signal SIGINT, Interrupt.
> >> >>
> >> >> 0x00007fff8c755e9a in ?? ()
> >> >>
> >> >> (gdb) b matrix-lib-test.cc:39
> >> >>
> >> >> Cannot access memory at address 0xab670
> >> >>
> >> >>
> >> >> On Wed, Dec 3, 2014 at 12:57 AM, Daniel Povey <dp...@gm...>
> wrote:
> >> >>
> >> >>
> >> >> Probably when you did ctrl-c it was in the Atlas library, sometimes
> it
> >> >> can't get a proper backtrace. Instead, after doing ctrl-c do:
> >> >> (gdb) b matrix-lib-test.cc:39
> >> >> (gdb) c
> >> >>
> >> >> to continue, and when it breaks, show me a backtrace.
> >> >> Dan
> >> >>
> >> >>
> >> >> On Wed, Dec 3, 2014 at 12:48 AM, Joaquin Antonio Ruales
> >> >> <ja...@co...> wrote:
> >> >>
> >> >> This is what I get:
> >> >>
> >> >>
> >> >> Program received signal SIGINT, Interrupt.
> >> >>
> >> >> 0x00007fff8b2c6342 in ?? ()
> >> >>
> >> >> (gdb) bt
> >> >>
> >> >> #0 0x00007fff8b2c6342 in ?? ()
> >> >>
> >> >> #1 0x00007fff5fbfdbc8 in ?? ()
> >> >>
> >> >> #2 0x0000000000000001 in ?? ()
> >> >>
> >> >> #3 0x00007fff5fbfdfdc in ?? ()
> >> >>
> >> >> #4 0x00007fff5fbfdfe0 in ?? ()
> >> >>
> >> >> #5 0x00007fff5fbfdbbe in ?? ()
> >> >>
> >> >> #6 0x0000000100102672 in ?? ()
> >> >>
> >> >> #7 0x00007fff5fbfdc10 in ?? ()
> >> >>
> >> >> #8 0x00007fff8b00eac4 in ?? ()
> >> >>
> >> >> #9 0x00007fff5fbfda90 in ?? ()
> >> >>
> >> >> #10 0x00007fff8e2bf272 in ?? ()
> >> >>
> >> >> #11 0x00000002001a0a00 in ?? ()
> >> >>
> >> >> #12 0x00000001001a0a00 in ?? ()
> >> >>
> >> >> #13 0x000000010019d000 in ?? ()
> >> >>
> >> >> #14 0x00000001001a0a00 in ?? ()
> >> >>
> >> >> #15 0x00000001003002ca in ?? ()
> >> >>
> >> >> #16 0x0000000000000000 in ?? ()
> >> >>
> >> >>
> >> >> On Wed, Dec 3, 2014 at 12:42 AM, Daniel Povey <dp...@gm...>
> wrote:
> >> >>
> >> >>
> >> >> That isn't normal and it doesn't happen when I compile on a Mac.
> >> >> Can you run it in gdb by doing
> >> >> gdb ./matrix-lib-test
> >> >> (gdb) r
> >> >>
> >> >> and then do ctrl-c when it gets in the loop and type "bt" and show me
> >> >> the backtrace?
> >> >> Dan
> >> >>
> >> >>
> >> >> On Wed, Dec 3, 2014 at 12:36 AM, Joaquin Antonio Ruales
> >> >> <ja...@co...> wrote:
> >> >>
> >> >> Hi Dan,
> >> >>
> >> >> Thanks for the quick reply. I have "svn up"ed and recompiled
> >> >> everything,
> >> >> but
> >> >> now the same matrix test runs into an infinite loop. Would it be safe
> >> >> to
> >> >> ignore the test results and continue with the tutorial? Here are the
> >> >> few
> >> >> lines of output from the test:
> >> >>
> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
> >> >> random
> >> >> matrix large 7.50727e+16, trying again (this is normal)
> >> >>
> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
> >> >> random
> >> >> matrix large 1.91265e+16, trying again (this is normal)
> >> >>
> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
> >> >> random
> >> >> matrix large 7.70738e+16, trying again (this is normal)
> >> >>
> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
> >> >> random
> >> >> matrix large 2.32488e+17, trying again (this is normal)
> >> >>
> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
> >> >> random
> >> >> matrix large 8.44565e+17, trying again (this is normal)
> >> >>
> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
> >> >> random
> >> >> matrix large 1.83557e+17, trying again (this is normal)
> >> >>
> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
> >> >> random
> >> >> matrix large 2.61736e+17, trying again (this is normal)
> >> >>
> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
> >> >> random
> >> >> matrix large 8.67803e+16, trying again (this is normal)
> >> >>
> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
> >> >> random
> >> >> matrix large 1.80053e+16, trying again (this is normal)
> >> >>
> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
> >> >> random
> >> >> matrix large 1.67101e+16, trying again (this is normal)
> >> >>
> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
> >> >> random
> >> >> matrix large 1.49485e+16, trying again (this is normal)
> >> >>
> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
> >> >> random
> >> >> matrix large 8.31813e+16, trying again (this is normal)
> >> >>
> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
> >> >> random
> >> >> matrix large 1.6285e+16, trying again (this is normal)
> >> >>
> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
> >> >> random
> >> >> matrix large 2.48172e+16, trying again (this is normal)
> >> >>
> >> >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of
> >> >> random
> >> >> matrix large 2.1569e+16, trying again (this is normal)
> >> >>
> >> >>
> >> >> On Tue, Dec 2, 2014 at 4:35 PM, Daniel Povey <dp...@gm...>
> >> >> wrote:
> >> >>
> >> >>
> >> >> I think I had noticed this before, and I fixed the test by changing
> >> >> the threshold.
> >> >> If you do "svn up" and recompile, it should pass.
> >> >> Dan
> >> >>
> >> >>
> >> >> On Tue, Dec 2, 2014 at 4:20 PM, Joaquín Ruales
> >> >> <ja...@co...>
> >> >> wrote:
> >> >>
> >> >> Hi Kaldi Team,
> >> >>
> >> >> I'm running into trouble when running the Kaldi tests (make test)
> >> >> described
> >> >> in the Kaldi tutorial. I'm running it on a Mac (Mavericks) and the
> >> >> error
> >> >> is
> >> >> in matrix-lib-test: KALDI_ASSERT: at
> >> >> UnitTestLinearCgd:matrix-lib-test.cc:3118, failed: error < 1.0e-05
> >> >> *
> >> >> b.Norm(2.0)
> >> >>
> >> >> Has anyone faced this problem before or have any suggestions?
> >> >>
> >> >> Thanks,
> >> >> Joaquín
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> ------------------------------------------------------------------------------
> >> >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> >> >> from Actuate! Instantly Supercharge Your Business Reports and
> >> >> Dashboards
> >> >> with Interactivity, Sharing, Native Excel Exports, App Integration
> >> >> &
> >> >> more
> >> >> Get technology previously reserved for billion-dollar
> >> >> corporations,
> >> >> FREE
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
> >> >> _______________________________________________
> >> >> Kaldi-developers mailing list
> >> >> Kal...@li...
> >> >> https://lists.sourceforge.net/lists/listinfo/kaldi-developers
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >
> >
> >
>
|
|
From: Joaquin A. R. <ja...@co...> - 2014-12-30 04:22:51
|
Hi Daniel, All tests passed as expected. I am now trying to run the Voxforge recipe, and after train_mono.sh succeeds, I get the error: *mkgraph.sh: expected data/lang_test/phones/silence.csl to exist* I have checked the directories and that file is in fact missing. However, I have found a "silence.csl" file as well as other expected files inside data/lang_test/silence.csl (one level above from the expected directory). I have thus modified mkgraph.sh to reference the expected file with the line: *required="$lang/L.fst $lang/G.fst $lang/phones.txt $lang/words.txt $lang/silence.csl $lang/disambig.int <http://disambig.int/> $model $tree"* which now references the file one directory higher. Is this the correct thing to do? or am I referencing the wrong file? After I made this change I reran ./run.sh, and the recipe didn't throw the error mentioned above anymore. However, I obtain more errors later on: *fstcomposecontext --context-size=1 --central-position=0 --read-disambig-syms=data/lang_test/phones/disambig.int <http://disambig.int/> --write-disambig-syms=data/lang_test/tmp/disambig_ilabels_1_0.int <http://disambig_ilabels_1_0.int/> data/lang_test/tmp/ilabels_1_0 * *ERROR (fstcomposecontext:main():fstcomposecontext.cc:126) fstcomposecontext: Could not read disambiguation symbols from data/lang_test/phones/disambig.int <http://disambig.int/>* *ERROR (fstcomposecontext:main():fstcomposecontext.cc:126) fstcomposecontext: Could not read disambiguation symbols from data/lang_test/phones/disambig.int <http://disambig.int/>* Again, the file data/lang_test/phones/disambig.int does not exist, but the file data/lang_test/disambig.int (one folder above) does exist. Is it correct to change the referenced disambig.int to be the one on the parent folder? I want to make sure that I am not making changes that don't make sense with mkgraph.sh and disambig.int as mentioned above. Thanks, Joaquín On Thu, Dec 4, 2014 at 3:05 PM, Daniel Povey <dp...@gm...> wrote: > I have modified that test program and it should pass now. > > We normally don't get so many issues discovered by one person- it only > happens when we encounter a new platform that hasn't been properly > tested before. > > Dan > > > On Thu, Dec 4, 2014 at 4:34 AM, Joaquin Antonio Ruales > <ja...@co...> wrote: > > Yep! now the matrix test works. Now I'm getting an error on the > > mle-full-gmm-test. could it be another problem only relevant to OSX > > Mavericks and Yosemite? Here are the last few lines of output of > > ./mle-full-gmm-test: > > > > Condition number of random matrix large 1.558446e+02, trying again (this > is > > normal) > > > > Avg log-like per frame [full-cov, 1-mix] should be: -2.535968e+01 > > > > Total log-like [full-cov, 1-mix] should be: -3.550355e+04 > > > > Cov eigs are [ 1.639155e+01 1.397001e+01 1.149592e+01 1.090052e+01 > > 1.049452e+01 8.523492e+00 7.486929e+00 7.382205e+00 6.812490e+00 > > 5.214448e+00 ] > > > > LOG (UnitTestEstimateFullGmm():mle-full-gmm-test.cc:361) Testing > > natural<>normal conversion > > > > KALDI_ASSERT: at UnitTestEstimateFullGmm:mle-full-gmm-test.cc:374, > failed: > > std::abs(gmm->means_invcovars().Row(0)(d) - > > rgmm.means_invcovars().Row(0)(d)) < prec_v > > > > Stack trace is: > > > > 0 mle-full-gmm-test 0x000000010b873bb7 > > _ZN5kaldi18KaldiGetStackTraceEv + 71 > > > > 1 mle-full-gmm-test 0x000000010b873f86 > > _ZN5kaldi19KaldiAssertFailure_EPKcS1_iS1_ + 230 > > > > 2 mle-full-gmm-test 0x000000010b78f8c6 > > _Z23UnitTestEstimateFullGmmv + 4774 > > > > 3 mle-full-gmm-test 0x000000010b790a48 main + 40 > > > > 4 libdyld.dylib 0x000000010bb725fd start + 1 > > > > 5 ??? 0x0000000000000001 0x0 + 1 > > > > Abort trap: 6 > > > > > > On Wed, Dec 3, 2014 at 8:10 PM, Dogan Can <dog...@us...> wrote: > >> > >> Hi Dan, > >> > >> It turns out this was due to a bug in our RandGauss2 function that > accepts > >> double arguments. I’m committing the following fix to kaldi trunk right > >> away. > >> > >> Cheers, > >> Dogan > >> > >> diff --git a/src/base/kaldi-math.cc b/src/base/kaldi-math.cc > >> index 5b30320..0de73bf 100644 > >> --- a/src/base/kaldi-math.cc > >> +++ b/src/base/kaldi-math.cc > >> @@ -147,7 +147,7 @@ void RandGauss2(double *a, double *b, RandomState > >> *state) > >> float a_float, b_float; > >> // Just because we're using doubles doesn't mean we need > >> super-high-quality > >> // random numbers, so we just use the floating-point version > >> internally. > >> - RandGauss2(&a_float, &b_float); > >> + RandGauss2(&a_float, &b_float, state); > >> *a = a_float; *b = b_float; > >> } > >> > >> On Dec 3, 2014, at 2:23 AM, Dogan Can <dog...@us...> wrote: > >> > >> Hi Dan > >> > >> I don’t have a Mavericks setup but I can replicate the failure on > Yosemite > >> with clang. When everything is compiled with gcc, matrix-lib-test > finishes > >> successfully. > >> > >> I looked a bit into the root cause of why the test was going into an > >> infinite loop. The random 3x3 matrices generated during the failing > test end > >> up having the same first and third columns and fail the condition number > >> test, hence the generation loop never terminates. I did some sleuthing > to > >> figure out how that was happening and it turns out the value of > rstate.seed > >> (see the snippet below from matrix/kaldi-matrix.cc) is the same before > and > >> after the call to RandGauss2 (note that inner for loop executes only > once). > >> It seems like we are hitting a race condition in the system provided > rand_r > >> implementation or a compiler bug. > >> > >> > >> 1093 template<typename Real> > >> 1094 void MatrixBase<Real>::SetRandn() { > >> 1095 kaldi::RandomState rstate; > >> 1096 for (MatrixIndexT row = 0; row < num_rows_; row++) { > >> 1097 Real *row_data = this->RowData(row); > >> 1098 MatrixIndexT nc = (num_cols_ % 2 == 1) ? num_cols_ - 1 : > >> num_cols_; > >> 1099 for (MatrixIndexT col = 0; col < nc; col += 2) { > >> 1100 kaldi::RandGauss2(row_data + col, row_data + col + 1, > &rstate); > >> 1101 } > >> 1102 if (nc != num_cols_) row_data[nc] = > >> static_cast<Real>(kaldi::RandGauss(&rstate)); > >> 1103 } > >> 1104 } > >> > >> > >> Cheers, > >> Dogan > >> > >> On Dec 2, 2014, at 10:13 PM, Daniel Povey <dp...@gm...> wrote: > >> > >> According to > >> > http://stackoverflow.com/questions/19554439/gdb-missing-in-os-x-mavericks > >> gdb is no longer supported in mavericks and you have to use lldb. I've > >> never used that so I don't know what to tell you to do. > >> Dogan, do you have a mavericks setup, and can you see if you get the > >> infinite loop that he gets when you test? > >> Joaqin: I would just continue through the tutorial for now. Probably > >> it's not going to be a problem. > >> Dan > >> > >> > >> On Wed, Dec 3, 2014 at 1:09 AM, Daniel Povey <dp...@gm...> wrote: > >> > >> Hm. That makes me think there may be a problem with your gcc/gdb > >> installation- maybe you could try uninstalling it and installing it > >> again. Did you get it from MacPorts? > >> Dan > >> > >> > >> On Wed, Dec 3, 2014 at 1:05 AM, Joaquin Antonio Ruales > >> <ja...@co...> wrote: > >> > >> Now I get: > >> > >> > >> Program received signal SIGINT, Interrupt. > >> > >> 0x00007fff8c755e9a in ?? () > >> > >> (gdb) b matrix-lib-test.cc:39 > >> > >> Cannot access memory at address 0xab670 > >> > >> > >> On Wed, Dec 3, 2014 at 12:57 AM, Daniel Povey <dp...@gm...> wrote: > >> > >> > >> Probably when you did ctrl-c it was in the Atlas library, sometimes it > >> can't get a proper backtrace. Instead, after doing ctrl-c do: > >> (gdb) b matrix-lib-test.cc:39 > >> (gdb) c > >> > >> to continue, and when it breaks, show me a backtrace. > >> Dan > >> > >> > >> On Wed, Dec 3, 2014 at 12:48 AM, Joaquin Antonio Ruales > >> <ja...@co...> wrote: > >> > >> This is what I get: > >> > >> > >> Program received signal SIGINT, Interrupt. > >> > >> 0x00007fff8b2c6342 in ?? () > >> > >> (gdb) bt > >> > >> #0 0x00007fff8b2c6342 in ?? () > >> > >> #1 0x00007fff5fbfdbc8 in ?? () > >> > >> #2 0x0000000000000001 in ?? () > >> > >> #3 0x00007fff5fbfdfdc in ?? () > >> > >> #4 0x00007fff5fbfdfe0 in ?? () > >> > >> #5 0x00007fff5fbfdbbe in ?? () > >> > >> #6 0x0000000100102672 in ?? () > >> > >> #7 0x00007fff5fbfdc10 in ?? () > >> > >> #8 0x00007fff8b00eac4 in ?? () > >> > >> #9 0x00007fff5fbfda90 in ?? () > >> > >> #10 0x00007fff8e2bf272 in ?? () > >> > >> #11 0x00000002001a0a00 in ?? () > >> > >> #12 0x00000001001a0a00 in ?? () > >> > >> #13 0x000000010019d000 in ?? () > >> > >> #14 0x00000001001a0a00 in ?? () > >> > >> #15 0x00000001003002ca in ?? () > >> > >> #16 0x0000000000000000 in ?? () > >> > >> > >> On Wed, Dec 3, 2014 at 12:42 AM, Daniel Povey <dp...@gm...> wrote: > >> > >> > >> That isn't normal and it doesn't happen when I compile on a Mac. > >> Can you run it in gdb by doing > >> gdb ./matrix-lib-test > >> (gdb) r > >> > >> and then do ctrl-c when it gets in the loop and type "bt" and show me > >> the backtrace? > >> Dan > >> > >> > >> On Wed, Dec 3, 2014 at 12:36 AM, Joaquin Antonio Ruales > >> <ja...@co...> wrote: > >> > >> Hi Dan, > >> > >> Thanks for the quick reply. I have "svn up"ed and recompiled > >> everything, > >> but > >> now the same matrix test runs into an infinite loop. Would it be safe > >> to > >> ignore the test results and continue with the tutorial? Here are the > >> few > >> lines of output from the test: > >> > >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of > >> random > >> matrix large 7.50727e+16, trying again (this is normal) > >> > >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of > >> random > >> matrix large 1.91265e+16, trying again (this is normal) > >> > >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of > >> random > >> matrix large 7.70738e+16, trying again (this is normal) > >> > >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of > >> random > >> matrix large 2.32488e+17, trying again (this is normal) > >> > >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of > >> random > >> matrix large 8.44565e+17, trying again (this is normal) > >> > >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of > >> random > >> matrix large 1.83557e+17, trying again (this is normal) > >> > >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of > >> random > >> matrix large 2.61736e+17, trying again (this is normal) > >> > >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of > >> random > >> matrix large 8.67803e+16, trying again (this is normal) > >> > >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of > >> random > >> matrix large 1.80053e+16, trying again (this is normal) > >> > >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of > >> random > >> matrix large 1.67101e+16, trying again (this is normal) > >> > >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of > >> random > >> matrix large 1.49485e+16, trying again (this is normal) > >> > >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of > >> random > >> matrix large 8.31813e+16, trying again (this is normal) > >> > >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of > >> random > >> matrix large 1.6285e+16, trying again (this is normal) > >> > >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of > >> random > >> matrix large 2.48172e+16, trying again (this is normal) > >> > >> LOG (RandPosdefSpMatrix():matrix-lib-test.cc:39) Condition number of > >> random > >> matrix large 2.1569e+16, trying again (this is normal) > >> > >> > >> On Tue, Dec 2, 2014 at 4:35 PM, Daniel Povey <dp...@gm...> > >> wrote: > >> > >> > >> I think I had noticed this before, and I fixed the test by changing > >> the threshold. > >> If you do "svn up" and recompile, it should pass. > >> Dan > >> > >> > >> On Tue, Dec 2, 2014 at 4:20 PM, Joaquín Ruales > >> <ja...@co...> > >> wrote: > >> > >> Hi Kaldi Team, > >> > >> I'm running into trouble when running the Kaldi tests (make test) > >> described > >> in the Kaldi tutorial. I'm running it on a Mac (Mavericks) and the > >> error > >> is > >> in matrix-lib-test: KALDI_ASSERT: at > >> UnitTestLinearCgd:matrix-lib-test.cc:3118, failed: error < 1.0e-05 > >> * > >> b.Norm(2.0) > >> > >> Has anyone faced this problem before or have any suggestions? > >> > >> Thanks, > >> Joaquín > >> > >> > >> > >> > >> > >> > ------------------------------------------------------------------------------ > >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > >> from Actuate! Instantly Supercharge Your Business Reports and > >> Dashboards > >> with Interactivity, Sharing, Native Excel Exports, App Integration > >> & > >> more > >> Get technology previously reserved for billion-dollar > >> corporations, > >> FREE > >> > >> > >> > >> > >> > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > >> _______________________________________________ > >> Kaldi-developers mailing list > >> Kal...@li... > >> https://lists.sourceforge.net/lists/listinfo/kaldi-developers > >> > >> > >> > >> > >> > >> > >> > >> > >> > > > |
|
From: Daniel P. <dp...@gm...> - 2014-12-29 20:30:12
|
It seems to me from the attachment that it was working; but you just pressed ctrl-c before it was finished. Dan On Mon, Dec 29, 2014 at 6:05 AM, Nguyễn Thanh <ngu...@gm...> wrote: > Dear You, > Could you help me, please! > I installed Kaldi toolkit. But it not succees. I can't fix it. > When I typed "make" command. > Informed following: > ............. > ^Cmake[3]: *** [invert.lo] Error 1 > make[2]: *** [install-recursive] Interrupt > make[1]: *** [install-recursive] Interrupt > make: *** [openfst_compiled] Interrupt > ....................... > You can see the attachment below. > Could you help me, please! > > Thanks very much! > > Best and Regards! > Thanh, > > ------------------------------------------------------------------------------ > Dive into the World of Parallel Programming! The Go Parallel Website, > sponsored by Intel and developed in partnership with Slashdot Media, is your > hub for all things parallel software development, from weekly thought > leadership blogs to news, videos, case studies, tutorials and more. Take a > look and join the conversation now. http://goparallel.sourceforge.net > _______________________________________________ > Kaldi-developers mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-developers > |
|
From: Nguyễn T. <ngu...@gm...> - 2014-12-29 14:05:31
|
(cd chfilt; make install)
make[3]: Entering directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/chfilt'
cp chfilt.pl /home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk/bin
make[3]: Leaving directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/chfilt'
(cd hamzaNorm; make install)
make[3]: Entering directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/hamzaNorm'
cp hamzaNorm.pl /home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk/bin
make[3]: Leaving directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/hamzaNorm'
(cd tanweenFilt; make install)
make[3]: Entering directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/tanweenFilt'
cp tanweenFilt.pl /home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk/bin
make[3]: Leaving directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/tanweenFilt'
(cd acomp; make install)
make[3]: Entering directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/acomp'
cp acomp.pl /home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk/bin
make[3]: Leaving directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/acomp'
(cd def_art; make install)
make[3]: Entering directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/def_art'
cp def_art.pl /home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk/bin
make[3]: Leaving directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/def_art'
(cd hubscr; make install)
make[3]: Entering directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/hubscr'
cp hubscr.pl /home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk/bin
make[3]: Leaving directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/hubscr'
(cd utf_filt; make install)
make[3]: Entering directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/utf_filt'
make[3]: Leaving directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/utf_filt'
(cd rttmSort; make install)
make[3]: Entering directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/rttmSort'
cp rttmSort.pl /home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk/bin
make[3]: Leaving directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/rttmSort'
(cd rttm2ctm; make install)
make[3]: Entering directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/rttm2ctm'
cp rttm2ctm.pl /home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk/bin
make[3]: Leaving directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/rttm2ctm'
(cd rttmSmooth; make install)
make[3]: Entering directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/rttmSmooth'
cp rttmSmooth.pl /home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk/bin
make[3]: Leaving directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/rttmSmooth'
(cd spkr2sad; make install)
make[3]: Entering directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/spkr2sad'
cp spkr2sad.pl /home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk/bin
make[3]: Leaving directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/spkr2sad'
(cd mergectm2rttm; make install)
make[3]: Entering directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/mergectm2rttm'
cp mergectm2rttm.pl /home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk/bin
make[3]: Leaving directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/mergectm2rttm'
(cd align2html; make install)
make[3]: Entering directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/align2html'
perl buildInstallVersion.pl > /home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk/bin/align2html.pl
chmod +x /home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk/bin/align2html.pl
make[3]: Leaving directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/align2html'
(cd md-eval; make install)
make[3]: Entering directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/md-eval'
cp md-eval.pl /home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk/bin
make[3]: Leaving directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/md-eval'
(cd stm2rttm; make install)
make[3]: Entering directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/stm2rttm'
cp stm2rttm.pl /home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk/bin
make[3]: Leaving directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/stm2rttm'
(cd stmValidator; make install)
make[3]: Entering directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/stmValidator'
perl buildInstallVersion.pl > stmValidator.pl
chmod +x stmValidator.pl
perl buildInstallVersion.pl > /home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk/bin/stmValidator.pl
chmod +x /home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk/bin/stmValidator.pl
make[3]: Leaving directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/stmValidator'
(cd rttmValidator; make install)
make[3]: Entering directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/rttmValidator'
cp rttmValidator.pl /home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk/bin
make[3]: Leaving directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/rttmValidator'
(cd ctmValidator; make install)
make[3]: Entering directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/ctmValidator'
cp ctmValidator.pl /home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk/bin
make[3]: Leaving directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/ctmValidator'
(cd slatreport; make install)
make[3]: Entering directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/slatreport'
cp slatreport.pl /home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk/bin
make[3]: Leaving directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src/slatreport'
make[2]: Leaving directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/src'
make[1]: Leaving directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9'
make[1]: Entering directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9'
(cd doc; make all)
make[2]: Entering directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/doc'
for p in align2html asclite ctmValidator mergectm2rttm; do \
pod2man $p.pod -o $p.1; \
pod2html $p.pod > $p.html; \
done 2> /dev/null
make[2]: Leaving directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9/doc'
make[1]: Leaving directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/sctk-2.4.9'
cd openfst-1.3.4/ && \
make install
make[1]: Entering directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4'
Making install in src
make[2]: Entering directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/src'
Making install in include
make[3]: Entering directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/src/include'
make[4]: Entering directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/src/include'
make[4]: Nothing to be done for `install-exec-am'.
test -z "/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/include" || /bin/mkdir -p "/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/include"
/bin/mkdir -p '/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/include/fst'
/usr/bin/install -c -m 644 fst/arc.h fst/determinize.h fst/intersect.h fst/queue.h fst/statesort.h fst/arcfilter.h fst/dfs-visit.h fst/invert.h fst/randequivalent.h fst/string-weight.h fst/difference.h fst/lexicographic-weight.h fst/randgen.h fst/symbol-table.h fst/arcsort.h fst/encode.h fst/lock.h fst/random-weight.h fst/synchronize.h fst/epsnormalize.h fst/log.h fst/rational.h fst/test-properties.h fst/cache.h fst/equal.h fst/arc-map.h fst/map.h fst/register.h fst/topsort.h fst/closure.h fst/equivalent.h fst/matcher.h fst/matcher-fst.h fst/relabel.h fst/union-find.h fst/compact-fst.h fst/expanded-fst.h fst/minimize.h fst/replace.h fst/union.h '/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/include/fst'
/bin/mkdir -p '/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/include/fst'
/usr/bin/install -c -m 644 fst/compat.h fst/factor-weight.h fst/state-map.h fst/mutable-fst.h fst/reverse.h fst/util.h fst/complement.h fst/flags.h fst/partition.h fst/reweight.h fst/vector-fst.h fst/compose-filter.h fst/float-weight.h fst/product-weight.h fst/rmepsilon.h fst/verify.h fst/compose.h fst/fst-decl.h fst/project.h fst/rmfinalepsilon.h fst/visit.h fst/concat.h fst/fst.h fst/properties.h fst/shortest-distance.h fst/weight.h fst/connect.h fst/fstlib.h fst/prune.h fst/shortest-path.h fst/const-fst.h fst/heap.h fst/push.h fst/state-table.h fst/slist.h fst/pair-weight.h fst/config.h fst/tuple-weight.h fst/power-weight.h fst/lookahead-matcher.h '/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/include/fst'
/bin/mkdir -p '/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/include/fst/script'
/usr/bin/install -c -m 644 fst/script/arcsort.h fst/script/arg-packs.h fst/script/closure.h fst/script/compile-impl.h fst/script/compile.h fst/script/compose.h fst/script/concat.h fst/script/connect.h fst/script/convert.h fst/script/decode.h fst/script/determinize.h fst/script/difference.h fst/script/disambiguate.h fst/script/draw-impl.h fst/script/draw.h fst/script/encode.h fst/script/epsnormalize.h fst/script/equal.h fst/script/equivalent.h fst/script/fst-class.h fst/script/fstscript.h fst/script/info-impl.h fst/script/info.h fst/script/intersect.h fst/script/invert.h fst/script/map.h fst/script/minimize.h fst/script/print-impl.h fst/script/print.h fst/script/project.h fst/script/prune.h fst/script/push.h fst/script/randequivalent.h fst/script/randgen.h fst/script/register.h fst/script/relabel.h fst/script/replace.h fst/script/reverse.h fst/script/reweight.h fst/script/rmepsilon.h '/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/include/fst/script'
/bin/mkdir -p '/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/include/fst'
/usr/bin/install -c -m 644 fst/types.h fst/add-on.h fst/label-reachable.h fst/accumulator.h fst/interval-set.h fst/state-reachable.h fst/lookahead-filter.h fst/generic-register.h fst/edit-fst.h fst/replace-util.h fst/icu.h fst/string.h fst/signed-log-weight.h fst/sparse-tuple-weight.h fst/sparse-power-weight.h fst/expectation-weight.h fst/symbol-table-ops.h fst/bi-table.h fst/mapped-file.h '/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/include/fst'
/bin/mkdir -p '/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/include/fst/script'
/usr/bin/install -c -m 644 fst/script/script-impl.h fst/script/shortest-distance.h fst/script/shortest-path.h fst/script/symbols.h fst/script/synchronize.h fst/script/text-io.h fst/script/topsort.h fst/script/union.h fst/script/weight-class.h fst/script/fstscript-decl.h fst/script/verify.h '/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/include/fst/script'
/bin/mkdir -p '/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/include/fst/extensions/ngram'
/usr/bin/install -c -m 644 fst/extensions/ngram/bitmap-index.h fst/extensions/ngram/ngram-fst.h fst/extensions/ngram/nthbit.h '/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/include/fst/extensions/ngram'
/bin/mkdir -p '/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/include/fst/extensions/far'
/usr/bin/install -c -m 644 fst/extensions/far/compile-strings.h fst/extensions/far/create.h fst/extensions/far/equal.h fst/extensions/far/extract.h fst/extensions/far/far.h fst/extensions/far/farlib.h fst/extensions/far/farscript.h fst/extensions/far/info.h fst/extensions/far/main.h fst/extensions/far/print-strings.h fst/extensions/far/stlist.h fst/extensions/far/sttable.h '/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/include/fst/extensions/far'
make[4]: Leaving directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/src/include'
make[3]: Leaving directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/src/include'
Making install in lib
make[3]: Entering directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/src/lib'
/bin/sh ../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I./../include -MT compat.lo -MD -MP -MF .deps/compat.Tpo -c -o compat.lo compat.cc
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT compat.lo -MD -MP -MF .deps/compat.Tpo -c compat.cc -fPIC -DPIC -o .libs/compat.o
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT compat.lo -MD -MP -MF .deps/compat.Tpo -c compat.cc -o compat.o >/dev/null 2>&1
mv -f .deps/compat.Tpo .deps/compat.Plo
/bin/sh ../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I./../include -MT flags.lo -MD -MP -MF .deps/flags.Tpo -c -o flags.lo flags.cc
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT flags.lo -MD -MP -MF .deps/flags.Tpo -c flags.cc -fPIC -DPIC -o .libs/flags.o
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT flags.lo -MD -MP -MF .deps/flags.Tpo -c flags.cc -o flags.o >/dev/null 2>&1
mv -f .deps/flags.Tpo .deps/flags.Plo
/bin/sh ../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I./../include -MT fst.lo -MD -MP -MF .deps/fst.Tpo -c -o fst.lo fst.cc
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT fst.lo -MD -MP -MF .deps/fst.Tpo -c fst.cc -fPIC -DPIC -o .libs/fst.o
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT fst.lo -MD -MP -MF .deps/fst.Tpo -c fst.cc -o fst.o >/dev/null 2>&1
mv -f .deps/fst.Tpo .deps/fst.Plo
/bin/sh ../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I./../include -MT properties.lo -MD -MP -MF .deps/properties.Tpo -c -o properties.lo properties.cc
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT properties.lo -MD -MP -MF .deps/properties.Tpo -c properties.cc -fPIC -DPIC -o .libs/properties.o
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT properties.lo -MD -MP -MF .deps/properties.Tpo -c properties.cc -o properties.o >/dev/null 2>&1
mv -f .deps/properties.Tpo .deps/properties.Plo
/bin/sh ../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I./../include -MT symbol-table.lo -MD -MP -MF .deps/symbol-table.Tpo -c -o symbol-table.lo symbol-table.cc
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT symbol-table.lo -MD -MP -MF .deps/symbol-table.Tpo -c symbol-table.cc -fPIC -DPIC -o .libs/symbol-table.o
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT symbol-table.lo -MD -MP -MF .deps/symbol-table.Tpo -c symbol-table.cc -o symbol-table.o >/dev/null 2>&1
mv -f .deps/symbol-table.Tpo .deps/symbol-table.Plo
/bin/sh ../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I./../include -MT util.lo -MD -MP -MF .deps/util.Tpo -c -o util.lo util.cc
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT util.lo -MD -MP -MF .deps/util.Tpo -c util.cc -fPIC -DPIC -o .libs/util.o
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT util.lo -MD -MP -MF .deps/util.Tpo -c util.cc -o util.o >/dev/null 2>&1
mv -f .deps/util.Tpo .deps/util.Plo
/bin/sh ../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I./../include -MT symbol-table-ops.lo -MD -MP -MF .deps/symbol-table-ops.Tpo -c -o symbol-table-ops.lo symbol-table-ops.cc
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT symbol-table-ops.lo -MD -MP -MF .deps/symbol-table-ops.Tpo -c symbol-table-ops.cc -fPIC -DPIC -o .libs/symbol-table-ops.o
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT symbol-table-ops.lo -MD -MP -MF .deps/symbol-table-ops.Tpo -c symbol-table-ops.cc -o symbol-table-ops.o >/dev/null 2>&1
mv -f .deps/symbol-table-ops.Tpo .deps/symbol-table-ops.Plo
/bin/sh ../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I./../include -MT mapped-file.lo -MD -MP -MF .deps/mapped-file.Tpo -c -o mapped-file.lo mapped-file.cc
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT mapped-file.lo -MD -MP -MF .deps/mapped-file.Tpo -c mapped-file.cc -fPIC -DPIC -o .libs/mapped-file.o
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT mapped-file.lo -MD -MP -MF .deps/mapped-file.Tpo -c mapped-file.cc -o mapped-file.o >/dev/null 2>&1
mv -f .deps/mapped-file.Tpo .deps/mapped-file.Plo
/bin/sh ../../libtool --tag=CXX --mode=link g++ -version-info 1:0:0 -o libfst.la -rpath /home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/lib compat.lo flags.lo fst.lo properties.lo symbol-table.lo util.lo symbol-table-ops.lo mapped-file.lo -ldl
libtool: link: g++ -fPIC -DPIC -shared -nostdlib /usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../lib64/crti.o /usr/lib/gcc/x86_64-redhat-linux/4.4.7/crtbeginS.o .libs/compat.o .libs/flags.o .libs/fst.o .libs/properties.o .libs/symbol-table.o .libs/util.o .libs/symbol-table-ops.o .libs/mapped-file.o -ldl -L/usr/lib/gcc/x86_64-redhat-linux/4.4.7 -L/usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../.. -lstdc++ -lm -lc -lgcc_s /usr/lib/gcc/x86_64-redhat-linux/4.4.7/crtendS.o /usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../lib64/crtn.o -Wl,-soname -Wl,libfst.so.1 -o .libs/libfst.so.1.0.0
libtool: link: (cd ".libs" && rm -f "libfst.so.1" && ln -s "libfst.so.1.0.0" "libfst.so.1")
libtool: link: (cd ".libs" && rm -f "libfst.so" && ln -s "libfst.so.1.0.0" "libfst.so")
libtool: link: ar cru .libs/libfst.a compat.o flags.o fst.o properties.o symbol-table.o util.o symbol-table-ops.o mapped-file.o
libtool: link: ranlib .libs/libfst.a
libtool: link: ( cd ".libs" && rm -f "libfst.la" && ln -s "../libfst.la" "libfst.la" )
make[4]: Entering directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/src/lib'
test -z "/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/lib" || /bin/mkdir -p "/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/lib"
/bin/sh ../../libtool --mode=install /usr/bin/install -c libfst.la '/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/lib'
libtool: install: /usr/bin/install -c .libs/libfst.so.1.0.0 /home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/lib/libfst.so.1.0.0
libtool: install: (cd /home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/lib && { ln -s -f libfst.so.1.0.0 libfst.so.1 || { rm -f libfst.so.1 && ln -s libfst.so.1.0.0 libfst.so.1; }; })
libtool: install: (cd /home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/lib && { ln -s -f libfst.so.1.0.0 libfst.so || { rm -f libfst.so && ln -s libfst.so.1.0.0 libfst.so; }; })
libtool: install: /usr/bin/install -c .libs/libfst.lai /home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/lib/libfst.la
libtool: install: /usr/bin/install -c .libs/libfst.a /home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/lib/libfst.a
libtool: install: chmod 644 /home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/lib/libfst.a
libtool: install: ranlib /home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/lib/libfst.a
libtool: finish: PATH="/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/Nguyenthanh/bin:/sbin" ldconfig -n /home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/lib
----------------------------------------------------------------------
Libraries have been installed in:
/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/lib
If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the `-LLIBDIR'
flag during linking and do at least one of the following:
- add LIBDIR to the `LD_LIBRARY_PATH' environment variable
during execution
- add LIBDIR to the `LD_RUN_PATH' environment variable
during linking
- use the `-Wl,-rpath -Wl,LIBDIR' linker flag
- have your system administrator add LIBDIR to `/etc/ld.so.conf'
See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.
----------------------------------------------------------------------
make[4]: Nothing to be done for `install-data-am'.
make[4]: Leaving directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/src/lib'
make[3]: Leaving directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/src/lib'
Making install in script
make[3]: Entering directory `/home/Nguyenthanh/Kaldi/kaldi-trunk/tools/openfst-1.3.4/src/script'
/bin/sh ../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I./../include -MT arcsort.lo -MD -MP -MF .deps/arcsort.Tpo -c -o arcsort.lo arcsort.cc
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT arcsort.lo -MD -MP -MF .deps/arcsort.Tpo -c arcsort.cc -fPIC -DPIC -o .libs/arcsort.o
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT arcsort.lo -MD -MP -MF .deps/arcsort.Tpo -c arcsort.cc -o arcsort.o >/dev/null 2>&1
mv -f .deps/arcsort.Tpo .deps/arcsort.Plo
/bin/sh ../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I./../include -MT closure.lo -MD -MP -MF .deps/closure.Tpo -c -o closure.lo closure.cc
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT closure.lo -MD -MP -MF .deps/closure.Tpo -c closure.cc -fPIC -DPIC -o .libs/closure.o
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT closure.lo -MD -MP -MF .deps/closure.Tpo -c closure.cc -o closure.o >/dev/null 2>&1
mv -f .deps/closure.Tpo .deps/closure.Plo
/bin/sh ../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I./../include -MT compile.lo -MD -MP -MF .deps/compile.Tpo -c -o compile.lo compile.cc
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT compile.lo -MD -MP -MF .deps/compile.Tpo -c compile.cc -fPIC -DPIC -o .libs/compile.o
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT compile.lo -MD -MP -MF .deps/compile.Tpo -c compile.cc -o compile.o >/dev/null 2>&1
mv -f .deps/compile.Tpo .deps/compile.Plo
/bin/sh ../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I./../include -MT compose.lo -MD -MP -MF .deps/compose.Tpo -c -o compose.lo compose.cc
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT compose.lo -MD -MP -MF .deps/compose.Tpo -c compose.cc -fPIC -DPIC -o .libs/compose.o
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT compose.lo -MD -MP -MF .deps/compose.Tpo -c compose.cc -o compose.o >/dev/null 2>&1
mv -f .deps/compose.Tpo .deps/compose.Plo
/bin/sh ../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I./../include -MT concat.lo -MD -MP -MF .deps/concat.Tpo -c -o concat.lo concat.cc
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT concat.lo -MD -MP -MF .deps/concat.Tpo -c concat.cc -fPIC -DPIC -o .libs/concat.o
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT concat.lo -MD -MP -MF .deps/concat.Tpo -c concat.cc -o concat.o >/dev/null 2>&1
mv -f .deps/concat.Tpo .deps/concat.Plo
/bin/sh ../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I./../include -MT connect.lo -MD -MP -MF .deps/connect.Tpo -c -o connect.lo connect.cc
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT connect.lo -MD -MP -MF .deps/connect.Tpo -c connect.cc -fPIC -DPIC -o .libs/connect.o
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT connect.lo -MD -MP -MF .deps/connect.Tpo -c connect.cc -o connect.o >/dev/null 2>&1
mv -f .deps/connect.Tpo .deps/connect.Plo
/bin/sh ../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I./../include -MT convert.lo -MD -MP -MF .deps/convert.Tpo -c -o convert.lo convert.cc
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT convert.lo -MD -MP -MF .deps/convert.Tpo -c convert.cc -fPIC -DPIC -o .libs/convert.o
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT convert.lo -MD -MP -MF .deps/convert.Tpo -c convert.cc -o convert.o >/dev/null 2>&1
mv -f .deps/convert.Tpo .deps/convert.Plo
/bin/sh ../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I./../include -MT decode.lo -MD -MP -MF .deps/decode.Tpo -c -o decode.lo decode.cc
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT decode.lo -MD -MP -MF .deps/decode.Tpo -c decode.cc -fPIC -DPIC -o .libs/decode.o
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT decode.lo -MD -MP -MF .deps/decode.Tpo -c decode.cc -o decode.o >/dev/null 2>&1
mv -f .deps/decode.Tpo .deps/decode.Plo
/bin/sh ../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I./../include -MT determinize.lo -MD -MP -MF .deps/determinize.Tpo -c -o determinize.lo determinize.cc
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT determinize.lo -MD -MP -MF .deps/determinize.Tpo -c determinize.cc -fPIC -DPIC -o .libs/determinize.o
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT determinize.lo -MD -MP -MF .deps/determinize.Tpo -c determinize.cc -o determinize.o >/dev/null 2>&1
mv -f .deps/determinize.Tpo .deps/determinize.Plo
/bin/sh ../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I./../include -MT difference.lo -MD -MP -MF .deps/difference.Tpo -c -o difference.lo difference.cc
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT difference.lo -MD -MP -MF .deps/difference.Tpo -c difference.cc -fPIC -DPIC -o .libs/difference.o
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT difference.lo -MD -MP -MF .deps/difference.Tpo -c difference.cc -o difference.o >/dev/null 2>&1
mv -f .deps/difference.Tpo .deps/difference.Plo
/bin/sh ../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I./../include -MT draw.lo -MD -MP -MF .deps/draw.Tpo -c -o draw.lo draw.cc
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT draw.lo -MD -MP -MF .deps/draw.Tpo -c draw.cc -fPIC -DPIC -o .libs/draw.o
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT draw.lo -MD -MP -MF .deps/draw.Tpo -c draw.cc -o draw.o >/dev/null 2>&1
mv -f .deps/draw.Tpo .deps/draw.Plo
/bin/sh ../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I./../include -MT encode.lo -MD -MP -MF .deps/encode.Tpo -c -o encode.lo encode.cc
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT encode.lo -MD -MP -MF .deps/encode.Tpo -c encode.cc -fPIC -DPIC -o .libs/encode.o
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT encode.lo -MD -MP -MF .deps/encode.Tpo -c encode.cc -o encode.o >/dev/null 2>&1
mv -f .deps/encode.Tpo .deps/encode.Plo
/bin/sh ../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I./../include -MT epsnormalize.lo -MD -MP -MF .deps/epsnormalize.Tpo -c -o epsnormalize.lo epsnormalize.cc
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT epsnormalize.lo -MD -MP -MF .deps/epsnormalize.Tpo -c epsnormalize.cc -fPIC -DPIC -o .libs/epsnormalize.o
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT epsnormalize.lo -MD -MP -MF .deps/epsnormalize.Tpo -c epsnormalize.cc -o epsnormalize.o >/dev/null 2>&1
mv -f .deps/epsnormalize.Tpo .deps/epsnormalize.Plo
/bin/sh ../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I./../include -MT equal.lo -MD -MP -MF .deps/equal.Tpo -c -o equal.lo equal.cc
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT equal.lo -MD -MP -MF .deps/equal.Tpo -c equal.cc -fPIC -DPIC -o .libs/equal.o
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT equal.lo -MD -MP -MF .deps/equal.Tpo -c equal.cc -o equal.o >/dev/null 2>&1
mv -f .deps/equal.Tpo .deps/equal.Plo
/bin/sh ../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I./../include -MT equivalent.lo -MD -MP -MF .deps/equivalent.Tpo -c -o equivalent.lo equivalent.cc
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT equivalent.lo -MD -MP -MF .deps/equivalent.Tpo -c equivalent.cc -fPIC -DPIC -o .libs/equivalent.o
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT equivalent.lo -MD -MP -MF .deps/equivalent.Tpo -c equivalent.cc -o equivalent.o >/dev/null 2>&1
mv -f .deps/equivalent.Tpo .deps/equivalent.Plo
/bin/sh ../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I./../include -MT fst-class.lo -MD -MP -MF .deps/fst-class.Tpo -c -o fst-class.lo fst-class.cc
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT fst-class.lo -MD -MP -MF .deps/fst-class.Tpo -c fst-class.cc -fPIC -DPIC -o .libs/fst-class.o
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT fst-class.lo -MD -MP -MF .deps/fst-class.Tpo -c fst-class.cc -o fst-class.o >/dev/null 2>&1
mv -f .deps/fst-class.Tpo .deps/fst-class.Plo
/bin/sh ../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I./../include -MT info.lo -MD -MP -MF .deps/info.Tpo -c -o info.lo info.cc
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT info.lo -MD -MP -MF .deps/info.Tpo -c info.cc -fPIC -DPIC -o .libs/info.o
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT info.lo -MD -MP -MF .deps/info.Tpo -c info.cc -o info.o >/dev/null 2>&1
mv -f .deps/info.Tpo .deps/info.Plo
/bin/sh ../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I./../include -MT intersect.lo -MD -MP -MF .deps/intersect.Tpo -c -o intersect.lo intersect.cc
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT intersect.lo -MD -MP -MF .deps/intersect.Tpo -c intersect.cc -fPIC -DPIC -o .libs/intersect.o
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT intersect.lo -MD -MP -MF .deps/intersect.Tpo -c intersect.cc -o intersect.o >/dev/null 2>&1
mv -f .deps/intersect.Tpo .deps/intersect.Plo
/bin/sh ../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I./../include -MT invert.lo -MD -MP -MF .deps/invert.Tpo -c -o invert.lo invert.cc
libtool: compile: g++ -DHAVE_CONFIG_H -I./../include -MT invert.lo -MD -MP -MF .deps/invert.Tpo -c invert.cc -fPIC -DPIC -o .libs/invert.o
^Cmake[3]: *** [invert.lo] Error 1
make[2]: *** [install-recursive] Interrupt
make[1]: *** [install-recursive] Interrupt
make: *** [openfst_compiled] Interrupt
[root@Nguyenthanh tools]# |
|
From: Sean T. <se...@se...> - 2014-12-29 06:52:20
|
I wanted to echo Ondrej's comment about preferring Python to bash/perl for
scripting. Python wrappers for the command line utilities are useful ...
I've spent a few hours systematically wrapping them, parsing the output of
the --help command as a guide to functionality.
This gives wrappers of the general form:
def acc_lda(transition_gmm_model, features_rspecifier,
posteriors_rspecifier, lda_acc_out, *args, **kwargs):
"""Accumulate LDA statistics based on pdf-ids.
Executable usage: acc-lda [options] <transition-gmm/model>
<features-rspecifier> <posteriors-rspecifier> <lda-acc-out>
Options:
binary: Write accumulators in binary mode. (bool,true)
rand_prune: Randomized pruning threshold for posteriors (float,0)"""
cmd = sh.Command(kaldi_path("src/bin/acc-lda"))
option_defs = {'binary': ('binary', 'bool', 'true'), 'help': ('help',
'bool', 'false'), 'rand_prune': ('rand-prune', 'float', '0'), 'config':
('config', 'string', ''), 'print_args': ('print-args', 'bool', 'true'),
'verbose': ('verbose', 'int', '0')}
myOptions = create_options(option_defs, kwargs)
myArgs = [transition_gmm_model, features_rspecifier,
posteriors_rspecifier, lda_acc_out]+list(args)
return cmd (myOptions + myArgs)
There are some refinements that could be added (*args does not make sense
for this function).
Because of the rather elegant Python sh package (
https://pypi.python.org/pypi/sh) these functions will create pipelines if
composed:
>>> from sh import ls, wc
>>> wc(ls("."))
8 23 222
There are a few places where constructing from help output is not
straightforward (for instance, fstrand --help does
not do the expected thing).
-- Sean
On Fri, Dec 19, 2014 at 6:48 AM, Ondrej Platek <ond...@gm...>
wrote:
>
> Hi Matthew,
>
> I made some subjective comments below.
>
> PS: Note that I like the proposed wrappers, but I am not sure how
boost::python is easy to install on all supported platforms.
>
> On Fri, Dec 19, 2014 at 9:30 AM, Matthew Aylett <mat...@gm...>
wrote:
>>
>> Hi
>>
>> Apologies, I've been snowed under here.
>>
>> I haven' had a chance to look over your work. I also don't have any
views on the 'right' way to do it. My thoughts on this are in a previous
thread. See subject "Using SWIG to wrap kaldi for python" where I discussed
this with ondrej platek and
>> Vassil Panayotov.
>>
>> In the idlak branch there is an example of python wrappers that I put
together some time ago. These are based on SWIG. In the end I didn't need
this at this stage because in the build system command line executables
work very well. Its in run time wrappers are very useful. The advantage
with SWIG is that the much of the same work will also contribute to C#,
Java, Perl wrappers as well. In my experience the most important were Java
wrappers to help produce a library for Android. I have no experience with
C# and moved to Python from Perl so only use Perl in legacy code ;-).
>>
>> So some questions to consider:
>>
>> 1. Why is python wrapping required for training. using sys.Process to
run command lines, structured output directories etc mirrors the current
Perl recipes, what is the added benefit in this case?
>
> Well bash and Perl is the current scripting language for Kaldi. For
example I prefer to use Python instead of both of them.
>
>>
>> 2. If its for run time decoding shouldn't we create a cross platfom C
API? Perhaps things have changed but C++ APIs were never cross compiler
compatible in the past so you couldn't do stuff like compile using gnu and
link in MSN. With a C interface you can distribute libraries. But I am
possibly out of date on this.
>
> Well, I tried that and I gave it up since Kaldi nicely uses OpenFST and I
was not able to wrap OpenFST with just plain C (It may be possible).
> I used Cython and pyfst mainly because pyfst solved for me wrapping up
OpenFST and I am really glad that 99% of wrapping OpenFST templates was
carried out by somebody else (Victor Chahuneau).
>>
>>
>> 3. If 2 is correct shouldn't we define our API and wrap that? Producing
a formal list of functionality that should be exposed to things like client
and server applications?
>>
>>
>> I would encourage some care here. Unconstrained wrapping can lead to
systems which HAVE to use the scripting language (We can already see how
difficult it is to move away from the Perl scripting if you wish to). Also
never, never, never reverse wrap (i.e. call python from within C++), yes it
can be done but that way lays madness.
>>
>> v best
>>
>> Matthew
>>
>>
>> On Thu, Dec 18, 2014 at 11:37 PM, Daniel Povey <dp...@gm...> wrote:
>>>
>>> Jan-
>>> I haven't seen any objections to your setup. I'd say we should plan
>>> to include it in Kaldi at some point (e.g. within the next few
>>> months), but in the meantime hopefully you can continue to work on it,
>>> and maybe come up with some other examples of how it's useful to do
>>> the interfacing with Python- e.g. some kind of application level or
>>> service-level thing?
>>> Dan
>>>
>>>
>>> On Sat, Dec 13, 2014 at 4:01 PM, Yajie Miao <yaj...@gm...> wrote:
>>> > Hi Jan,
>>> > This is very nice work! In our PDNN toolkit, we also have simple
python
>>> > wrappers to read and write Kaldi features, mainly for DNN training.
Your
>>> > implementation looks like a more comprehensive version.
>>> >
>>> > Do you have the functions/commands to do feature splicing? I ask this
>>> > because we found doing splicing on the fly with Python highly
expensive.
>>> > That's why we still stick to PFiles instead of Kaldi features (.scp
.ark)
>>> > for DNN triaining. I am very interested to know the efficiency of
your
>>> > splicing implementation.
>>> >
>>> > Thanks,
>>> > Yajie
>>> >
>>> > On Sat, Dec 13, 2014 at 5:59 PM, Daniel Povey <dp...@gm...>
wrote:
>>> >>
>>> >> OK, thanks.
>>> >> cc'ing Yajie in case he wants to comment.
>>> >> Dan
>>> >>
>>> >>
>>> >> On Sat, Dec 13, 2014 at 2:31 PM, Jan Chorowski <
jan...@gm...>
>>> >> wrote:
>>> >> > Hi All,
>>> >> >
>>> >> > the wrapper is built during Kaldi compilation. I build it using
provided
>>> >> > Makefile. The build depends on:
>>> >> > 1. Python and numpy (by default it queries the python interpreter
found
>>> >> > on
>>> >> > the path for header file location)
>>> >> > 2. Boost with Boost::Python library. It is quite heavy to build,
but
>>> >> > most
>>> >> > Linux distributions ship it. Boost python doesn't require any code
>>> >> > generation steps, the wrapper is defined in a normal c++ code file.
>>> >> >
>>> >> > During build Python and Boost libraries and Kaldi object files are
>>> >> > linked
>>> >> > into a CPython extention module,
kaldi/src/python/kaldi_io_internal.so.
>>> >> > It
>>> >> > works with both static and shared Kaldi builds. Further usage
requires
>>> >> > that
>>> >> > python finds kaldi_io.py and kaldi_io_internal.so on the
PYTHONPATH - it
>>> >> > can
>>> >> > be for example added to the PYTHONPATH variable in the path.sh
script of
>>> >> > a
>>> >> > recipe.
>>> >> >
>>> >> > Jan
>>> >> >
>>> >> >
>>> >> > On 12/13/2014 3:33 PM, Daniel Povey wrote:
>>> >> >>
>>> >> >> Also, Jan- could you send us an email explaining how this works-
>>> >> >> How does Python "see" the C++ headers? Do you have to invoke
some
>>> >> >> special program, like swig? Do you have to write some special
kind of
>>> >> >> header that shows how the C++ objects are to be interpreted by
python?
>>> >> >> A brief example would be helpful, if so.
>>> >> >> How is the resulting program linked, if at all? If you require
>>> >> >> functions C++ libraries, are these obtained from the .a or .so
files
>>> >> >> at runtime, or compiled into some kind of executable-like blob at
>>> >> >> compile time? Does your framework require that Kaldi be compiled
>>> >> >> using dynamic (.so) libraries?
>>> >> >>
>>> >> >> Dan
>>> >> >>
>>> >> >>
>>> >> >> On Sat, Dec 13, 2014 at 12:04 PM, Jan Chorowski
>>> >> >> <jan...@gm...>
>>> >> >> wrote:
>>> >> >>>
>>> >> >>> Hello Dan,
>>> >> >>>
>>> >> >>> thank you for the comments. I tried to make it in the Kaldi
spirit,
>>> >> >>> consistency is important. Of course, the scripts can be removed
and
>>> >> >>> replaced
>>> >> >>> with some more useful examples. I don't have too much experience
with
>>> >> >>> bridging Python to C++, so any critique on the wrappers and the
>>> >> >>> approach
>>> >> >>> taken is welcome.
>>> >> >>>
>>> >> >>> Jan
>>> >> >>>
>>> >> >>>
>>> >> >>> On 12/13/2014 2:55 PM, Daniel Povey wrote:
>>> >> >>>>
>>> >> >>>> Hi all.
>>> >> >>>> From a first look, it does look very impressive, and nicely
>>> >> >>>> documented.
>>> >> >>>> I would appreciate it if people on the list who have Python
>>> >> >>>> experience
>>> >> >>>> would comment on this- you can either reply to this thread, or
to me.
>>> >> >>>> I don't know if this has been done in the "natural" way, or if
there
>>> >> >>>> is some reason why people in the future will say, "why did you
do it
>>> >> >>>> this way, you should have done XXX".
>>> >> >>>>
>>> >> >>>> Jan:
>>> >> >>>> in the scripts/ directory you seem to have some examples of how
you
>>> >> >>>> can create python programs that behave very much like Kaldi
>>> >> >>>> command-line programs, using your framework. This is very
useful.
>>> >> >>>> However, the programs
>>> >> >>>> apply-global-cmvn.py
>>> >> >>>> compute-global-cmvn-stats.py
>>> >> >>>> are perhaps a little confusing because they provide the same
>>> >> >>>> functionality that you could get with "compute-cmvn-stats ->
>>> >> >>>> matrix-sum" and "apply-cmvn" on the output of that command; and
they
>>> >> >>>> do so using different formats for the CMVN information. I know
the
>>> >> >>>> format of storing the CMVN stats in a two-row matrix is perhaps
not
>>> >> >>>> perfectly ideal, but it's a standard within Kaldi and it would
be
>>> >> >>>> confusing to deviate from that standard.
>>> >> >>>> Of course, this is a very minor issue that doesn't affect the
>>> >> >>>> validity
>>> >> >>>> of the framework as a whole. I am just pointing this out; the
main
>>> >> >>>> discussion should be about the framework and whether people
feel it's
>>> >> >>>> the "right" way to do this.
>>> >> >>>>
>>> >> >>>> Dan
>>> >> >>>>
>>> >> >>>> On Sat, Dec 13, 2014 at 6:28 AM, Jan Chorowski
>>> >> >>>> <jan...@gm...>
>>> >> >>>> wrote:
>>> >> >>>>>
>>> >> >>>>> Hi all!
>>> >> >>>>>
>>> >> >>>>> I've written wrappers to access Kaldi data files from within
Python
>>> >> >>>>> using boost::python (the code is on github
>>> >> >>>>>
https://github.com/janchorowski/kaldi-git/tree/python/src/python).
>>> >> >>>>> If
>>> >> >>>>> you think this would be an interesting addition please
instruct me
>>> >> >>>>> how
>>> >> >>>>> to contribute.
>>> >> >>>>>
>>> >> >>>>> Best Regards,
>>> >> >>>>> Jan Chorowski
>>> >> >>>>>
>>> >> >>>>>
>>> >> >>>>>
>>> >> >>>>>
>>> >> >>>>>
>>> >> >>>>>
------------------------------------------------------------------------------
>>> >> >>>>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT
Server
>>> >> >>>>> from Actuate! Instantly Supercharge Your Business Reports and
>>> >> >>>>> Dashboards
>>> >> >>>>> with Interactivity, Sharing, Native Excel Exports, App
Integration &
>>> >> >>>>> more
>>> >> >>>>> Get technology previously reserved for billion-dollar
corporations,
>>> >> >>>>> FREE
>>> >> >>>>>
>>> >> >>>>>
>>> >> >>>>>
>>> >> >>>>>
http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
>>> >> >>>>> _______________________________________________
>>> >> >>>>> Kaldi-developers mailing list
>>> >> >>>>> Kal...@li...
>>> >> >>>>> https://lists.sourceforge.net/lists/listinfo/kaldi-developers
>>> >> >>>
>>> >> >>>
>>> >> >
>>> >
>>> >
>>>
>>>
------------------------------------------------------------------------------
>>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
>>> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
>>> with Interactivity, Sharing, Native Excel Exports, App Integration &
more
>>> Get technology previously reserved for billion-dollar corporations, FREE
>>>
http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
>>> _______________________________________________
>>> Kaldi-developers mailing list
>>> Kal...@li...
>>> https://lists.sourceforge.net/lists/listinfo/kaldi-developers
>>
>>
>>
------------------------------------------------------------------------------
>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
>> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
>> with Interactivity, Sharing, Native Excel Exports, App Integration & more
>> Get technology previously reserved for billion-dollar corporations, FREE
>>
http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Kaldi-developers mailing list
>> Kal...@li...
>> https://lists.sourceforge.net/lists/listinfo/kaldi-developers
>>
>
>
> --
> Ondřej Plátek, +420 737 758 650, skype:ondrejplatek,
ond...@gm...
>
>
------------------------------------------------------------------------------
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> Get technology previously reserved for billion-dollar corporations, FREE
>
http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
> _______________________________________________
> Kaldi-developers mailing list
> Kal...@li...
> https://lists.sourceforge.net/lists/listinfo/kaldi-developers
>
|
|
From: Matthew A. <mat...@gm...> - 2014-12-22 09:03:49
|
Hi So asI understand it we have two rather different objectives. 1. Use python libraries and utilities to advance and prototype training i.e with neural nets. So this is the build side. 2. Wrap Kaldi for a future API to be used by say client and server apps. For 1. I would question how important it is to wrap the low level data strucutures, instead of say, converting/rereading from disk. I guess you want to mix and match fuctionality during training on the same data. Alo, a very useful function of doing thisis that it 'opens' up the system making allowing interaction and testing. More important actually for synthesis where less operations require multi-core batch style processing. For 2. we should specify an API for run time, we shouldn't need to wrap FST as that is to low level and its better to write utiltity functions and wrap them. Here multi language support is more useful. Might be nice to do both the same sort of way but depending on boost could be a little extreme for doing it. Matthew On Mon, Dec 22, 2014 at 8:12 AM, Jan Chorowski <jan...@gm...> wrote: > Hi All, > > First I'd like to clarify that the wrappers are only meant to read and > write a limited set of objects (matrices, vectors, scalars, pairs) from > .scp and .ark into Python with a Pythonic feel (dict-like interface for > radom access iterators, iterator interface for sequential ones, context > management). I didn't consider wrapping fst classes nor anything more > specific to Kaldi. > > In their limited form the wrappers are useful to e.g. use some python > neural network library to train nets on Kaldi's features and then use the > Kaldi decoders, in a manner similar to Kaldi DNN recipes. > > I decided not to write the wrappers as generic SWIG wrappers because I > target conversion from Kaldi data files to Python specific classes, such as > ndarray. Wrappers for other languages would need to address other > language-specific matrix libraries etc. I don't know SWIG too much and > maybe it would be useful to make wrappers for several languages at once, > though. > > Since the wrappers are optional, maybe the best way to proceed is to > extract them into a separate repository, that can be built against against > an installation of Kaldi and see whether it is used by anyone? I can work > on it sometimes during the holiday break and let you know once it is > published. > > @Yajie > splicing a single utterance is easy. You pad it with zeros, then set the > stride to be one frame and shape to be several ones. Thus it looks like one > large spliced array. However, to train a network you want to shuffle > examples (i.e. spliced frames) from multiple utterances - I resorted to > reading a few hundred megs of data in to memory, then shuffling it, then > reading another batch form the disk. I can send you the code if you would > be interested. > > Jan > > > On 12/21/2014 9:36 PM, Daniel Povey wrote: > >> Jan, perhaps when you have time you could respond to the above >> comments on the list? >> I'm not 100% sure what to do about this. >> BTW, if we do include this, it will likely be optionally compiled, >> because I don't want the generic Kaldi compilation to be dependent on >> boost. >> >> Dan >> >> >> On Fri, Dec 19, 2014 at 3:48 AM, Ondrej Platek <ond...@gm...> >> wrote: >> >>> Hi Matthew, >>> >>> I made some subjective comments below. >>> >>> PS: Note that I like the proposed wrappers, but I am not sure how >>> boost::python is easy to install on all supported platforms. >>> >>> On Fri, Dec 19, 2014 at 9:30 AM, Matthew Aylett <mat...@gm... >>> > >>> wrote: >>> >>>> Hi >>>> >>>> Apologies, I've been snowed under here. >>>> >>>> I haven' had a chance to look over your work. I also don't have any >>>> views >>>> on the 'right' way to do it. My thoughts on this are in a previous >>>> thread. >>>> See subject "Using SWIG to wrap kaldi for python" where I discussed this >>>> with ondrej platek and >>>> Vassil Panayotov. >>>> >>>> In the idlak branch there is an example of python wrappers that I put >>>> together some time ago. These are based on SWIG. In the end I didn't >>>> need >>>> this at this stage because in the build system command line executables >>>> work >>>> very well. Its in run time wrappers are very useful. The advantage with >>>> SWIG >>>> is that the much of the same work will also contribute to C#, Java, Perl >>>> wrappers as well. In my experience the most important were Java >>>> wrappers to >>>> help produce a library for Android. I have no experience with C# and >>>> moved >>>> to Python from Perl so only use Perl in legacy code ;-). >>>> >>>> So some questions to consider: >>>> >>>> 1. Why is python wrapping required for training. using sys.Process to >>>> run >>>> command lines, structured output directories etc mirrors the current >>>> Perl >>>> recipes, what is the added benefit in this case? >>>> >>> Well bash and Perl is the current scripting language for Kaldi. For >>> example >>> I prefer to use Python instead of both of them. >>> >>> 2. If its for run time decoding shouldn't we create a cross platfom C >>>> API? Perhaps things have changed but C++ APIs were never cross compiler >>>> compatible in the past so you couldn't do stuff like compile using gnu >>>> and >>>> link in MSN. With a C interface you can distribute libraries. But I am >>>> possibly out of date on this. >>>> >>> Well, I tried that and I gave it up since Kaldi nicely uses OpenFST and I >>> was not able to wrap OpenFST with just plain C (It may be possible). >>> I used Cython and pyfst mainly because pyfst solved for me wrapping up >>> OpenFST and I am really glad that 99% of wrapping OpenFST templates was >>> carried out by somebody else (Victor Chahuneau). >>> >>>> >>>> 3. If 2 is correct shouldn't we define our API and wrap that? Producing >>>> a >>>> formal list of functionality that should be exposed to things like >>>> client >>>> and server applications? >>>> >>>> >>>> I would encourage some care here. Unconstrained wrapping can lead to >>>> systems which HAVE to use the scripting language (We can already see how >>>> difficult it is to move away from the Perl scripting if you wish to). >>>> Also >>>> never, never, never reverse wrap (i.e. call python from within C++), >>>> yes it >>>> can be done but that way lays madness. >>>> >>>> v best >>>> >>>> Matthew >>>> >>>> >>>> On Thu, Dec 18, 2014 at 11:37 PM, Daniel Povey <dp...@gm...> >>>> wrote: >>>> >>>>> Jan- >>>>> I haven't seen any objections to your setup. I'd say we should plan >>>>> to include it in Kaldi at some point (e.g. within the next few >>>>> months), but in the meantime hopefully you can continue to work on it, >>>>> and maybe come up with some other examples of how it's useful to do >>>>> the interfacing with Python- e.g. some kind of application level or >>>>> service-level thing? >>>>> Dan >>>>> >>>>> >>>>> On Sat, Dec 13, 2014 at 4:01 PM, Yajie Miao <yaj...@gm...> >>>>> wrote: >>>>> >>>>>> Hi Jan, >>>>>> This is very nice work! In our PDNN toolkit, we also have simple >>>>>> python >>>>>> wrappers to read and write Kaldi features, mainly for DNN training. >>>>>> Your >>>>>> implementation looks like a more comprehensive version. >>>>>> >>>>>> Do you have the functions/commands to do feature splicing? I ask this >>>>>> because we found doing splicing on the fly with Python highly >>>>>> expensive. >>>>>> That's why we still stick to PFiles instead of Kaldi features (.scp >>>>>> .ark) >>>>>> for DNN triaining. I am very interested to know the efficiency of >>>>>> your >>>>>> splicing implementation. >>>>>> >>>>>> Thanks, >>>>>> Yajie >>>>>> >>>>>> On Sat, Dec 13, 2014 at 5:59 PM, Daniel Povey <dp...@gm...> >>>>>> wrote: >>>>>> >>>>>>> OK, thanks. >>>>>>> cc'ing Yajie in case he wants to comment. >>>>>>> Dan >>>>>>> >>>>>>> >>>>>>> On Sat, Dec 13, 2014 at 2:31 PM, Jan Chorowski >>>>>>> <jan...@gm...> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi All, >>>>>>>> >>>>>>>> the wrapper is built during Kaldi compilation. I build it using >>>>>>>> provided >>>>>>>> Makefile. The build depends on: >>>>>>>> 1. Python and numpy (by default it queries the python interpreter >>>>>>>> found >>>>>>>> on >>>>>>>> the path for header file location) >>>>>>>> 2. Boost with Boost::Python library. It is quite heavy to build, but >>>>>>>> most >>>>>>>> Linux distributions ship it. Boost python doesn't require any code >>>>>>>> generation steps, the wrapper is defined in a normal c++ code file. >>>>>>>> >>>>>>>> During build Python and Boost libraries and Kaldi object files are >>>>>>>> linked >>>>>>>> into a CPython extention module, >>>>>>>> kaldi/src/python/kaldi_io_internal.so. >>>>>>>> It >>>>>>>> works with both static and shared Kaldi builds. Further usage >>>>>>>> requires >>>>>>>> that >>>>>>>> python finds kaldi_io.py and kaldi_io_internal.so on the PYTHONPATH >>>>>>>> - it >>>>>>>> can >>>>>>>> be for example added to the PYTHONPATH variable in the path.sh >>>>>>>> script of >>>>>>>> a >>>>>>>> recipe. >>>>>>>> >>>>>>>> Jan >>>>>>>> >>>>>>>> >>>>>>>> On 12/13/2014 3:33 PM, Daniel Povey wrote: >>>>>>>> >>>>>>>>> Also, Jan- could you send us an email explaining how this works- >>>>>>>>> How does Python "see" the C++ headers? Do you have to invoke >>>>>>>>> some >>>>>>>>> special program, like swig? Do you have to write some special kind >>>>>>>>> of >>>>>>>>> header that shows how the C++ objects are to be interpreted by >>>>>>>>> python? >>>>>>>>> A brief example would be helpful, if so. >>>>>>>>> How is the resulting program linked, if at all? If you require >>>>>>>>> functions C++ libraries, are these obtained from the .a or .so >>>>>>>>> files >>>>>>>>> at runtime, or compiled into some kind of executable-like blob at >>>>>>>>> compile time? Does your framework require that Kaldi be compiled >>>>>>>>> using dynamic (.so) libraries? >>>>>>>>> >>>>>>>>> Dan >>>>>>>>> >>>>>>>>> >>>>>>>>> On Sat, Dec 13, 2014 at 12:04 PM, Jan Chorowski >>>>>>>>> <jan...@gm...> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hello Dan, >>>>>>>>>> >>>>>>>>>> thank you for the comments. I tried to make it in the Kaldi >>>>>>>>>> spirit, >>>>>>>>>> consistency is important. Of course, the scripts can be removed >>>>>>>>>> and >>>>>>>>>> replaced >>>>>>>>>> with some more useful examples. I don't have too much experience >>>>>>>>>> with >>>>>>>>>> bridging Python to C++, so any critique on the wrappers and the >>>>>>>>>> approach >>>>>>>>>> taken is welcome. >>>>>>>>>> >>>>>>>>>> Jan >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 12/13/2014 2:55 PM, Daniel Povey wrote: >>>>>>>>>> >>>>>>>>>>> Hi all. >>>>>>>>>>> From a first look, it does look very impressive, and nicely >>>>>>>>>>> documented. >>>>>>>>>>> I would appreciate it if people on the list who have Python >>>>>>>>>>> experience >>>>>>>>>>> would comment on this- you can either reply to this thread, or to >>>>>>>>>>> me. >>>>>>>>>>> I don't know if this has been done in the "natural" way, or if >>>>>>>>>>> there >>>>>>>>>>> is some reason why people in the future will say, "why did you do >>>>>>>>>>> it >>>>>>>>>>> this way, you should have done XXX". >>>>>>>>>>> >>>>>>>>>>> Jan: >>>>>>>>>>> in the scripts/ directory you seem to have some examples of how >>>>>>>>>>> you >>>>>>>>>>> can create python programs that behave very much like Kaldi >>>>>>>>>>> command-line programs, using your framework. This is very >>>>>>>>>>> useful. >>>>>>>>>>> However, the programs >>>>>>>>>>> apply-global-cmvn.py >>>>>>>>>>> compute-global-cmvn-stats.py >>>>>>>>>>> are perhaps a little confusing because they provide the same >>>>>>>>>>> functionality that you could get with "compute-cmvn-stats -> >>>>>>>>>>> matrix-sum" and "apply-cmvn" on the output of that command; and >>>>>>>>>>> they >>>>>>>>>>> do so using different formats for the CMVN information. I know >>>>>>>>>>> the >>>>>>>>>>> format of storing the CMVN stats in a two-row matrix is perhaps >>>>>>>>>>> not >>>>>>>>>>> perfectly ideal, but it's a standard within Kaldi and it would be >>>>>>>>>>> confusing to deviate from that standard. >>>>>>>>>>> Of course, this is a very minor issue that doesn't affect the >>>>>>>>>>> validity >>>>>>>>>>> of the framework as a whole. I am just pointing this out; the >>>>>>>>>>> main >>>>>>>>>>> discussion should be about the framework and whether people feel >>>>>>>>>>> it's >>>>>>>>>>> the "right" way to do this. >>>>>>>>>>> >>>>>>>>>>> Dan >>>>>>>>>>> >>>>>>>>>>> On Sat, Dec 13, 2014 at 6:28 AM, Jan Chorowski >>>>>>>>>>> <jan...@gm...> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi all! >>>>>>>>>>>> >>>>>>>>>>>> I've written wrappers to access Kaldi data files from within >>>>>>>>>>>> Python >>>>>>>>>>>> using boost::python (the code is on github >>>>>>>>>>>> >>>>>>>>>>>> https://github.com/janchorowski/kaldi-git/tree/ >>>>>>>>>>>> python/src/python). >>>>>>>>>>>> If >>>>>>>>>>>> you think this would be an interesting addition please instruct >>>>>>>>>>>> me >>>>>>>>>>>> how >>>>>>>>>>>> to contribute. >>>>>>>>>>>> >>>>>>>>>>>> Best Regards, >>>>>>>>>>>> Jan Chorowski >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> ------------------------------------------------------------ >>>>>>>>>>>> ------------------ >>>>>>>>>>>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT >>>>>>>>>>>> Server >>>>>>>>>>>> from Actuate! Instantly Supercharge Your Business Reports and >>>>>>>>>>>> Dashboards >>>>>>>>>>>> with Interactivity, Sharing, Native Excel Exports, App >>>>>>>>>>>> Integration & >>>>>>>>>>>> more >>>>>>>>>>>> Get technology previously reserved for billion-dollar >>>>>>>>>>>> corporations, >>>>>>>>>>>> FREE >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> http://pubads.g.doubleclick.net/gampad/clk?id=164703151& >>>>>>>>>>>> iu=/4140/ostg.clktrk >>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>> Kaldi-developers mailing list >>>>>>>>>>>> Kal...@li... >>>>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/kaldi-developers >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>> >>>>> ------------------------------------------------------------ >>>>> ------------------ >>>>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >>>>> from Actuate! Instantly Supercharge Your Business Reports and >>>>> Dashboards >>>>> with Interactivity, Sharing, Native Excel Exports, App Integration & >>>>> more >>>>> Get technology previously reserved for billion-dollar corporations, >>>>> FREE >>>>> >>>>> http://pubads.g.doubleclick.net/gampad/clk?id=164703151& >>>>> iu=/4140/ostg.clktrk >>>>> _______________________________________________ >>>>> Kaldi-developers mailing list >>>>> Kal...@li... >>>>> https://lists.sourceforge.net/lists/listinfo/kaldi-developers >>>>> >>>> >>>> >>>> ------------------------------------------------------------ >>>> ------------------ >>>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >>>> from Actuate! Instantly Supercharge Your Business Reports and Dashboards >>>> with Interactivity, Sharing, Native Excel Exports, App Integration & >>>> more >>>> Get technology previously reserved for billion-dollar corporations, FREE >>>> >>>> http://pubads.g.doubleclick.net/gampad/clk?id=164703151& >>>> iu=/4140/ostg.clktrk >>>> _______________________________________________ >>>> Kaldi-developers mailing list >>>> Kal...@li... >>>> https://lists.sourceforge.net/lists/listinfo/kaldi-developers >>>> >>>> >>> -- >>> Ondřej Plátek, +420 737 758 650, skype:ondrejplatek, >>> ond...@gm... >>> >> > |
|
From: Jan C. <jan...@gm...> - 2014-12-22 08:12:26
|
Hi All, First I'd like to clarify that the wrappers are only meant to read and write a limited set of objects (matrices, vectors, scalars, pairs) from .scp and .ark into Python with a Pythonic feel (dict-like interface for radom access iterators, iterator interface for sequential ones, context management). I didn't consider wrapping fst classes nor anything more specific to Kaldi. In their limited form the wrappers are useful to e.g. use some python neural network library to train nets on Kaldi's features and then use the Kaldi decoders, in a manner similar to Kaldi DNN recipes. I decided not to write the wrappers as generic SWIG wrappers because I target conversion from Kaldi data files to Python specific classes, such as ndarray. Wrappers for other languages would need to address other language-specific matrix libraries etc. I don't know SWIG too much and maybe it would be useful to make wrappers for several languages at once, though. Since the wrappers are optional, maybe the best way to proceed is to extract them into a separate repository, that can be built against against an installation of Kaldi and see whether it is used by anyone? I can work on it sometimes during the holiday break and let you know once it is published. @Yajie splicing a single utterance is easy. You pad it with zeros, then set the stride to be one frame and shape to be several ones. Thus it looks like one large spliced array. However, to train a network you want to shuffle examples (i.e. spliced frames) from multiple utterances - I resorted to reading a few hundred megs of data in to memory, then shuffling it, then reading another batch form the disk. I can send you the code if you would be interested. Jan On 12/21/2014 9:36 PM, Daniel Povey wrote: > Jan, perhaps when you have time you could respond to the above > comments on the list? > I'm not 100% sure what to do about this. > BTW, if we do include this, it will likely be optionally compiled, > because I don't want the generic Kaldi compilation to be dependent on > boost. > > Dan > > > On Fri, Dec 19, 2014 at 3:48 AM, Ondrej Platek <ond...@gm...> wrote: >> Hi Matthew, >> >> I made some subjective comments below. >> >> PS: Note that I like the proposed wrappers, but I am not sure how >> boost::python is easy to install on all supported platforms. >> >> On Fri, Dec 19, 2014 at 9:30 AM, Matthew Aylett <mat...@gm...> >> wrote: >>> Hi >>> >>> Apologies, I've been snowed under here. >>> >>> I haven' had a chance to look over your work. I also don't have any views >>> on the 'right' way to do it. My thoughts on this are in a previous thread. >>> See subject "Using SWIG to wrap kaldi for python" where I discussed this >>> with ondrej platek and >>> Vassil Panayotov. >>> >>> In the idlak branch there is an example of python wrappers that I put >>> together some time ago. These are based on SWIG. In the end I didn't need >>> this at this stage because in the build system command line executables work >>> very well. Its in run time wrappers are very useful. The advantage with SWIG >>> is that the much of the same work will also contribute to C#, Java, Perl >>> wrappers as well. In my experience the most important were Java wrappers to >>> help produce a library for Android. I have no experience with C# and moved >>> to Python from Perl so only use Perl in legacy code ;-). >>> >>> So some questions to consider: >>> >>> 1. Why is python wrapping required for training. using sys.Process to run >>> command lines, structured output directories etc mirrors the current Perl >>> recipes, what is the added benefit in this case? >> Well bash and Perl is the current scripting language for Kaldi. For example >> I prefer to use Python instead of both of them. >> >>> 2. If its for run time decoding shouldn't we create a cross platfom C >>> API? Perhaps things have changed but C++ APIs were never cross compiler >>> compatible in the past so you couldn't do stuff like compile using gnu and >>> link in MSN. With a C interface you can distribute libraries. But I am >>> possibly out of date on this. >> Well, I tried that and I gave it up since Kaldi nicely uses OpenFST and I >> was not able to wrap OpenFST with just plain C (It may be possible). >> I used Cython and pyfst mainly because pyfst solved for me wrapping up >> OpenFST and I am really glad that 99% of wrapping OpenFST templates was >> carried out by somebody else (Victor Chahuneau). >>> >>> 3. If 2 is correct shouldn't we define our API and wrap that? Producing a >>> formal list of functionality that should be exposed to things like client >>> and server applications? >>> >>> >>> I would encourage some care here. Unconstrained wrapping can lead to >>> systems which HAVE to use the scripting language (We can already see how >>> difficult it is to move away from the Perl scripting if you wish to). Also >>> never, never, never reverse wrap (i.e. call python from within C++), yes it >>> can be done but that way lays madness. >>> >>> v best >>> >>> Matthew >>> >>> >>> On Thu, Dec 18, 2014 at 11:37 PM, Daniel Povey <dp...@gm...> wrote: >>>> Jan- >>>> I haven't seen any objections to your setup. I'd say we should plan >>>> to include it in Kaldi at some point (e.g. within the next few >>>> months), but in the meantime hopefully you can continue to work on it, >>>> and maybe come up with some other examples of how it's useful to do >>>> the interfacing with Python- e.g. some kind of application level or >>>> service-level thing? >>>> Dan >>>> >>>> >>>> On Sat, Dec 13, 2014 at 4:01 PM, Yajie Miao <yaj...@gm...> wrote: >>>>> Hi Jan, >>>>> This is very nice work! In our PDNN toolkit, we also have simple python >>>>> wrappers to read and write Kaldi features, mainly for DNN training. >>>>> Your >>>>> implementation looks like a more comprehensive version. >>>>> >>>>> Do you have the functions/commands to do feature splicing? I ask this >>>>> because we found doing splicing on the fly with Python highly >>>>> expensive. >>>>> That's why we still stick to PFiles instead of Kaldi features (.scp >>>>> .ark) >>>>> for DNN triaining. I am very interested to know the efficiency of your >>>>> splicing implementation. >>>>> >>>>> Thanks, >>>>> Yajie >>>>> >>>>> On Sat, Dec 13, 2014 at 5:59 PM, Daniel Povey <dp...@gm...> wrote: >>>>>> OK, thanks. >>>>>> cc'ing Yajie in case he wants to comment. >>>>>> Dan >>>>>> >>>>>> >>>>>> On Sat, Dec 13, 2014 at 2:31 PM, Jan Chorowski >>>>>> <jan...@gm...> >>>>>> wrote: >>>>>>> Hi All, >>>>>>> >>>>>>> the wrapper is built during Kaldi compilation. I build it using >>>>>>> provided >>>>>>> Makefile. The build depends on: >>>>>>> 1. Python and numpy (by default it queries the python interpreter >>>>>>> found >>>>>>> on >>>>>>> the path for header file location) >>>>>>> 2. Boost with Boost::Python library. It is quite heavy to build, but >>>>>>> most >>>>>>> Linux distributions ship it. Boost python doesn't require any code >>>>>>> generation steps, the wrapper is defined in a normal c++ code file. >>>>>>> >>>>>>> During build Python and Boost libraries and Kaldi object files are >>>>>>> linked >>>>>>> into a CPython extention module, >>>>>>> kaldi/src/python/kaldi_io_internal.so. >>>>>>> It >>>>>>> works with both static and shared Kaldi builds. Further usage >>>>>>> requires >>>>>>> that >>>>>>> python finds kaldi_io.py and kaldi_io_internal.so on the PYTHONPATH >>>>>>> - it >>>>>>> can >>>>>>> be for example added to the PYTHONPATH variable in the path.sh >>>>>>> script of >>>>>>> a >>>>>>> recipe. >>>>>>> >>>>>>> Jan >>>>>>> >>>>>>> >>>>>>> On 12/13/2014 3:33 PM, Daniel Povey wrote: >>>>>>>> Also, Jan- could you send us an email explaining how this works- >>>>>>>> How does Python "see" the C++ headers? Do you have to invoke >>>>>>>> some >>>>>>>> special program, like swig? Do you have to write some special kind >>>>>>>> of >>>>>>>> header that shows how the C++ objects are to be interpreted by >>>>>>>> python? >>>>>>>> A brief example would be helpful, if so. >>>>>>>> How is the resulting program linked, if at all? If you require >>>>>>>> functions C++ libraries, are these obtained from the .a or .so >>>>>>>> files >>>>>>>> at runtime, or compiled into some kind of executable-like blob at >>>>>>>> compile time? Does your framework require that Kaldi be compiled >>>>>>>> using dynamic (.so) libraries? >>>>>>>> >>>>>>>> Dan >>>>>>>> >>>>>>>> >>>>>>>> On Sat, Dec 13, 2014 at 12:04 PM, Jan Chorowski >>>>>>>> <jan...@gm...> >>>>>>>> wrote: >>>>>>>>> Hello Dan, >>>>>>>>> >>>>>>>>> thank you for the comments. I tried to make it in the Kaldi >>>>>>>>> spirit, >>>>>>>>> consistency is important. Of course, the scripts can be removed >>>>>>>>> and >>>>>>>>> replaced >>>>>>>>> with some more useful examples. I don't have too much experience >>>>>>>>> with >>>>>>>>> bridging Python to C++, so any critique on the wrappers and the >>>>>>>>> approach >>>>>>>>> taken is welcome. >>>>>>>>> >>>>>>>>> Jan >>>>>>>>> >>>>>>>>> >>>>>>>>> On 12/13/2014 2:55 PM, Daniel Povey wrote: >>>>>>>>>> Hi all. >>>>>>>>>> From a first look, it does look very impressive, and nicely >>>>>>>>>> documented. >>>>>>>>>> I would appreciate it if people on the list who have Python >>>>>>>>>> experience >>>>>>>>>> would comment on this- you can either reply to this thread, or to >>>>>>>>>> me. >>>>>>>>>> I don't know if this has been done in the "natural" way, or if >>>>>>>>>> there >>>>>>>>>> is some reason why people in the future will say, "why did you do >>>>>>>>>> it >>>>>>>>>> this way, you should have done XXX". >>>>>>>>>> >>>>>>>>>> Jan: >>>>>>>>>> in the scripts/ directory you seem to have some examples of how >>>>>>>>>> you >>>>>>>>>> can create python programs that behave very much like Kaldi >>>>>>>>>> command-line programs, using your framework. This is very >>>>>>>>>> useful. >>>>>>>>>> However, the programs >>>>>>>>>> apply-global-cmvn.py >>>>>>>>>> compute-global-cmvn-stats.py >>>>>>>>>> are perhaps a little confusing because they provide the same >>>>>>>>>> functionality that you could get with "compute-cmvn-stats -> >>>>>>>>>> matrix-sum" and "apply-cmvn" on the output of that command; and >>>>>>>>>> they >>>>>>>>>> do so using different formats for the CMVN information. I know >>>>>>>>>> the >>>>>>>>>> format of storing the CMVN stats in a two-row matrix is perhaps >>>>>>>>>> not >>>>>>>>>> perfectly ideal, but it's a standard within Kaldi and it would be >>>>>>>>>> confusing to deviate from that standard. >>>>>>>>>> Of course, this is a very minor issue that doesn't affect the >>>>>>>>>> validity >>>>>>>>>> of the framework as a whole. I am just pointing this out; the >>>>>>>>>> main >>>>>>>>>> discussion should be about the framework and whether people feel >>>>>>>>>> it's >>>>>>>>>> the "right" way to do this. >>>>>>>>>> >>>>>>>>>> Dan >>>>>>>>>> >>>>>>>>>> On Sat, Dec 13, 2014 at 6:28 AM, Jan Chorowski >>>>>>>>>> <jan...@gm...> >>>>>>>>>> wrote: >>>>>>>>>>> Hi all! >>>>>>>>>>> >>>>>>>>>>> I've written wrappers to access Kaldi data files from within >>>>>>>>>>> Python >>>>>>>>>>> using boost::python (the code is on github >>>>>>>>>>> >>>>>>>>>>> https://github.com/janchorowski/kaldi-git/tree/python/src/python). >>>>>>>>>>> If >>>>>>>>>>> you think this would be an interesting addition please instruct >>>>>>>>>>> me >>>>>>>>>>> how >>>>>>>>>>> to contribute. >>>>>>>>>>> >>>>>>>>>>> Best Regards, >>>>>>>>>>> Jan Chorowski >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> ------------------------------------------------------------------------------ >>>>>>>>>>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT >>>>>>>>>>> Server >>>>>>>>>>> from Actuate! Instantly Supercharge Your Business Reports and >>>>>>>>>>> Dashboards >>>>>>>>>>> with Interactivity, Sharing, Native Excel Exports, App >>>>>>>>>>> Integration & >>>>>>>>>>> more >>>>>>>>>>> Get technology previously reserved for billion-dollar >>>>>>>>>>> corporations, >>>>>>>>>>> FREE >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Kaldi-developers mailing list >>>>>>>>>>> Kal...@li... >>>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/kaldi-developers >>>>>>>>> >>>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >>>> from Actuate! Instantly Supercharge Your Business Reports and Dashboards >>>> with Interactivity, Sharing, Native Excel Exports, App Integration & more >>>> Get technology previously reserved for billion-dollar corporations, FREE >>>> >>>> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >>>> _______________________________________________ >>>> Kaldi-developers mailing list >>>> Kal...@li... >>>> https://lists.sourceforge.net/lists/listinfo/kaldi-developers >>> >>> >>> ------------------------------------------------------------------------------ >>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >>> from Actuate! Instantly Supercharge Your Business Reports and Dashboards >>> with Interactivity, Sharing, Native Excel Exports, App Integration & more >>> Get technology previously reserved for billion-dollar corporations, FREE >>> >>> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >>> _______________________________________________ >>> Kaldi-developers mailing list >>> Kal...@li... >>> https://lists.sourceforge.net/lists/listinfo/kaldi-developers >>> >> >> -- >> Ondřej Plátek, +420 737 758 650, skype:ondrejplatek, ond...@gm... |
|
From: Daniel P. <dp...@gm...> - 2014-12-21 20:36:27
|
Jan, perhaps when you have time you could respond to the above comments on the list? I'm not 100% sure what to do about this. BTW, if we do include this, it will likely be optionally compiled, because I don't want the generic Kaldi compilation to be dependent on boost. Dan On Fri, Dec 19, 2014 at 3:48 AM, Ondrej Platek <ond...@gm...> wrote: > Hi Matthew, > > I made some subjective comments below. > > PS: Note that I like the proposed wrappers, but I am not sure how > boost::python is easy to install on all supported platforms. > > On Fri, Dec 19, 2014 at 9:30 AM, Matthew Aylett <mat...@gm...> > wrote: >> >> Hi >> >> Apologies, I've been snowed under here. >> >> I haven' had a chance to look over your work. I also don't have any views >> on the 'right' way to do it. My thoughts on this are in a previous thread. >> See subject "Using SWIG to wrap kaldi for python" where I discussed this >> with ondrej platek and >> Vassil Panayotov. >> >> In the idlak branch there is an example of python wrappers that I put >> together some time ago. These are based on SWIG. In the end I didn't need >> this at this stage because in the build system command line executables work >> very well. Its in run time wrappers are very useful. The advantage with SWIG >> is that the much of the same work will also contribute to C#, Java, Perl >> wrappers as well. In my experience the most important were Java wrappers to >> help produce a library for Android. I have no experience with C# and moved >> to Python from Perl so only use Perl in legacy code ;-). >> >> So some questions to consider: >> >> 1. Why is python wrapping required for training. using sys.Process to run >> command lines, structured output directories etc mirrors the current Perl >> recipes, what is the added benefit in this case? > > Well bash and Perl is the current scripting language for Kaldi. For example > I prefer to use Python instead of both of them. > >> >> 2. If its for run time decoding shouldn't we create a cross platfom C >> API? Perhaps things have changed but C++ APIs were never cross compiler >> compatible in the past so you couldn't do stuff like compile using gnu and >> link in MSN. With a C interface you can distribute libraries. But I am >> possibly out of date on this. > > Well, I tried that and I gave it up since Kaldi nicely uses OpenFST and I > was not able to wrap OpenFST with just plain C (It may be possible). > I used Cython and pyfst mainly because pyfst solved for me wrapping up > OpenFST and I am really glad that 99% of wrapping OpenFST templates was > carried out by somebody else (Victor Chahuneau). >> >> >> 3. If 2 is correct shouldn't we define our API and wrap that? Producing a >> formal list of functionality that should be exposed to things like client >> and server applications? >> >> >> I would encourage some care here. Unconstrained wrapping can lead to >> systems which HAVE to use the scripting language (We can already see how >> difficult it is to move away from the Perl scripting if you wish to). Also >> never, never, never reverse wrap (i.e. call python from within C++), yes it >> can be done but that way lays madness. >> >> v best >> >> Matthew >> >> >> On Thu, Dec 18, 2014 at 11:37 PM, Daniel Povey <dp...@gm...> wrote: >>> >>> Jan- >>> I haven't seen any objections to your setup. I'd say we should plan >>> to include it in Kaldi at some point (e.g. within the next few >>> months), but in the meantime hopefully you can continue to work on it, >>> and maybe come up with some other examples of how it's useful to do >>> the interfacing with Python- e.g. some kind of application level or >>> service-level thing? >>> Dan >>> >>> >>> On Sat, Dec 13, 2014 at 4:01 PM, Yajie Miao <yaj...@gm...> wrote: >>> > Hi Jan, >>> > This is very nice work! In our PDNN toolkit, we also have simple python >>> > wrappers to read and write Kaldi features, mainly for DNN training. >>> > Your >>> > implementation looks like a more comprehensive version. >>> > >>> > Do you have the functions/commands to do feature splicing? I ask this >>> > because we found doing splicing on the fly with Python highly >>> > expensive. >>> > That's why we still stick to PFiles instead of Kaldi features (.scp >>> > .ark) >>> > for DNN triaining. I am very interested to know the efficiency of your >>> > splicing implementation. >>> > >>> > Thanks, >>> > Yajie >>> > >>> > On Sat, Dec 13, 2014 at 5:59 PM, Daniel Povey <dp...@gm...> wrote: >>> >> >>> >> OK, thanks. >>> >> cc'ing Yajie in case he wants to comment. >>> >> Dan >>> >> >>> >> >>> >> On Sat, Dec 13, 2014 at 2:31 PM, Jan Chorowski >>> >> <jan...@gm...> >>> >> wrote: >>> >> > Hi All, >>> >> > >>> >> > the wrapper is built during Kaldi compilation. I build it using >>> >> > provided >>> >> > Makefile. The build depends on: >>> >> > 1. Python and numpy (by default it queries the python interpreter >>> >> > found >>> >> > on >>> >> > the path for header file location) >>> >> > 2. Boost with Boost::Python library. It is quite heavy to build, but >>> >> > most >>> >> > Linux distributions ship it. Boost python doesn't require any code >>> >> > generation steps, the wrapper is defined in a normal c++ code file. >>> >> > >>> >> > During build Python and Boost libraries and Kaldi object files are >>> >> > linked >>> >> > into a CPython extention module, >>> >> > kaldi/src/python/kaldi_io_internal.so. >>> >> > It >>> >> > works with both static and shared Kaldi builds. Further usage >>> >> > requires >>> >> > that >>> >> > python finds kaldi_io.py and kaldi_io_internal.so on the PYTHONPATH >>> >> > - it >>> >> > can >>> >> > be for example added to the PYTHONPATH variable in the path.sh >>> >> > script of >>> >> > a >>> >> > recipe. >>> >> > >>> >> > Jan >>> >> > >>> >> > >>> >> > On 12/13/2014 3:33 PM, Daniel Povey wrote: >>> >> >> >>> >> >> Also, Jan- could you send us an email explaining how this works- >>> >> >> How does Python "see" the C++ headers? Do you have to invoke >>> >> >> some >>> >> >> special program, like swig? Do you have to write some special kind >>> >> >> of >>> >> >> header that shows how the C++ objects are to be interpreted by >>> >> >> python? >>> >> >> A brief example would be helpful, if so. >>> >> >> How is the resulting program linked, if at all? If you require >>> >> >> functions C++ libraries, are these obtained from the .a or .so >>> >> >> files >>> >> >> at runtime, or compiled into some kind of executable-like blob at >>> >> >> compile time? Does your framework require that Kaldi be compiled >>> >> >> using dynamic (.so) libraries? >>> >> >> >>> >> >> Dan >>> >> >> >>> >> >> >>> >> >> On Sat, Dec 13, 2014 at 12:04 PM, Jan Chorowski >>> >> >> <jan...@gm...> >>> >> >> wrote: >>> >> >>> >>> >> >>> Hello Dan, >>> >> >>> >>> >> >>> thank you for the comments. I tried to make it in the Kaldi >>> >> >>> spirit, >>> >> >>> consistency is important. Of course, the scripts can be removed >>> >> >>> and >>> >> >>> replaced >>> >> >>> with some more useful examples. I don't have too much experience >>> >> >>> with >>> >> >>> bridging Python to C++, so any critique on the wrappers and the >>> >> >>> approach >>> >> >>> taken is welcome. >>> >> >>> >>> >> >>> Jan >>> >> >>> >>> >> >>> >>> >> >>> On 12/13/2014 2:55 PM, Daniel Povey wrote: >>> >> >>>> >>> >> >>>> Hi all. >>> >> >>>> From a first look, it does look very impressive, and nicely >>> >> >>>> documented. >>> >> >>>> I would appreciate it if people on the list who have Python >>> >> >>>> experience >>> >> >>>> would comment on this- you can either reply to this thread, or to >>> >> >>>> me. >>> >> >>>> I don't know if this has been done in the "natural" way, or if >>> >> >>>> there >>> >> >>>> is some reason why people in the future will say, "why did you do >>> >> >>>> it >>> >> >>>> this way, you should have done XXX". >>> >> >>>> >>> >> >>>> Jan: >>> >> >>>> in the scripts/ directory you seem to have some examples of how >>> >> >>>> you >>> >> >>>> can create python programs that behave very much like Kaldi >>> >> >>>> command-line programs, using your framework. This is very >>> >> >>>> useful. >>> >> >>>> However, the programs >>> >> >>>> apply-global-cmvn.py >>> >> >>>> compute-global-cmvn-stats.py >>> >> >>>> are perhaps a little confusing because they provide the same >>> >> >>>> functionality that you could get with "compute-cmvn-stats -> >>> >> >>>> matrix-sum" and "apply-cmvn" on the output of that command; and >>> >> >>>> they >>> >> >>>> do so using different formats for the CMVN information. I know >>> >> >>>> the >>> >> >>>> format of storing the CMVN stats in a two-row matrix is perhaps >>> >> >>>> not >>> >> >>>> perfectly ideal, but it's a standard within Kaldi and it would be >>> >> >>>> confusing to deviate from that standard. >>> >> >>>> Of course, this is a very minor issue that doesn't affect the >>> >> >>>> validity >>> >> >>>> of the framework as a whole. I am just pointing this out; the >>> >> >>>> main >>> >> >>>> discussion should be about the framework and whether people feel >>> >> >>>> it's >>> >> >>>> the "right" way to do this. >>> >> >>>> >>> >> >>>> Dan >>> >> >>>> >>> >> >>>> On Sat, Dec 13, 2014 at 6:28 AM, Jan Chorowski >>> >> >>>> <jan...@gm...> >>> >> >>>> wrote: >>> >> >>>>> >>> >> >>>>> Hi all! >>> >> >>>>> >>> >> >>>>> I've written wrappers to access Kaldi data files from within >>> >> >>>>> Python >>> >> >>>>> using boost::python (the code is on github >>> >> >>>>> >>> >> >>>>> https://github.com/janchorowski/kaldi-git/tree/python/src/python). >>> >> >>>>> If >>> >> >>>>> you think this would be an interesting addition please instruct >>> >> >>>>> me >>> >> >>>>> how >>> >> >>>>> to contribute. >>> >> >>>>> >>> >> >>>>> Best Regards, >>> >> >>>>> Jan Chorowski >>> >> >>>>> >>> >> >>>>> >>> >> >>>>> >>> >> >>>>> >>> >> >>>>> >>> >> >>>>> >>> >> >>>>> ------------------------------------------------------------------------------ >>> >> >>>>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT >>> >> >>>>> Server >>> >> >>>>> from Actuate! Instantly Supercharge Your Business Reports and >>> >> >>>>> Dashboards >>> >> >>>>> with Interactivity, Sharing, Native Excel Exports, App >>> >> >>>>> Integration & >>> >> >>>>> more >>> >> >>>>> Get technology previously reserved for billion-dollar >>> >> >>>>> corporations, >>> >> >>>>> FREE >>> >> >>>>> >>> >> >>>>> >>> >> >>>>> >>> >> >>>>> >>> >> >>>>> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >>> >> >>>>> _______________________________________________ >>> >> >>>>> Kaldi-developers mailing list >>> >> >>>>> Kal...@li... >>> >> >>>>> https://lists.sourceforge.net/lists/listinfo/kaldi-developers >>> >> >>> >>> >> >>> >>> >> > >>> > >>> > >>> >>> >>> ------------------------------------------------------------------------------ >>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >>> from Actuate! Instantly Supercharge Your Business Reports and Dashboards >>> with Interactivity, Sharing, Native Excel Exports, App Integration & more >>> Get technology previously reserved for billion-dollar corporations, FREE >>> >>> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >>> _______________________________________________ >>> Kaldi-developers mailing list >>> Kal...@li... >>> https://lists.sourceforge.net/lists/listinfo/kaldi-developers >> >> >> >> ------------------------------------------------------------------------------ >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >> from Actuate! Instantly Supercharge Your Business Reports and Dashboards >> with Interactivity, Sharing, Native Excel Exports, App Integration & more >> Get technology previously reserved for billion-dollar corporations, FREE >> >> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >> _______________________________________________ >> Kaldi-developers mailing list >> Kal...@li... >> https://lists.sourceforge.net/lists/listinfo/kaldi-developers >> > > > -- > Ondřej Plátek, +420 737 758 650, skype:ondrejplatek, ond...@gm... |
|
From: Ondrej P. <ond...@gm...> - 2014-12-19 11:48:20
|
Hi Matthew, I made some subjective comments below. PS: Note that I like the proposed wrappers, but I am not sure how boost::python is easy to install on all supported platforms. On Fri, Dec 19, 2014 at 9:30 AM, Matthew Aylett <mat...@gm...> wrote: > > Hi > > Apologies, I've been snowed under here. > > I haven' had a chance to look over your work. I also don't have any views > on the 'right' way to do it. My thoughts on this are in a previous thread. > See subject "Using SWIG to wrap kaldi for python" where I discussed this > with ondrej platek and > Vassil Panayotov. > > In the idlak branch there is an example of python wrappers that I put > together some time ago. These are based on SWIG. In the end I didn't need > this at this stage because in the build system command line executables > work very well. Its in run time wrappers are very useful. The advantage > with SWIG is that the much of the same work will also contribute to C#, > Java, Perl wrappers as well. In my experience the most important were Java > wrappers to help produce a library for Android. I have no experience with > C# and moved to Python from Perl so only use Perl in legacy code ;-). > > So some questions to consider: > > 1. Why is python wrapping required for training. using sys.Process to run > command lines, structured output directories etc mirrors the current Perl > recipes, what is the added benefit in this case? > Well bash and Perl is the current scripting language for Kaldi. For example I prefer to use Python instead of both of them. > 2. If its for run time decoding shouldn't we create a cross platfom C > API? Perhaps things have changed but C++ APIs were never cross compiler > compatible in the past so you couldn't do stuff like compile using gnu and > link in MSN. With a C interface you can distribute libraries. But I am > possibly out of date on this. > Well, I tried that and I gave it up since Kaldi nicely uses OpenFST and I was not able to wrap OpenFST with just plain C (It may be possible). I used Cython and pyfst mainly because pyfst solved for me wrapping up OpenFST and I am really glad that 99% of wrapping OpenFST templates was carried out by somebody else (Victor Chahuneau). > > 3. If 2 is correct shouldn't we define our API and wrap that? Producing a > formal list of functionality that should be exposed to things like client > and server applications? > > I would encourage some care here. Unconstrained wrapping can lead to > systems which HAVE to use the scripting language (We can already see how > difficult it is to move away from the Perl scripting if you wish to). Also > never, never, never reverse wrap (i.e. call python from within C++), yes it > can be done but that way lays madness. > > v best > > Matthew > > > On Thu, Dec 18, 2014 at 11:37 PM, Daniel Povey <dp...@gm...> wrote: >> >> Jan- >> I haven't seen any objections to your setup. I'd say we should plan >> to include it in Kaldi at some point (e.g. within the next few >> months), but in the meantime hopefully you can continue to work on it, >> and maybe come up with some other examples of how it's useful to do >> the interfacing with Python- e.g. some kind of application level or >> service-level thing? >> Dan >> >> >> On Sat, Dec 13, 2014 at 4:01 PM, Yajie Miao <yaj...@gm...> wrote: >> > Hi Jan, >> > This is very nice work! In our PDNN toolkit, we also have simple python >> > wrappers to read and write Kaldi features, mainly for DNN training. Your >> > implementation looks like a more comprehensive version. >> > >> > Do you have the functions/commands to do feature splicing? I ask this >> > because we found doing splicing on the fly with Python highly expensive. >> > That's why we still stick to PFiles instead of Kaldi features (.scp >> .ark) >> > for DNN triaining. I am very interested to know the efficiency of your >> > splicing implementation. >> > >> > Thanks, >> > Yajie >> > >> > On Sat, Dec 13, 2014 at 5:59 PM, Daniel Povey <dp...@gm...> wrote: >> >> >> >> OK, thanks. >> >> cc'ing Yajie in case he wants to comment. >> >> Dan >> >> >> >> >> >> On Sat, Dec 13, 2014 at 2:31 PM, Jan Chorowski < >> jan...@gm...> >> >> wrote: >> >> > Hi All, >> >> > >> >> > the wrapper is built during Kaldi compilation. I build it using >> provided >> >> > Makefile. The build depends on: >> >> > 1. Python and numpy (by default it queries the python interpreter >> found >> >> > on >> >> > the path for header file location) >> >> > 2. Boost with Boost::Python library. It is quite heavy to build, but >> >> > most >> >> > Linux distributions ship it. Boost python doesn't require any code >> >> > generation steps, the wrapper is defined in a normal c++ code file. >> >> > >> >> > During build Python and Boost libraries and Kaldi object files are >> >> > linked >> >> > into a CPython extention module, >> kaldi/src/python/kaldi_io_internal.so. >> >> > It >> >> > works with both static and shared Kaldi builds. Further usage >> requires >> >> > that >> >> > python finds kaldi_io.py and kaldi_io_internal.so on the PYTHONPATH >> - it >> >> > can >> >> > be for example added to the PYTHONPATH variable in the path.sh >> script of >> >> > a >> >> > recipe. >> >> > >> >> > Jan >> >> > >> >> > >> >> > On 12/13/2014 3:33 PM, Daniel Povey wrote: >> >> >> >> >> >> Also, Jan- could you send us an email explaining how this works- >> >> >> How does Python "see" the C++ headers? Do you have to invoke >> some >> >> >> special program, like swig? Do you have to write some special kind >> of >> >> >> header that shows how the C++ objects are to be interpreted by >> python? >> >> >> A brief example would be helpful, if so. >> >> >> How is the resulting program linked, if at all? If you require >> >> >> functions C++ libraries, are these obtained from the .a or .so files >> >> >> at runtime, or compiled into some kind of executable-like blob at >> >> >> compile time? Does your framework require that Kaldi be compiled >> >> >> using dynamic (.so) libraries? >> >> >> >> >> >> Dan >> >> >> >> >> >> >> >> >> On Sat, Dec 13, 2014 at 12:04 PM, Jan Chorowski >> >> >> <jan...@gm...> >> >> >> wrote: >> >> >>> >> >> >>> Hello Dan, >> >> >>> >> >> >>> thank you for the comments. I tried to make it in the Kaldi spirit, >> >> >>> consistency is important. Of course, the scripts can be removed and >> >> >>> replaced >> >> >>> with some more useful examples. I don't have too much experience >> with >> >> >>> bridging Python to C++, so any critique on the wrappers and the >> >> >>> approach >> >> >>> taken is welcome. >> >> >>> >> >> >>> Jan >> >> >>> >> >> >>> >> >> >>> On 12/13/2014 2:55 PM, Daniel Povey wrote: >> >> >>>> >> >> >>>> Hi all. >> >> >>>> From a first look, it does look very impressive, and nicely >> >> >>>> documented. >> >> >>>> I would appreciate it if people on the list who have Python >> >> >>>> experience >> >> >>>> would comment on this- you can either reply to this thread, or to >> me. >> >> >>>> I don't know if this has been done in the "natural" way, or if >> there >> >> >>>> is some reason why people in the future will say, "why did you do >> it >> >> >>>> this way, you should have done XXX". >> >> >>>> >> >> >>>> Jan: >> >> >>>> in the scripts/ directory you seem to have some examples of how >> you >> >> >>>> can create python programs that behave very much like Kaldi >> >> >>>> command-line programs, using your framework. This is very useful. >> >> >>>> However, the programs >> >> >>>> apply-global-cmvn.py >> >> >>>> compute-global-cmvn-stats.py >> >> >>>> are perhaps a little confusing because they provide the same >> >> >>>> functionality that you could get with "compute-cmvn-stats -> >> >> >>>> matrix-sum" and "apply-cmvn" on the output of that command; and >> they >> >> >>>> do so using different formats for the CMVN information. I know >> the >> >> >>>> format of storing the CMVN stats in a two-row matrix is perhaps >> not >> >> >>>> perfectly ideal, but it's a standard within Kaldi and it would be >> >> >>>> confusing to deviate from that standard. >> >> >>>> Of course, this is a very minor issue that doesn't affect the >> >> >>>> validity >> >> >>>> of the framework as a whole. I am just pointing this out; the >> main >> >> >>>> discussion should be about the framework and whether people feel >> it's >> >> >>>> the "right" way to do this. >> >> >>>> >> >> >>>> Dan >> >> >>>> >> >> >>>> On Sat, Dec 13, 2014 at 6:28 AM, Jan Chorowski >> >> >>>> <jan...@gm...> >> >> >>>> wrote: >> >> >>>>> >> >> >>>>> Hi all! >> >> >>>>> >> >> >>>>> I've written wrappers to access Kaldi data files from within >> Python >> >> >>>>> using boost::python (the code is on github >> >> >>>>> https://github.com/janchorowski/kaldi-git/tree/python/src/python >> ). >> >> >>>>> If >> >> >>>>> you think this would be an interesting addition please instruct >> me >> >> >>>>> how >> >> >>>>> to contribute. >> >> >>>>> >> >> >>>>> Best Regards, >> >> >>>>> Jan Chorowski >> >> >>>>> >> >> >>>>> >> >> >>>>> >> >> >>>>> >> >> >>>>> >> >> >>>>> >> ------------------------------------------------------------------------------ >> >> >>>>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >> >> >>>>> from Actuate! Instantly Supercharge Your Business Reports and >> >> >>>>> Dashboards >> >> >>>>> with Interactivity, Sharing, Native Excel Exports, App >> Integration & >> >> >>>>> more >> >> >>>>> Get technology previously reserved for billion-dollar >> corporations, >> >> >>>>> FREE >> >> >>>>> >> >> >>>>> >> >> >>>>> >> >> >>>>> >> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >> >> >>>>> _______________________________________________ >> >> >>>>> Kaldi-developers mailing list >> >> >>>>> Kal...@li... >> >> >>>>> https://lists.sourceforge.net/lists/listinfo/kaldi-developers >> >> >>> >> >> >>> >> >> > >> > >> > >> >> >> ------------------------------------------------------------------------------ >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >> from Actuate! Instantly Supercharge Your Business Reports and Dashboards >> with Interactivity, Sharing, Native Excel Exports, App Integration & more >> Get technology previously reserved for billion-dollar corporations, FREE >> >> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >> _______________________________________________ >> Kaldi-developers mailing list >> Kal...@li... >> https://lists.sourceforge.net/lists/listinfo/kaldi-developers >> > > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > _______________________________________________ > Kaldi-developers mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-developers > > -- Ondřej Plátek, +420 737 758 650, skype:ondrejplatek, ond...@gm... |
|
From: Matthew A. <mat...@gm...> - 2014-12-19 08:30:33
|
Hi Apologies, I've been snowed under here. I haven' had a chance to look over your work. I also don't have any views on the 'right' way to do it. My thoughts on this are in a previous thread. See subject "Using SWIG to wrap kaldi for python" where I discussed this with ondrej platek and Vassil Panayotov. In the idlak branch there is an example of python wrappers that I put together some time ago. These are based on SWIG. In the end I didn't need this at this stage because in the build system command line executables work very well. Its in run time wrappers are very useful. The advantage with SWIG is that the much of the same work will also contribute to C#, Java, Perl wrappers as well. In my experience the most important were Java wrappers to help produce a library for Android. I have no experience with C# and moved to Python from Perl so only use Perl in legacy code ;-). So some questions to consider: 1. Why is python wrapping required for training. using sys.Process to run command lines, structured output directories etc mirrors the current Perl recipes, what is the added benefit in this case? 2. If its for run time decoding shouldn't we create a cross platfom C API? Perhaps things have changed but C++ APIs were never cross compiler compatible in the past so you couldn't do stuff like compile using gnu and link in MSN. With a C interface you can distribute libraries. But I am possibly out of date on this. 3. If 2 is correct shouldn't we define our API and wrap that? Producing a formal list of functionality that should be exposed to things like client and server applications? I would encourage some care here. Unconstrained wrapping can lead to systems which HAVE to use the scripting language (We can already see how difficult it is to move away from the Perl scripting if you wish to). Also never, never, never reverse wrap (i.e. call python from within C++), yes it can be done but that way lays madness. v best Matthew On Thu, Dec 18, 2014 at 11:37 PM, Daniel Povey <dp...@gm...> wrote: > > Jan- > I haven't seen any objections to your setup. I'd say we should plan > to include it in Kaldi at some point (e.g. within the next few > months), but in the meantime hopefully you can continue to work on it, > and maybe come up with some other examples of how it's useful to do > the interfacing with Python- e.g. some kind of application level or > service-level thing? > Dan > > > On Sat, Dec 13, 2014 at 4:01 PM, Yajie Miao <yaj...@gm...> wrote: > > Hi Jan, > > This is very nice work! In our PDNN toolkit, we also have simple python > > wrappers to read and write Kaldi features, mainly for DNN training. Your > > implementation looks like a more comprehensive version. > > > > Do you have the functions/commands to do feature splicing? I ask this > > because we found doing splicing on the fly with Python highly expensive. > > That's why we still stick to PFiles instead of Kaldi features (.scp .ark) > > for DNN triaining. I am very interested to know the efficiency of your > > splicing implementation. > > > > Thanks, > > Yajie > > > > On Sat, Dec 13, 2014 at 5:59 PM, Daniel Povey <dp...@gm...> wrote: > >> > >> OK, thanks. > >> cc'ing Yajie in case he wants to comment. > >> Dan > >> > >> > >> On Sat, Dec 13, 2014 at 2:31 PM, Jan Chorowski <jan...@gm... > > > >> wrote: > >> > Hi All, > >> > > >> > the wrapper is built during Kaldi compilation. I build it using > provided > >> > Makefile. The build depends on: > >> > 1. Python and numpy (by default it queries the python interpreter > found > >> > on > >> > the path for header file location) > >> > 2. Boost with Boost::Python library. It is quite heavy to build, but > >> > most > >> > Linux distributions ship it. Boost python doesn't require any code > >> > generation steps, the wrapper is defined in a normal c++ code file. > >> > > >> > During build Python and Boost libraries and Kaldi object files are > >> > linked > >> > into a CPython extention module, > kaldi/src/python/kaldi_io_internal.so. > >> > It > >> > works with both static and shared Kaldi builds. Further usage requires > >> > that > >> > python finds kaldi_io.py and kaldi_io_internal.so on the PYTHONPATH - > it > >> > can > >> > be for example added to the PYTHONPATH variable in the path.sh script > of > >> > a > >> > recipe. > >> > > >> > Jan > >> > > >> > > >> > On 12/13/2014 3:33 PM, Daniel Povey wrote: > >> >> > >> >> Also, Jan- could you send us an email explaining how this works- > >> >> How does Python "see" the C++ headers? Do you have to invoke some > >> >> special program, like swig? Do you have to write some special kind > of > >> >> header that shows how the C++ objects are to be interpreted by > python? > >> >> A brief example would be helpful, if so. > >> >> How is the resulting program linked, if at all? If you require > >> >> functions C++ libraries, are these obtained from the .a or .so files > >> >> at runtime, or compiled into some kind of executable-like blob at > >> >> compile time? Does your framework require that Kaldi be compiled > >> >> using dynamic (.so) libraries? > >> >> > >> >> Dan > >> >> > >> >> > >> >> On Sat, Dec 13, 2014 at 12:04 PM, Jan Chorowski > >> >> <jan...@gm...> > >> >> wrote: > >> >>> > >> >>> Hello Dan, > >> >>> > >> >>> thank you for the comments. I tried to make it in the Kaldi spirit, > >> >>> consistency is important. Of course, the scripts can be removed and > >> >>> replaced > >> >>> with some more useful examples. I don't have too much experience > with > >> >>> bridging Python to C++, so any critique on the wrappers and the > >> >>> approach > >> >>> taken is welcome. > >> >>> > >> >>> Jan > >> >>> > >> >>> > >> >>> On 12/13/2014 2:55 PM, Daniel Povey wrote: > >> >>>> > >> >>>> Hi all. > >> >>>> From a first look, it does look very impressive, and nicely > >> >>>> documented. > >> >>>> I would appreciate it if people on the list who have Python > >> >>>> experience > >> >>>> would comment on this- you can either reply to this thread, or to > me. > >> >>>> I don't know if this has been done in the "natural" way, or if > there > >> >>>> is some reason why people in the future will say, "why did you do > it > >> >>>> this way, you should have done XXX". > >> >>>> > >> >>>> Jan: > >> >>>> in the scripts/ directory you seem to have some examples of how you > >> >>>> can create python programs that behave very much like Kaldi > >> >>>> command-line programs, using your framework. This is very useful. > >> >>>> However, the programs > >> >>>> apply-global-cmvn.py > >> >>>> compute-global-cmvn-stats.py > >> >>>> are perhaps a little confusing because they provide the same > >> >>>> functionality that you could get with "compute-cmvn-stats -> > >> >>>> matrix-sum" and "apply-cmvn" on the output of that command; and > they > >> >>>> do so using different formats for the CMVN information. I know the > >> >>>> format of storing the CMVN stats in a two-row matrix is perhaps not > >> >>>> perfectly ideal, but it's a standard within Kaldi and it would be > >> >>>> confusing to deviate from that standard. > >> >>>> Of course, this is a very minor issue that doesn't affect the > >> >>>> validity > >> >>>> of the framework as a whole. I am just pointing this out; the main > >> >>>> discussion should be about the framework and whether people feel > it's > >> >>>> the "right" way to do this. > >> >>>> > >> >>>> Dan > >> >>>> > >> >>>> On Sat, Dec 13, 2014 at 6:28 AM, Jan Chorowski > >> >>>> <jan...@gm...> > >> >>>> wrote: > >> >>>>> > >> >>>>> Hi all! > >> >>>>> > >> >>>>> I've written wrappers to access Kaldi data files from within > Python > >> >>>>> using boost::python (the code is on github > >> >>>>> https://github.com/janchorowski/kaldi-git/tree/python/src/python > ). > >> >>>>> If > >> >>>>> you think this would be an interesting addition please instruct me > >> >>>>> how > >> >>>>> to contribute. > >> >>>>> > >> >>>>> Best Regards, > >> >>>>> Jan Chorowski > >> >>>>> > >> >>>>> > >> >>>>> > >> >>>>> > >> >>>>> > >> >>>>> > ------------------------------------------------------------------------------ > >> >>>>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > >> >>>>> from Actuate! Instantly Supercharge Your Business Reports and > >> >>>>> Dashboards > >> >>>>> with Interactivity, Sharing, Native Excel Exports, App > Integration & > >> >>>>> more > >> >>>>> Get technology previously reserved for billion-dollar > corporations, > >> >>>>> FREE > >> >>>>> > >> >>>>> > >> >>>>> > >> >>>>> > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > >> >>>>> _______________________________________________ > >> >>>>> Kaldi-developers mailing list > >> >>>>> Kal...@li... > >> >>>>> https://lists.sourceforge.net/lists/listinfo/kaldi-developers > >> >>> > >> >>> > >> > > > > > > > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > _______________________________________________ > Kaldi-developers mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-developers > |
|
From: Daniel P. <dp...@gm...> - 2014-12-18 23:37:10
|
Jan- I haven't seen any objections to your setup. I'd say we should plan to include it in Kaldi at some point (e.g. within the next few months), but in the meantime hopefully you can continue to work on it, and maybe come up with some other examples of how it's useful to do the interfacing with Python- e.g. some kind of application level or service-level thing? Dan On Sat, Dec 13, 2014 at 4:01 PM, Yajie Miao <yaj...@gm...> wrote: > Hi Jan, > This is very nice work! In our PDNN toolkit, we also have simple python > wrappers to read and write Kaldi features, mainly for DNN training. Your > implementation looks like a more comprehensive version. > > Do you have the functions/commands to do feature splicing? I ask this > because we found doing splicing on the fly with Python highly expensive. > That's why we still stick to PFiles instead of Kaldi features (.scp .ark) > for DNN triaining. I am very interested to know the efficiency of your > splicing implementation. > > Thanks, > Yajie > > On Sat, Dec 13, 2014 at 5:59 PM, Daniel Povey <dp...@gm...> wrote: >> >> OK, thanks. >> cc'ing Yajie in case he wants to comment. >> Dan >> >> >> On Sat, Dec 13, 2014 at 2:31 PM, Jan Chorowski <jan...@gm...> >> wrote: >> > Hi All, >> > >> > the wrapper is built during Kaldi compilation. I build it using provided >> > Makefile. The build depends on: >> > 1. Python and numpy (by default it queries the python interpreter found >> > on >> > the path for header file location) >> > 2. Boost with Boost::Python library. It is quite heavy to build, but >> > most >> > Linux distributions ship it. Boost python doesn't require any code >> > generation steps, the wrapper is defined in a normal c++ code file. >> > >> > During build Python and Boost libraries and Kaldi object files are >> > linked >> > into a CPython extention module, kaldi/src/python/kaldi_io_internal.so. >> > It >> > works with both static and shared Kaldi builds. Further usage requires >> > that >> > python finds kaldi_io.py and kaldi_io_internal.so on the PYTHONPATH - it >> > can >> > be for example added to the PYTHONPATH variable in the path.sh script of >> > a >> > recipe. >> > >> > Jan >> > >> > >> > On 12/13/2014 3:33 PM, Daniel Povey wrote: >> >> >> >> Also, Jan- could you send us an email explaining how this works- >> >> How does Python "see" the C++ headers? Do you have to invoke some >> >> special program, like swig? Do you have to write some special kind of >> >> header that shows how the C++ objects are to be interpreted by python? >> >> A brief example would be helpful, if so. >> >> How is the resulting program linked, if at all? If you require >> >> functions C++ libraries, are these obtained from the .a or .so files >> >> at runtime, or compiled into some kind of executable-like blob at >> >> compile time? Does your framework require that Kaldi be compiled >> >> using dynamic (.so) libraries? >> >> >> >> Dan >> >> >> >> >> >> On Sat, Dec 13, 2014 at 12:04 PM, Jan Chorowski >> >> <jan...@gm...> >> >> wrote: >> >>> >> >>> Hello Dan, >> >>> >> >>> thank you for the comments. I tried to make it in the Kaldi spirit, >> >>> consistency is important. Of course, the scripts can be removed and >> >>> replaced >> >>> with some more useful examples. I don't have too much experience with >> >>> bridging Python to C++, so any critique on the wrappers and the >> >>> approach >> >>> taken is welcome. >> >>> >> >>> Jan >> >>> >> >>> >> >>> On 12/13/2014 2:55 PM, Daniel Povey wrote: >> >>>> >> >>>> Hi all. >> >>>> From a first look, it does look very impressive, and nicely >> >>>> documented. >> >>>> I would appreciate it if people on the list who have Python >> >>>> experience >> >>>> would comment on this- you can either reply to this thread, or to me. >> >>>> I don't know if this has been done in the "natural" way, or if there >> >>>> is some reason why people in the future will say, "why did you do it >> >>>> this way, you should have done XXX". >> >>>> >> >>>> Jan: >> >>>> in the scripts/ directory you seem to have some examples of how you >> >>>> can create python programs that behave very much like Kaldi >> >>>> command-line programs, using your framework. This is very useful. >> >>>> However, the programs >> >>>> apply-global-cmvn.py >> >>>> compute-global-cmvn-stats.py >> >>>> are perhaps a little confusing because they provide the same >> >>>> functionality that you could get with "compute-cmvn-stats -> >> >>>> matrix-sum" and "apply-cmvn" on the output of that command; and they >> >>>> do so using different formats for the CMVN information. I know the >> >>>> format of storing the CMVN stats in a two-row matrix is perhaps not >> >>>> perfectly ideal, but it's a standard within Kaldi and it would be >> >>>> confusing to deviate from that standard. >> >>>> Of course, this is a very minor issue that doesn't affect the >> >>>> validity >> >>>> of the framework as a whole. I am just pointing this out; the main >> >>>> discussion should be about the framework and whether people feel it's >> >>>> the "right" way to do this. >> >>>> >> >>>> Dan >> >>>> >> >>>> On Sat, Dec 13, 2014 at 6:28 AM, Jan Chorowski >> >>>> <jan...@gm...> >> >>>> wrote: >> >>>>> >> >>>>> Hi all! >> >>>>> >> >>>>> I've written wrappers to access Kaldi data files from within Python >> >>>>> using boost::python (the code is on github >> >>>>> https://github.com/janchorowski/kaldi-git/tree/python/src/python). >> >>>>> If >> >>>>> you think this would be an interesting addition please instruct me >> >>>>> how >> >>>>> to contribute. >> >>>>> >> >>>>> Best Regards, >> >>>>> Jan Chorowski >> >>>>> >> >>>>> >> >>>>> >> >>>>> >> >>>>> >> >>>>> ------------------------------------------------------------------------------ >> >>>>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >> >>>>> from Actuate! Instantly Supercharge Your Business Reports and >> >>>>> Dashboards >> >>>>> with Interactivity, Sharing, Native Excel Exports, App Integration & >> >>>>> more >> >>>>> Get technology previously reserved for billion-dollar corporations, >> >>>>> FREE >> >>>>> >> >>>>> >> >>>>> >> >>>>> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >> >>>>> _______________________________________________ >> >>>>> Kaldi-developers mailing list >> >>>>> Kal...@li... >> >>>>> https://lists.sourceforge.net/lists/listinfo/kaldi-developers >> >>> >> >>> >> > > > |
|
From: Dogan C. <dog...@us...> - 2014-12-14 19:32:44
|
Hi everyone, OpenFst-1.4/C++11 related changes were committed to Kaldi trunk as of yesterday. I tested the new setup on Mac OS X 10.10 and Ubuntu 12.04 with both gcc and clang. Dan also tested it on his Linux setup. See tools/INSTALL for instructions. Note that you will need a relatively recent compiler with C++11 support. Cheers, Dogan > On Dec 9, 2014, at 8:41 PM, Daniel Povey <dp...@gm...> wrote: > > OK, I tested your patch and it seems to work for the current OpenFst. > > I noticed another issue-- you seem to be using openfst-1.3.2 as the > "older" version, but the current Kaldi uses openfst-1.3.4. > This may be why you have to use that new patch to fix begin_->begin > and end_->end in IntervalSet. I think perhaps you should update your > setup to use openfst-1.3.4 as the older supported version. > > I will check in a fix for the sgmm-*-multi-test separately. > > Something else is that it's not very nice to have to specify > OPENFST_VERSION=1.4.2 on the command line to 'make'. If someone were > to do this, and later type "make" again, it would go back to the older > version. I think it might be better to instruct people to uncomment a > line in the Makefile if they want this. Any comment? Adding a > 'configure' script at this level seems too heavyweight for now. > > Dan > > > On Tue, Dec 9, 2014 at 10:33 PM, Dogan Can <dog...@us...> wrote: >> Hi Dan >> >> I’m attaching a new patch against kaldi-trunk. Please see my inline answers >> to your comments. >> >> I also addressed the openfst symbolic linking problem in tools/Makefile that >> was mentioned in your other email. >> >> I tested these changes on Ubuntu 12.04 with gcc-4.6 and clang-3.3. Clang >> build is printing a bunch of benign warnings on Linux (that we are >> suppressing in darwin makefiles with some extra flags). Maybe we should >> extract those flags from darwin makefiles to a common makefile shared on all >> builds. I mean I don’t see any reason why we should not fully support clang >> on Linux. Other than that, everything seems to be in working order on my >> end. >> >> Cheers, >> Dogan >> >> >> >> >> On Dec 9, 2014, at 1:53 PM, Daniel Povey <dp...@gm...> wrote: >> >> Thanks.. >> There are a few minor things I think are worth changing: >> >> - patches should now go not in tools/ but in tools/extras/. >> >> >> Done. I also moved the sequitur.patch into tools/extras and changed its >> build script to use this new location. >> >> >> - I'd prefer it if the test for the OpenFST version and the c++0x flag >> were in the configure script rather than in the makefiles/* (if some >> setups need extra flags you can put a test in the corresponding >> makefile though). This will avoid the need to edit all the >> makefiles/*, and also I prefer kaldi.mk to just have a few simple >> options in it rather than having all kinds of conditional statements. >> Then it's easier to hand-edit if the user feels like it. I'm OK with >> things breaking if someone re-installs OpenFst without rerunning >> "configure", because the concept is that you rerun "configure" any >> time you change something bug. >> >> >> Done. >> >> >> -in util/stl-utils.h, there are redundant #ifdef/#elif blocks, I think >> it would be easier to just do >> #if defined(_MSC_VER) || (__cplusplus > 199711L) >> #include <unordered_map> >> #include <unordered_set> >> using std::unordered_map; >> using std::unordered_set; >> #else >> … >> >> >> It seems the support for these macros is quite a mess at the moment. For >> instance I just noticed that the __cplusplus macro is defined to be 1 in >> gcc-4.6 (default on Ubuntu 12.04) when c++11 support is enabled. I had to >> change that condition to the following one to get things compile on Ubuntu. >> >> if __cplusplus > 199711L || defined(__GXX_EXPERIMENTAL_CXX0X__) >> >> I left redundant #ifdef blocks in place in case we need to further adjust >> these checks in the future. Once these things are somewhat settled we can >> get rid of the redundant cases. What do you think? >> >> - Wouldn't it be less confusing to add the -std=c++11 option instead >> of the -std=c++0x option? >> >> >> -std=c++11 is not supported on gcc-4.6. >> >> - For me, on Linux, the kaldi.mk did not work out of the box because >> your statement adding the -std=c++0x option was above the statement >> >> CXXFLAGS = -msse -msse2 -Wall -I.. \ >> -pthread \ >> [etc.] >> >> Presumably when you put this test in the configure script instead of >> kaldi.mk, you can add this at the end rather than the beginning of >> kaldi.mk. >> >> >> I fixed this on Ubuntu 12.04 by adding the necessary c++11 flags to >> EXTRA_CXXFLAGS. Could you check if your Linux distro is happy after this >> change? >> >> I am not able to test this on my Mac (OS X 10.7.5). I get this: >> configure: error: in `/Users/danielpovey/kaldi/trunk/tools/openfst-1.4.1': >> configure: error: C++ compiler cannot create executables >> See `config.log' for more details >> make: *** [openfst-1.4.1/Makefile] Error 77 >> and openfst/config.log has nothing interesting. I am guessing it >> means my C++ compiler is out of date. I checked MacPorts and there is >> nothing newer available. [my version is "gcc version 4.2.1 (Based on >> Apple Inc. build 5658) (LLVM build 2336.1.00)”] >> >> >> You need gcc >= 4.6 for c++11 support. You can install the latest gcc (as >> well as older ones if needed) with homebrew. >> >> Anyway, this probably doesn't matter much. It just means you should >> show me what the error was in that failing test so I will know how to >> fix it. >> >> >> I’m attaching an output log. >> >> >> Dan >> >> On Tue, Dec 9, 2014 at 4:36 AM, Dogan Can <dog...@us...> wrote: >> >> Hi Dan >> >> Sure, I’m attaching a patch against the kaldi-trunk. I just made a bunch of >> additional changes to get the failing tests succeed. At the moment, the only >> test failing on Mac OS 10.10 is sgmm/estimate-am-sgmm-multi-test. Could you >> take a look at that? I think it needs a threshold adjustment or something >> like that. >> >> Cheers, >> Dogan >> >> ------------------------------------- >> >> Installation instructions: >> >> cd tools >> make -j 8 OPENFST_VERSION=1.4.1 >> cd ../src >> ./configure >> make -j 8 depend >> make -j 8 >> >> If OPENFST_VERSION is not set during the make call, then it defaults to >> 1.3.2, i.e. current installation instructions remain as they are. >> >> Apart from the script changes and a bunch of conditional includes, the major >> change in this patch is the addition of "util/basic-filebuf.h” which defines >> a modified version of the std::basic_filebuf class from libc++. It allows >> one to create a basic_filebuf object from an existing FILE* handle or file >> descriptor when linking against libc++. This is a fairly large change and >> has the potential to cause some regressions. >> >> By the way, “make -j 8 depend” step produces a bunch of “subframework" >> warnings on Mac OS 10.10 when compiling with GCC. It seems Apple dropped >> support for some previously deprecated stuff and GCC hasn’t caught up yet. >> >> Also note that this patch includes a few minor/cosmetic fixes here and >> there. >> >> >> >> >> On Dec 8, 2014, at 10:39 AM, Daniel Povey <dp...@gm...> wrote: >> >> Dogan, maybe you could attach your patch here and let Simon test it? >> Dan >> >> >> On Mon, Dec 8, 2014 at 1:38 PM, Dogan Can <dog...@us...> wrote: >> >> Hi Dan >> >> It builds on Mac OS X 10.10 both with g++ and clang. I haven't done any >> regression tests though. Also I haven't tested it on Linux yet. >> >> Paul, did you get a chance to test it on your end? >> >> Cheers >> Dogan >> >> — >> Sent from Mailbox >> >> >> On Mon, Dec 8, 2014 at 10:27 AM, Daniel Povey <dp...@gm...> wrote: >> >> >> Paul Dixon has already done some work to get Kaldi to compile with >> OpenFst 1.4 (and C++11), but it is not committed yet. Dogan was going >> to help get it committed, but I don't know how far along this is. >> Dogan, any comment? >> Dan >> >> >> On Mon, Dec 8, 2014 at 12:19 PM, Jan Trmal <af...@ce...> wrote: >> >> The OpenFST 1.4.1 needs latest C++ standard (C++11). I'm not sure which >> of >> the C++11 features it needs, but not all compilers support that standard >> fully, or you have to use relatively recent version. That might be a bit >> of >> inconvenience for some people. Feel free to to experiment with it in >> connection to Kaldi -- there is not specific reason why to avoid it. >> y. >> >> On Mon, Dec 8, 2014 at 8:47 AM, Simon Klüpfel <sim...@gm...> >> wrote: >> >> >> Hi, >> >> I wondered if there exists any version of Kaldi using OpenFST version >> 1.4.1 by default, or if there would be any interest in having this >> option or any reason not to use it?! >> >> I am using Thrax for a rule based grammar used as the language model, >> and the current version requires the latest version of OpenFST. >> >> Is there any interest in reviewing, if I should add this as an option >> to >> the installation of the tools? I.e., adding support for (a patched) >> 1.4.1 version, as well as options for the OpenGrm tools (Thrax and >> perhaps as well NGram Library, if anyone might want to try it out)? >> >> All the best, >> >> Simon >> >> >> >> ------------------------------------------------------------------------------ >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >> from Actuate! Instantly Supercharge Your Business Reports and >> Dashboards >> with Interactivity, Sharing, Native Excel Exports, App Integration & >> more >> Get technology previously reserved for billion-dollar corporations, >> FREE >> >> >> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >> _______________________________________________ >> Kaldi-developers mailing list >> Kal...@li... >> https://lists.sourceforge.net/lists/listinfo/kaldi-developers >> >> >> >> >> >> ------------------------------------------------------------------------------ >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >> from Actuate! Instantly Supercharge Your Business Reports and Dashboards >> with Interactivity, Sharing, Native Excel Exports, App Integration & >> more >> Get technology previously reserved for billion-dollar corporations, FREE >> >> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >> _______________________________________________ >> Kaldi-developers mailing list >> Kal...@li... >> https://lists.sourceforge.net/lists/listinfo/kaldi-developers >> >> >> >> >> >> >> |
|
From: Yajie M. <yaj...@gm...> - 2014-12-14 00:01:21
|
Hi Jan, This is very nice work! In our PDNN toolkit, we also have simple python wrappers to read and write Kaldi features, mainly for DNN training. Your implementation looks like a more comprehensive version. Do you have the functions/commands to do feature splicing? I ask this because we found doing splicing on the fly with Python highly expensive. That's why we still stick to PFiles instead of Kaldi features (.scp .ark) for DNN triaining. I am very interested to know the efficiency of your splicing implementation. Thanks, Yajie On Sat, Dec 13, 2014 at 5:59 PM, Daniel Povey <dp...@gm...> wrote: > OK, thanks. > cc'ing Yajie in case he wants to comment. > Dan > > > On Sat, Dec 13, 2014 at 2:31 PM, Jan Chorowski <jan...@gm...> > wrote: > > Hi All, > > > > the wrapper is built during Kaldi compilation. I build it using provided > > Makefile. The build depends on: > > 1. Python and numpy (by default it queries the python interpreter found > on > > the path for header file location) > > 2. Boost with Boost::Python library. It is quite heavy to build, but most > > Linux distributions ship it. Boost python doesn't require any code > > generation steps, the wrapper is defined in a normal c++ code file. > > > > During build Python and Boost libraries and Kaldi object files are linked > > into a CPython extention module, kaldi/src/python/kaldi_io_internal.so. > It > > works with both static and shared Kaldi builds. Further usage requires > that > > python finds kaldi_io.py and kaldi_io_internal.so on the PYTHONPATH - it > can > > be for example added to the PYTHONPATH variable in the path.sh script of > a > > recipe. > > > > Jan > > > > > > On 12/13/2014 3:33 PM, Daniel Povey wrote: > >> > >> Also, Jan- could you send us an email explaining how this works- > >> How does Python "see" the C++ headers? Do you have to invoke some > >> special program, like swig? Do you have to write some special kind of > >> header that shows how the C++ objects are to be interpreted by python? > >> A brief example would be helpful, if so. > >> How is the resulting program linked, if at all? If you require > >> functions C++ libraries, are these obtained from the .a or .so files > >> at runtime, or compiled into some kind of executable-like blob at > >> compile time? Does your framework require that Kaldi be compiled > >> using dynamic (.so) libraries? > >> > >> Dan > >> > >> > >> On Sat, Dec 13, 2014 at 12:04 PM, Jan Chorowski < > jan...@gm...> > >> wrote: > >>> > >>> Hello Dan, > >>> > >>> thank you for the comments. I tried to make it in the Kaldi spirit, > >>> consistency is important. Of course, the scripts can be removed and > >>> replaced > >>> with some more useful examples. I don't have too much experience with > >>> bridging Python to C++, so any critique on the wrappers and the > approach > >>> taken is welcome. > >>> > >>> Jan > >>> > >>> > >>> On 12/13/2014 2:55 PM, Daniel Povey wrote: > >>>> > >>>> Hi all. > >>>> From a first look, it does look very impressive, and nicely > >>>> documented. > >>>> I would appreciate it if people on the list who have Python experience > >>>> would comment on this- you can either reply to this thread, or to me. > >>>> I don't know if this has been done in the "natural" way, or if there > >>>> is some reason why people in the future will say, "why did you do it > >>>> this way, you should have done XXX". > >>>> > >>>> Jan: > >>>> in the scripts/ directory you seem to have some examples of how you > >>>> can create python programs that behave very much like Kaldi > >>>> command-line programs, using your framework. This is very useful. > >>>> However, the programs > >>>> apply-global-cmvn.py > >>>> compute-global-cmvn-stats.py > >>>> are perhaps a little confusing because they provide the same > >>>> functionality that you could get with "compute-cmvn-stats -> > >>>> matrix-sum" and "apply-cmvn" on the output of that command; and they > >>>> do so using different formats for the CMVN information. I know the > >>>> format of storing the CMVN stats in a two-row matrix is perhaps not > >>>> perfectly ideal, but it's a standard within Kaldi and it would be > >>>> confusing to deviate from that standard. > >>>> Of course, this is a very minor issue that doesn't affect the validity > >>>> of the framework as a whole. I am just pointing this out; the main > >>>> discussion should be about the framework and whether people feel it's > >>>> the "right" way to do this. > >>>> > >>>> Dan > >>>> > >>>> On Sat, Dec 13, 2014 at 6:28 AM, Jan Chorowski < > jan...@gm...> > >>>> wrote: > >>>>> > >>>>> Hi all! > >>>>> > >>>>> I've written wrappers to access Kaldi data files from within Python > >>>>> using boost::python (the code is on github > >>>>> https://github.com/janchorowski/kaldi-git/tree/python/src/python). > If > >>>>> you think this would be an interesting addition please instruct me > how > >>>>> to contribute. > >>>>> > >>>>> Best Regards, > >>>>> Jan Chorowski > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > ------------------------------------------------------------------------------ > >>>>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > >>>>> from Actuate! Instantly Supercharge Your Business Reports and > >>>>> Dashboards > >>>>> with Interactivity, Sharing, Native Excel Exports, App Integration & > >>>>> more > >>>>> Get technology previously reserved for billion-dollar corporations, > >>>>> FREE > >>>>> > >>>>> > >>>>> > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > >>>>> _______________________________________________ > >>>>> Kaldi-developers mailing list > >>>>> Kal...@li... > >>>>> https://lists.sourceforge.net/lists/listinfo/kaldi-developers > >>> > >>> > > > |
|
From: Daniel P. <dp...@gm...> - 2014-12-13 23:00:05
|
OK, thanks. cc'ing Yajie in case he wants to comment. Dan On Sat, Dec 13, 2014 at 2:31 PM, Jan Chorowski <jan...@gm...> wrote: > Hi All, > > the wrapper is built during Kaldi compilation. I build it using provided > Makefile. The build depends on: > 1. Python and numpy (by default it queries the python interpreter found on > the path for header file location) > 2. Boost with Boost::Python library. It is quite heavy to build, but most > Linux distributions ship it. Boost python doesn't require any code > generation steps, the wrapper is defined in a normal c++ code file. > > During build Python and Boost libraries and Kaldi object files are linked > into a CPython extention module, kaldi/src/python/kaldi_io_internal.so. It > works with both static and shared Kaldi builds. Further usage requires that > python finds kaldi_io.py and kaldi_io_internal.so on the PYTHONPATH - it can > be for example added to the PYTHONPATH variable in the path.sh script of a > recipe. > > Jan > > > On 12/13/2014 3:33 PM, Daniel Povey wrote: >> >> Also, Jan- could you send us an email explaining how this works- >> How does Python "see" the C++ headers? Do you have to invoke some >> special program, like swig? Do you have to write some special kind of >> header that shows how the C++ objects are to be interpreted by python? >> A brief example would be helpful, if so. >> How is the resulting program linked, if at all? If you require >> functions C++ libraries, are these obtained from the .a or .so files >> at runtime, or compiled into some kind of executable-like blob at >> compile time? Does your framework require that Kaldi be compiled >> using dynamic (.so) libraries? >> >> Dan >> >> >> On Sat, Dec 13, 2014 at 12:04 PM, Jan Chorowski <jan...@gm...> >> wrote: >>> >>> Hello Dan, >>> >>> thank you for the comments. I tried to make it in the Kaldi spirit, >>> consistency is important. Of course, the scripts can be removed and >>> replaced >>> with some more useful examples. I don't have too much experience with >>> bridging Python to C++, so any critique on the wrappers and the approach >>> taken is welcome. >>> >>> Jan >>> >>> >>> On 12/13/2014 2:55 PM, Daniel Povey wrote: >>>> >>>> Hi all. >>>> From a first look, it does look very impressive, and nicely >>>> documented. >>>> I would appreciate it if people on the list who have Python experience >>>> would comment on this- you can either reply to this thread, or to me. >>>> I don't know if this has been done in the "natural" way, or if there >>>> is some reason why people in the future will say, "why did you do it >>>> this way, you should have done XXX". >>>> >>>> Jan: >>>> in the scripts/ directory you seem to have some examples of how you >>>> can create python programs that behave very much like Kaldi >>>> command-line programs, using your framework. This is very useful. >>>> However, the programs >>>> apply-global-cmvn.py >>>> compute-global-cmvn-stats.py >>>> are perhaps a little confusing because they provide the same >>>> functionality that you could get with "compute-cmvn-stats -> >>>> matrix-sum" and "apply-cmvn" on the output of that command; and they >>>> do so using different formats for the CMVN information. I know the >>>> format of storing the CMVN stats in a two-row matrix is perhaps not >>>> perfectly ideal, but it's a standard within Kaldi and it would be >>>> confusing to deviate from that standard. >>>> Of course, this is a very minor issue that doesn't affect the validity >>>> of the framework as a whole. I am just pointing this out; the main >>>> discussion should be about the framework and whether people feel it's >>>> the "right" way to do this. >>>> >>>> Dan >>>> >>>> On Sat, Dec 13, 2014 at 6:28 AM, Jan Chorowski <jan...@gm...> >>>> wrote: >>>>> >>>>> Hi all! >>>>> >>>>> I've written wrappers to access Kaldi data files from within Python >>>>> using boost::python (the code is on github >>>>> https://github.com/janchorowski/kaldi-git/tree/python/src/python). If >>>>> you think this would be an interesting addition please instruct me how >>>>> to contribute. >>>>> >>>>> Best Regards, >>>>> Jan Chorowski >>>>> >>>>> >>>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >>>>> from Actuate! Instantly Supercharge Your Business Reports and >>>>> Dashboards >>>>> with Interactivity, Sharing, Native Excel Exports, App Integration & >>>>> more >>>>> Get technology previously reserved for billion-dollar corporations, >>>>> FREE >>>>> >>>>> >>>>> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >>>>> _______________________________________________ >>>>> Kaldi-developers mailing list >>>>> Kal...@li... >>>>> https://lists.sourceforge.net/lists/listinfo/kaldi-developers >>> >>> > |
23 messages has been excluded from this view by a project administrator.
