Menu
▾
▴
Re: [Kaldi-users] data preparation problem
|
From: Simon K. <sim...@gm...> - 2014-07-21 15:13:15
|
Hi, wasn't gcc 4.8.2. the buggy version? If I remember right, it had caused crashes before, sometimes not reproducible and at different stages of the process. Downgrading to 4.7 helped for me. @Dan: Would it be worth to add a warning (or even error) when running configure, if this version of gcc is used? It seems to be the standard compiler in some distros right now. That might avoid some of the problems on this list. All the best, Simon On 07/21/2014 02:36 PM, Zibo Meng wrote: > Hi Dan and Jan, > > Thanks for you help! > > I ran the bash script one more time and I got the error at the 19th > pass. Since I don't know how to debug c++ program called from the shell > script. So I take Jan's advice to run the ulimit -c unlimited before I > ran the code. I got the core file when the core dump error occurred > whose size is 303MB that can not be attached to this email. Please tell > me what else I should do. > > BTW, I used the following script : > steps/train_mono.sh --nj 10 data/train.1k data/lang exp/mono > where I changed the number of job from 4 to 10 and now I am at the 39th > pass without suffering a core dump as before. > > One more question, if I want to use run_nnet2.sh to do the training and > testing, should I run all the scripts in the run.sh file first? > > Thank you very much. > > Best, > > Zibo > > > On Wed, Jul 16, 2014 at 3:18 PM, Daniel Povey <dp...@gm... > <mailto:dp...@gm...>> wrote: > > I think it's possible that this is caused by a bug in Kaldi itself. > The way I would debug this is to first figure out which of the log > files corresponds to the error (probably one of > exp/mono/log/align.12.*.log), and run the command line that you'll > see at the top of the log file manually to verify that you can > reproduce the error by running it again. > (BTW, I'm a little confused here as normally the stderr of the job > should go to the log file, and this error is produced on the console. ). > > > If you can, then instead of running > <program> <args> > from the console, you'll run > gdb --args <program> <args> > and at the (gdb) prompt you'll type > r > Then hopefully it will run until you get an error. At that point > you can type > bt > to get a backtrace, which you'll show us. > > Dan > > > > On Wed, Jul 16, 2014 at 7:43 AM, Zibo Meng <mzb...@gm... > <mailto:mzb...@gm...>> wrote: > > Hi Jan, > Thank you so much for your reply. > > Here is the information about my distribution, gcc and glibc: > > Fedora release 19 (Schrödinger’s Cat) > NAME=Fedora > VERSION="19 (Schrödinger’s Cat)" > ID=fedora > VERSION_ID=19 > PRETTY_NAME="Fedora 19 (Schrödinger’s Cat)" > ANSI_COLOR="0;34" > CPE_NAME="cpe:/o:fedoraproject:fedora:19" > Fedora release 19 (Schrödinger’s Cat) > Fedora release 19 (Schrödinger’s Cat) > > gcc (GCC) 4.8.2 20131212 (Red Hat 4.8.2-7) > > ldd (GNU libc) 2.17 > > Thank you! > > Zibo > > > > On Wed, Jul 16, 2014 at 10:35 AM, Jan Trmal <af...@ce... > <mailto:af...@ce...>> wrote: > > This looks like a problem with your machine or the toolchain > that was used to compiled kaldi (especially the compiler > and/or the glibc). > If you have experience with debugging, you can run the > command again, generate core dump (using ulimit –c > unlimited) and load it into gdb to figure out the details. > What distribution and gcc and glibc are you using? > > y. > > > > On Wed, Jul 16, 2014 at 10:18 AM, Zibo Meng > <mzb...@gm... <mailto:mzb...@gm...>> wrote: > > Hi, > > After I created the lang directory, I used > steps/train_mono.sh --nj 4 data/train.1k data/lang > exp/mono. But I got the error message as follows: > > steps/train_mono.sh --nj 4 data/train.1k data/lang exp/mono > steps/train_mono.sh --nj 4 data/train.1k data/lang exp/mono > vads = data/train.1k/split4/1/vad.scp > vads = data/train.1k/split4/1/vad.scp > data/train.1k/split4/2/vad.scp > vads = data/train.1k/split4/1/vad.scp > data/train.1k/split4/2/vad.scp > data/train.1k/split4/3/vad.scp > vads = data/train.1k/split4/1/vad.scp > data/train.1k/split4/2/vad.scp > data/train.1k/split4/3/vad.scp > data/train.1k/split4/4/vad.scp > steps/train_mono.sh: Initializing monophone system. > steps/train_mono.sh: Compiling training graphs > steps/train_mono.sh: Aligning data equally (pass 0) > steps/train_mono.sh: Pass 1 > steps/train_mono.sh: Aligning data > steps/train_mono.sh: Pass 2 > steps/train_mono.sh: Aligning data > steps/train_mono.sh: Pass 3 > steps/train_mono.sh: Aligning data > steps/train_mono.sh: Pass 4 > steps/train_mono.sh: Aligning data > steps/train_mono.sh: Pass 5 > steps/train_mono.sh: Aligning data > steps/train_mono.sh: Pass 6 > steps/train_mono.sh: Aligning data > steps/train_mono.sh: Pass 7 > steps/train_mono.sh: Aligning data > steps/train_mono.sh: Pass 8 > steps/train_mono.sh: Aligning data > steps/train_mono.sh: Pass 9 > steps/train_mono.sh: Aligning data > steps/train_mono.sh: Pass 10 > steps/train_mono.sh: Aligning data > steps/train_mono.sh: Pass 11 > steps/train_mono.sh: Pass 12 > steps/train_mono.sh: Aligning data > *** Error in `gmm-acc-stats-ali': free(): corrupted > unsorted chunks: 0x0000000001e10e60 *** > ======= Backtrace: ========= > /lib64/libc.so.6[0x367887d0b8] > gmm-acc-stats-ali(_ZN5kaldi6VectorIfE7DestroyEv+0x27)[0x59f127] > gmm-acc-stats-ali(_ZN5kaldi6VectorIfED1Ev+0x19)[0x4da151] > gmm-acc-stats-ali(_ZNK5kaldi7DiagGmm14LogLikelihoodsERKNS_10VectorBaseIfEEPNS_6VectorIfEE+0x1e6)[0x4fc156] > gmm-acc-stats-ali(_ZNK5kaldi7DiagGmm19ComponentPosteriorsERKNS_10VectorBaseIfEEPNS_6VectorIfEE+0x10a)[0x4fc946] > gmm-acc-stats-ali(_ZN5kaldi12AccumDiagGmm18AccumulateFromDiagERKNS_7DiagGmmERKNS_10VectorBaseIfEEf+0x118)[0x507410] > gmm-acc-stats-ali(_ZN5kaldi14AccumAmDiagGmm16AccumulateForGmmERKNS_9AmDiagGmmERKNS_10VectorBaseIfEEif+0x9e)[0x4f2ec0] > gmm-acc-stats-ali(main+0x56c)[0x4d7aec] > /lib64/libc.so.6(__libc_start_main+0xf5)[0x3678821b45] > gmm-acc-stats-ali[0x4d74b9] > > Can you please tell me what went wrong here? > > Thank you so much! > > Zibo > > > > On Fri, Jul 11, 2014 at 11:24 AM, Zibo Meng > <mzb...@gm... <mailto:mzb...@gm...>> wrote: > > Hi, > > I got another problem. > > When I tried make_mfcc.sh to create the feats.scp > files it did not work. > > I checked the log file where it said some thing like: > > compute-mfcc-feats --verbose=2 > --config=conf/mfcc.conf > scp,p:exp/make_mfcc/train/wav_data.1.scp ark:- > ERROR (compute-mfcc-feats:Read():wave-reader.cc:144) > WaveData: can read only PCM data, audio_format is > not 1: 65534 > WARNING > (compute-mfcc-feats:Read():feat/wave-reader.h:148) > Exception caught in WaveHolder object (reading). > WARNING > (compute-mfcc-feats:LoadCurrent():util/kaldi-table-inl.h:232) > TableReader: failed to load object from 'test.wav' > > Then I checked the attributes of my test.wav file > which were as follows: > Input File : 'test.wav' > Channels : 1 > Sample Rate : 48000 > Precision : 24-bit > Duration : 00:03:30.09 = 10084224 samples ~ > 15756.6 CDDA sectors > File Size : 30.3M > Bit Rate : 1.15M > Sample Encoding: 24-bit Signed Integer PCM > > Can you tell me what should I modify to my audio > files. Thank you so much! > > Best, > > Zibo > > > > On Thu, Jul 10, 2014 at 3:37 PM, Zibo Meng > <mzb...@gm... <mailto:mzb...@gm...>> wrote: > > Hi, > > I am preparing the data for dnn training using > my own data set. I followed the instruction on > http://kaldi.sourceforge.net/data_prep.html. > > I created the file "text" as the first 3 lines: > S002-U-000300-000470 OH > S002-U-000470-000630 I'D > S002-U-000630-000870 LIKE > > the wav.scp file: > S002-U <path to the corresponding wav file> > S002-O <path to the corresponding wav file> > S003-U <path to the corresponding wav file> > > and the utt2spk file: > S002-U-000300-000470 002-U > S002-U-000470-000630 002-U > S002-U-000630-000870 002-U > > Then I used utt2spk_to_spk2utt.pl > <http://utt2spk_to_spk2utt.pl> to create the > spk2utt file. Everything went well until I tried > to use the mak_mfcc.sh to create the feats.scp > file where I got the error message like: > > utils/validate_data_dir.sh: file data/utt2spk is > not in sorted order or has duplicates > > seems like my utt2spk file could not pass > through the validation. > > Can any body help me out of here? Thank you so much. > > Best, > > Zibo > > > > > ------------------------------------------------------------------------------ > Want fast and easy access to all the code in your > enterprise? Index and > search up to 200,000 lines of code with a free copy of > Black Duck > Code Sight - the same software that powers the world's > largest code > search on Ohloh, the Black Duck Open Hub! Try it now. > http://p.sf.net/sfu/bds > > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > <mailto:Kal...@li...> > https://lists.sourceforge.net/lists/listinfo/kaldi-users > > > > > ------------------------------------------------------------------------------ > Want fast and easy access to all the code in your enterprise? > Index and > search up to 200,000 lines of code with a free copy of Black Duck > Code Sight - the same software that powers the world's largest code > search on Ohloh, the Black Duck Open Hub! Try it now. > http://p.sf.net/sfu/bds > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > <mailto:Kal...@li...> > https://lists.sourceforge.net/lists/listinfo/kaldi-users > > > > > > ------------------------------------------------------------------------------ > Want fast and easy access to all the code in your enterprise? Index and > search up to 200,000 lines of code with a free copy of Black Duck > Code Sight - the same software that powers the world's largest code > search on Ohloh, the Black Duck Open Hub! Try it now. > http://p.sf.net/sfu/bds > > > > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users > |
