Re: [Kaldi-users] data preparation problem

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

Hi Dan and Jan,

Thanks for you help!

I ran the bash script one more time and I got the error at the 19th pass.
Since I don't know how to debug c++ program called from the shell script.
So I take Jan's advice to run the ulimit -c unlimited before I ran the
code. I got the core file when the core dump error occurred whose size is
303MB that can not be attached to this email. Please tell me what else I
should do.

BTW, I used the following script :
steps/train_mono.sh --nj 10 data/train.1k data/lang exp/mono
where I changed the number of job from 4 to 10 and now I am at the 39th
pass without suffering a core dump as before.

One more question, if I want to use run_nnet2.sh to do the training and
testing, should I run all the scripts in the run.sh file first?

Thank you very much.

Best,

Zibo

On Wed, Jul 16, 2014 at 3:18 PM, Daniel Povey <dp...@gm...> wrote:

> I think it's possible that this is caused by a bug in Kaldi itself.
> The way I would debug this is to first figure out which of the log files
> corresponds to the error (probably one of exp/mono/log/align.12.*.log), and
> run the command line that you'll see at the top of the log file manually to
> verify that you can reproduce the error by running it again.
> (BTW, I'm a little confused here as normally the stderr of the job should
> go to the log file, and this error is produced on the console. ).
>
>
> If you can, then instead of running
> <program> <args>
> from the console, you'll run
> gdb --args <program> <args>
> and at the (gdb) prompt you'll type
> r
> Then hopefully it will run until you get an error.  At that point you can
> type
> bt
> to get a backtrace, which you'll show us.
>
> Dan
>
>
>
> On Wed, Jul 16, 2014 at 7:43 AM, Zibo Meng <mzb...@gm...> wrote:
>
>> Hi Jan,
>> Thank you so much for your reply.
>>
>> Here is the information about my distribution, gcc and glibc:
>>
>> Fedora release 19 (Schrödinger’s Cat)
>> NAME=Fedora
>> VERSION="19 (Schrödinger’s Cat)"
>> ID=fedora
>> VERSION_ID=19
>> PRETTY_NAME="Fedora 19 (Schrödinger’s Cat)"
>> ANSI_COLOR="0;34"
>> CPE_NAME="cpe:/o:fedoraproject:fedora:19"
>> Fedora release 19 (Schrödinger’s Cat)
>> Fedora release 19 (Schrödinger’s Cat)
>>
>> gcc (GCC) 4.8.2 20131212 (Red Hat 4.8.2-7)
>>
>> ldd (GNU libc) 2.17
>>
>> Thank you!
>>
>> Zibo
>>
>>
>>
>> On Wed, Jul 16, 2014 at 10:35 AM, Jan Trmal <af...@ce...> wrote:
>>
>>> This looks like a problem with your machine or the toolchain that was
>>> used to compiled kaldi (especially the compiler and/or the glibc).
>>> If you have experience with debugging, you can run the command again,
>>> generate core dump (using ulimit –c unlimited) and load it into gdb to
>>> figure out the details.
>>> What distribution and gcc and glibc  are you using?
>>>
>>> y.
>>>
>>>
>>>
>>> On Wed, Jul 16, 2014 at 10:18 AM, Zibo Meng <mzb...@gm...> wrote:
>>>
>>>> Hi,
>>>>
>>>> After I created the lang directory, I used steps/train_mono.sh --nj 4
>>>> data/train.1k data/lang exp/mono. But I got the error message as follows:
>>>>
>>>> steps/train_mono.sh --nj 4 data/train.1k data/lang exp/mono
>>>> steps/train_mono.sh --nj 4 data/train.1k data/lang exp/mono
>>>> vads =  data/train.1k/split4/1/vad.scp
>>>> vads =  data/train.1k/split4/1/vad.scp data/train.1k/split4/2/vad.scp
>>>> vads =  data/train.1k/split4/1/vad.scp data/train.1k/split4/2/vad.scp
>>>> data/train.1k/split4/3/vad.scp
>>>> vads =  data/train.1k/split4/1/vad.scp data/train.1k/split4/2/vad.scp
>>>> data/train.1k/split4/3/vad.scp data/train.1k/split4/4/vad.scp
>>>> steps/train_mono.sh: Initializing monophone system.
>>>> steps/train_mono.sh: Compiling training graphs
>>>> steps/train_mono.sh: Aligning data equally (pass 0)
>>>> steps/train_mono.sh: Pass 1
>>>> steps/train_mono.sh: Aligning data
>>>> steps/train_mono.sh: Pass 2
>>>> steps/train_mono.sh: Aligning data
>>>> steps/train_mono.sh: Pass 3
>>>> steps/train_mono.sh: Aligning data
>>>> steps/train_mono.sh: Pass 4
>>>> steps/train_mono.sh: Aligning data
>>>> steps/train_mono.sh: Pass 5
>>>> steps/train_mono.sh: Aligning data
>>>> steps/train_mono.sh: Pass 6
>>>> steps/train_mono.sh: Aligning data
>>>> steps/train_mono.sh: Pass 7
>>>> steps/train_mono.sh: Aligning data
>>>> steps/train_mono.sh: Pass 8
>>>> steps/train_mono.sh: Aligning data
>>>> steps/train_mono.sh: Pass 9
>>>> steps/train_mono.sh: Aligning data
>>>> steps/train_mono.sh: Pass 10
>>>> steps/train_mono.sh: Aligning data
>>>> steps/train_mono.sh: Pass 11
>>>> steps/train_mono.sh: Pass 12
>>>> steps/train_mono.sh: Aligning data
>>>> *** Error in `gmm-acc-stats-ali': free(): corrupted unsorted chunks:
>>>> 0x0000000001e10e60 ***
>>>> ======= Backtrace: =========
>>>> /lib64/libc.so.6[0x367887d0b8]
>>>> gmm-acc-stats-ali(_ZN5kaldi6VectorIfE7DestroyEv+0x27)[0x59f127]
>>>> gmm-acc-stats-ali(_ZN5kaldi6VectorIfED1Ev+0x19)[0x4da151]
>>>>
>>>> gmm-acc-stats-ali(_ZNK5kaldi7DiagGmm14LogLikelihoodsERKNS_10VectorBaseIfEEPNS_6VectorIfEE+0x1e6)[0x4fc156]
>>>>
>>>> gmm-acc-stats-ali(_ZNK5kaldi7DiagGmm19ComponentPosteriorsERKNS_10VectorBaseIfEEPNS_6VectorIfEE+0x10a)[0x4fc946]
>>>>
>>>> gmm-acc-stats-ali(_ZN5kaldi12AccumDiagGmm18AccumulateFromDiagERKNS_7DiagGmmERKNS_10VectorBaseIfEEf+0x118)[0x507410]
>>>>
>>>> gmm-acc-stats-ali(_ZN5kaldi14AccumAmDiagGmm16AccumulateForGmmERKNS_9AmDiagGmmERKNS_10VectorBaseIfEEif+0x9e)[0x4f2ec0]
>>>> gmm-acc-stats-ali(main+0x56c)[0x4d7aec]
>>>> /lib64/libc.so.6(__libc_start_main+0xf5)[0x3678821b45]
>>>> gmm-acc-stats-ali[0x4d74b9]
>>>>
>>>> Can you please tell me what went wrong here?
>>>>
>>>> Thank you so much!
>>>>
>>>> Zibo
>>>>
>>>>
>>>>
>>>> On Fri, Jul 11, 2014 at 11:24 AM, Zibo Meng <mzb...@gm...> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I got another problem.
>>>>>
>>>>> When I tried make_mfcc.sh to create the feats.scp files it did not
>>>>> work.
>>>>>
>>>>> I checked the log file where it said some thing like:
>>>>>
>>>>> compute-mfcc-feats --verbose=2 --config=conf/mfcc.conf
>>>>> scp,p:exp/make_mfcc/train/wav_data.1.scp ark:-
>>>>> ERROR (compute-mfcc-feats:Read():wave-reader.cc:144) WaveData: can
>>>>> read only PCM data, audio_format is not 1: 65534
>>>>> WARNING (compute-mfcc-feats:Read():feat/wave-reader.h:148) Exception
>>>>> caught in WaveHolder object (reading).
>>>>> WARNING (compute-mfcc-feats:LoadCurrent():util/kaldi-table-inl.h:232)
>>>>> TableReader: failed to load object from 'test.wav'
>>>>>
>>>>> Then I checked the attributes of my test.wav file which were as
>>>>> follows:
>>>>> Input File     : 'test.wav'
>>>>> Channels       : 1
>>>>> Sample Rate    : 48000
>>>>> Precision      : 24-bit
>>>>> Duration       : 00:03:30.09 = 10084224 samples ~ 15756.6 CDDA sectors
>>>>> File Size      : 30.3M
>>>>> Bit Rate       : 1.15M
>>>>> Sample Encoding: 24-bit Signed Integer PCM
>>>>>
>>>>> Can you tell me what should I modify to my audio files. Thank you so
>>>>> much!
>>>>>
>>>>> Best,
>>>>>
>>>>> Zibo
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Jul 10, 2014 at 3:37 PM, Zibo Meng <mzb...@gm...> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I am preparing the data for dnn training using my own data set. I
>>>>>> followed the instruction on
>>>>>> http://kaldi.sourceforge.net/data_prep.html.
>>>>>>
>>>>>> I created the file "text" as the first 3 lines:
>>>>>> S002-U-000300-000470 OH
>>>>>> S002-U-000470-000630 I'D
>>>>>> S002-U-000630-000870 LIKE
>>>>>>
>>>>>> the wav.scp file:
>>>>>> S002-U <path to the corresponding wav file>
>>>>>> S002-O <path to the corresponding wav file>
>>>>>> S003-U <path to the corresponding wav file>
>>>>>>
>>>>>> and the utt2spk file:
>>>>>> S002-U-000300-000470 002-U
>>>>>> S002-U-000470-000630 002-U
>>>>>> S002-U-000630-000870 002-U
>>>>>>
>>>>>> Then I used utt2spk_to_spk2utt.pl to create the spk2utt file.
>>>>>> Everything went well until I tried to use the mak_mfcc.sh to create the
>>>>>> feats.scp file where I got the error message like:
>>>>>>
>>>>>> utils/validate_data_dir.sh: file data/utt2spk is not in sorted order
>>>>>> or has duplicates
>>>>>>
>>>>>> seems like my utt2spk file could not pass through the validation.
>>>>>>
>>>>>> Can any body help me out of here? Thank you so much.
>>>>>>
>>>>>> Best,
>>>>>>
>>>>>> Zibo
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Want fast and easy access to all the code in your enterprise? Index and
>>>> search up to 200,000 lines of code with a free copy of Black Duck
>>>> Code Sight - the same software that powers the world's largest code
>>>> search on Ohloh, the Black Duck Open Hub! Try it now.
>>>> http://p.sf.net/sfu/bds
>>>>
>>>> _______________________________________________
>>>> Kaldi-users mailing list
>>>> Kal...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>>>>
>>>>
>>>
>>
>>
>> ------------------------------------------------------------------------------
>> Want fast and easy access to all the code in your enterprise? Index and
>> search up to 200,000 lines of code with a free copy of Black Duck
>> Code Sight - the same software that powers the world's largest code
>> search on Ohloh, the Black Duck Open Hub! Try it now.
>> http://p.sf.net/sfu/bds
>> _______________________________________________
>> Kaldi-users mailing list
>> Kal...@li...
>> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>>
>>
>