Libraries sphinx3

[Arslan]

Hi,

I'm using sphinx3_decode binary file to execute sphinx3. I would like to execute the exact same thing but I need to do it directly from the libray given in /include/sphinx3.
Is it possible, I guess it is, but how to implement it?

I hope you'll be able to give me a lead on this.

Thank's
Best regards
Cyril

[Arslan]

Hi again,

One more question (it never ends).
where is define utt_decode used in decode.c.

I know the function utt_decode is define in utt.h

but in decode.c, "utt_decode" is used used as a parameter to ctl_process function.

Actually it wasn't a problem for me before. But i'm now trying to compile the project with a c++ compiler, and it cannot build the library telling me that
utt_decode is undefined. And indeed I can't find it neither.

Would you have any ideas qbout that.
Thank's
Best regards.

[Arslan]

I don't know if my message is very clear. (Since I didn't get any reply).
I'm willing to launch sphinx3 using with C++, using the libraies instead of using an excecutable file.
I don't know exactly where to start, could guys explain me how could I do it??

Any help would be sincerely appeciated
Thanks
Best regards

[Nickolay V. Shmyrev]

What's a problem, you can just include C headers of sphinx libraries, link to them and call some functions. For example you can look into programs in src/programs like decode.c and others. They are really simple.

[Arslan]

Thank's for answering Nickolay,

Sorry, my questions might sound a bit stupid, but I'm no use to use such big programs, I always have programmed simple small applications.

Decode.c is actually pretty much what I was looking for, an exemple showing how to implement it.

I still have questions to understand it.
How should I pass language model, the dictionnary argument.
should I modify this part of the program by remplacing the corresponding line???

static arg_t arg[] = {
log_table_command_line_macro()
cepstral_to_feature_command_line_macro()
acoustic_model_command_line_macro()
speaker_adaptation_command_line_macro()
language_model_command_line_macro()
dictionary_command_line_macro()
...

And what do you mean by link to them, isn't the #include xxx.h enough.

Thanks

[Nickolay V. Shmyrev]

You are welcome

> How should I pass language model, the dictionnary argument.
> should I modify this part of the program by remplacing the corresponding line???

Hm, probably decode.c is not a good way to start with sphinx3 api. main_livepretend.c can be better. About arguments, you can put them into a file and use a function

if (cmd_ln_parse_file(S3_DECODE_ARG_DEFS, "config.file")) {
return -1;
}

or pass them directly through argv char array.

cmd_ln_appl_enter(argc, argv, "default.arg", defn);

It depends on your taste and program requirements. The following sphinxbase header describes
command line argument parsing functions:

http://cmusphinx.svn.sourceforge.net/viewvc/cmusphinx/trunk/sphinxbase/include/cmd_ln.h?revision=6650&view=markup

>And what do you mean by link to them, isn't the #include xxx.h enough.

No, you should also give a command to linker to combine your program with library's code. If you are using g++ you should use command line options like -L<path to sphinx libraries> -lsphinx3 and so on. On Windows you can add libraries in project properties. A short introduction about libraries in gcc:

http://www.network-theory.co.uk/docs/gccintro/gccintro_17.html

[Arslan]

Hi,

Why shouldn't I use decode.c, I would like to use the same recognizer that I was using with ./ sphinx3_decode.

About cmd_ln_parse_file function, How does it work:
I give the config file I was using with the binary file just like this:
-mdef ../../Sphinx3/sphinx3-0.6/model/hmm/hub4_cd_continuous_8gau_1s_c_d_dd/hub4opensrc.6000.mdef -fdict...
and what is S3_DECODE_ARG_DEFS value???

Wether I use cmd_ln_enter or cmd_ln_parse_file, does it has the same effect???

You didn't answer what the part of the code is for:
static arg_t arg[] = {
log_table_command_line_macro()
cepstral_to_feature_command_line_macro()

I don't understand neither the part of the code between the line 99 and 126 of the decode.c file.

I might be asking to much, I hope you don't mind. Anyway thank's for your answers, and for the link about the linker, this helps me a lot.

Best regards.

[Nickolay V. Shmyrev]

67 static arg_t arg[] = {
68 log_table_command_line_macro()
69 cepstral_to_feature_command_line_macro()
70 acoustic_model_command_line_macro()

Since sphinx engine requires advanced configuration and has different options of various types, sphinx libraries allow you to work with program configuration in rather extensible way. Configuration options are described with an arg_t structure. Each option has a type, a default value and a description. Here goes the description of options for decode.c program. Each macro describes an option, for example:

111 {"-bestscoredir",
112 ARG_STRING,
113 NULL,
114 "(Mode 3) Directory for writing best score/frame (used to set beamwidth; "
115 "one file/utterance)"},

defines an option with type string default value NULL and apropriate doc string.

In the beginning you call:

cmd_ln_appl_enter(argc, argv, "default.arg", arg);

you pass command line arguments there through argc and argv, pass an optional file where option could be listed and pass a description of options you defined above (arg array). This
function analyses argv and argc, looks at the file and parse it. Then it fills values of parameters taking default parameters if required.

instead of definition of your own list of args you can use a global variable S3_DECODE_ARG_DEFS which is defined in include/s3_decode.h and has description of all standard option in the same arg array:

extern arg_t S3_DECODE_ARG_DEFS[];

They are declared in s3_decode.c as usual:

arg_t S3_DECODE_ARG_DEFS[] = {
waveform_to_cepstral_command_line_macro(),
waveform_command_line_macro()
cepstral_to_feature_command_line_macro()
acoustic_model_command_line_macro()
speaker_adaptation_command_line_macro()

Basically you shouldn't care about that and can use the same default list of arguments. About
functions cmd_ln_parse_file I suggest you to read their documentation in headers, it just opens file name and parses arguments from it.

Then one you've parsed arguments and stored their values you can start decoding:

You initialize knowledge base:

kb_init(&kb);

and depending on option process utterances:

if (cmd_ln_str("-ctl")) {
146 / When -ctlfile is speicified, corpus.c will look at -ctl_lm and
147 -ctl_mllr to get the corresponding LM and MLLR for the utterance /
148 st->tm = ctl_process(cmd_ln_str("-ctl"),
149 cmd_ln_str("-ctl_lm"),
150 cmd_ln_str("-ctl_mllr"),
151 cmd_ln_int32("-ctloffset"),
152 cmd_ln_int32("-ctlcount"), utt_decode, &kb);
153 }

[Arslan]

Hi Nickolay,

First of all, thank's for your support, I wouldn't have a chance to succed without you.
Since your last mail, I've been trying to make it work, and I have made some progress.
I am using the cmd_ln_parse_file function, and it's seems to work pretty well.

I still have some problem though. I've been compiling decoder.c line by line to see where is the problem. It seem that the problem appears when kb is initialise.
kb_init(&kb);
st = kb.stat;

I hop eyou already have had this kind of error, so you'll be able to tell me what's wrong.
Here's the compilation result:

/users/carslani/test_compile/lib/libs3decoder.a(word_fsg.o)(.text+0x1825): In function word_fsg_write': /users/carslani/Sphinx3/sphinx3-0.6/src/libs3decoder/libsearch/word_fsg.c:986: undefined reference toexp'
/users/carslani/test_compile/lib/libs3decoder.a(word_fsg.o)(.text+0x18d8):/users/carslani/Sphinx3/sphinx3-0.6/src/libs3decoder/libsearch/word_fsg.c:999: unde fined reference to exp' /users/carslani/test_compile/lib/libs3decoder.a(cont_mgau.o)(.text+0x1cf8): In functionmgau_init':
/users/carslani/Sphinx3/sphinx3-0.6/src/libs3decoder/libam/cont_mgau.c:864: undefined reference to log' /users/carslani/test_compile/lib/libs3decoder.a(cont_mgau.o)(.text+0x1d51):/users/carslani/Sphinx3/sphinx3-0.6/src/libs3decoder/libam/cont_mgau.c:871: undefi ned reference tolog'
/users/carslani/test_compile/lib/libs3decoder.a(cont_mgau.o)(.text+0x1ed5):/users/carslani/Sphinx3/sphinx3-0.6/src/libs3decoder/libam/cont_mgau.c:856: undefi ned reference to log' /users/carslani/test_compile/lib/libs3decoder.a(cont_mgau.o)(.text+0x261c):/users/carslani/Sphinx3/sphinx3-0.6/src/libs3decoder/libam/cont_mgau.c:639: undefi ned reference tolog'
/users/carslani/test_compile/lib/libs3decoder.a(s2_semi_mgau.o)(.text+0x1e87): In function s3_precomp': /users/carslani/Sphinx3/sphinx3-0.6/src/libs3decoder/libam/s2_semi_mgau.c:1144: undefined reference tolog'
/users/carslani/test_compile/lib/libs3decoder.a(s2_semi_mgau.o)(.text+0x1f9a):/users/carslani/Sphinx3/sphinx3-0.6/src/libs3decoder/libam/s2_semi_mgau.c:1178: undefined reference to sqrt' /users/carslani/test_compile/lib/libs3decoder.a(logs3.o)(.text+0xb1): In functionlogs3_init':
/users/carslani/Sphinx3/sphinx3-0.6/src/libs3decoder/libcommon/logs3.c:167: undefined reference to log' /users/carslani/test_compile/lib/libs3decoder.a(logs3.o)(.text+0xc4):/users/carslani/Sphinx3/sphinx3-0.6/src/libs3decoder/libcommon/logs3.c:168: undefined re ference tolog10'
/users/carslani/test_compile/lib/libs3decoder.a(logs3.o)(.text+0x105):/users/carslani/Sphinx3/sphinx3-0.6/src/libs3decoder/libcommon/logs3.c:174: undefined r eference to log' /users/carslani/test_compile/lib/libs3decoder.a(logs3.o)(.text+0x16d):/users/carslani/Sphinx3/sphinx3-0.6/src/libs3decoder/libcommon/logs3.c:185: undefined r eference tolog'
/users/carslani/test_compile/lib/libs3decoder.a(logs3.o)(.text+0x1e5):/users/carslani/Sphinx3/sphinx3-0.6/src/libs3decoder/libcommon/logs3.c:205: undefined r eference to log' /users/carslani/test_compile/lib/libs3decoder.a(logs3.o)(.text+0x385): In functionlogs3_add':
/users/carslani/Sphinx3/sphinx3-0.6/src/libs3decoder/libcommon/logs3.c:246: undefined reference to pow' /users/carslani/test_compile/lib/libs3decoder.a(logs3.o)(.text+0x392):/users/carslani/Sphinx3/sphinx3-0.6/src/libs3decoder/libcommon/logs3.c:246: undefined r eference tolog'
/users/carslani/test_compile/lib/libs3decoder.a(logs3.o)(.text+0x3f4): In function logs3': /users/carslani/Sphinx3/sphinx3-0.6/src/libs3decoder/libcommon/logs3.c:264: undefined reference tolog'
/users/carslani/test_compile/lib/libs3decoder.a(logs3.o)(.text+0x5bd): In function logs3_to_p': /users/carslani/Sphinx3/sphinx3-0.6/src/libs3decoder/libcommon/logs3.c:307: undefined reference topow'
/users/carslani/test_compile/lib/libs3decoder.a(vector.o)(.text+0x42d): In function vector_maha_precomp': /users/carslani/Sphinx3/sphinx3-0.6/src/libs3decoder/libcommon/vector.c:327: undefined reference tolog'
/users/carslani/test_compile/lib/libs3decoder.a(vector.o)(.text+0x46f):/users/carslani/Sphinx3/sphinx3-0.6/src/libs3decoder/libcommon/vector.c:330: undefined reference to log' /users/carslani/test_compile/lib/libs3decoder.a(vector.o)(.text+0xb38): In functionvector_pdf_entropy':
/users/carslani/Sphinx3/sphinx3-0.6/src/libs3decoder/libcommon/vector.c:508: undefined reference to log' /users/carslani/test_compile/lib/libs3decoder.a(vector.o)(.text+0xb69):/users/carslani/Sphinx3/sphinx3-0.6/src/libs3decoder/libcommon/vector.c:510: undefined reference tolog'
/users/carslani/test_compile/lib/libs3decoder.a(vector.o)(.text+0xbd3): In function vector_pdf_cross_entropy': /users/carslani/Sphinx3/sphinx3-0.6/src/libs3decoder/libcommon/vector.c:527: undefined reference tolog'
/users/carslani/test_compile/lib/libs3decoder.a(vector.o)(.text+0xc04):/users/carslani/Sphinx3/sphinx3-0.6/src/libs3decoder/libcommon/vector.c:525: more unde fined references to log' follow /users/carslani/test_compile/lib/libs3decoder.a(ms_gauden.o)(.text+0xb8f): In functiongauden_init':
/users/carslani/Sphinx3/sphinx3-0.6/src/libs3decoder/libam/ms_gauden.c:355: undefined reference to logf' /users/carslani/test_compile/lib/libs3decoder.a(ms_gauden.o)(.text+0xbcf):/users/carslani/Sphinx3/sphinx3-0.6/src/libs3decoder/libam/ms_gauden.c:362: undefin ed reference tolog'
/users/carslani/test_compile/lib/libs3decoder.a(cmn.o)(.text+0x1eb): In function cmn': /users/carslani/Sphinx3/sphinx3-0.6/src/libs3decoder/libcep_feat/cmn.c:158: undefined reference tosqrt'
collect2: ld returned 1 exit status

It's seem to be a library problem again. I have read the tutorial you give me about that, it helped me a lot and here is the command line I used to compile:

gcc -Wall decode.c -I /users/carslani/test_compile/include -L /users/carslani/test_compile/lib -ls3decoder -ls3util -ls3audio -o decode

I hope you'll have an idea where is this comming from. And if you need more information, just ask.

Best regards.
Cyril

[David Huggins-Daines]

This one's easy :-) You just have to link with -lm in addition to the Sphinx libraries.

[Arslan]

Yes!!! It works.
can't believe I was so close to succed.

I didn't realize, It was all math function.
Anyway, Thank's again to you and to all the sphinx team.

Cheers.
Cyril

[Arslan]

I actually have one more (and let's hope the last) question.

Is it possible to the result of the recognition in a file, or a variable, so it can be used later.

I guess I should modify this part of the code but I don't know how:

{"-bestscoredir",
ARG_STRING,
NULL,
"(Mode 3) Directory for writing best score/frame (used to set beamwidth; one file/utterance)"},

Any idea??

Best regards.
Cyril

[Nickolay V. Shmyrev]

No, bestscore dir is used for completely different things. It just selects per-utterance best score parameter.

How do you plan to use sphinx library, do you need a files as a result of recognition or is it better to have access to in-memory results? How is your audio data created, is it files or also in-memory values?

[Arslan]

The best would be to get the recognized utterance in a string variable accessible in the programm.
But if it is stored in a file, it would be good enough.

Concerning the audio data, i'm using mfc files.

regards

[Nickolay V. Shmyrev]

Then I suppose you should pass your own function to ctl_process_utt instead of utt_decode. Look at main_livepretend for example:

   st->tm = ctl_process(ctrlfn,                                            
                         cmd_ln_str("-ctl_lm"),                             
                         cmd_ln_str("-ctl_mllr"),                           
                         cmd_ln_int32("-ctloffset"),                        
                         cmd_ln_int32("-ctlcount"),                         
                         utt_livepretend, &(decoder.kb))

And earlier

static void
utt_livepretend(void data, utt_res_t * ur, int32 sf, int32 ef,
char uttid)

....

while (len > 0) {                                                           
    ptmr_start(&(st->tm));

    fe_process_utt(fe, samples, len, &frames, &n_frames);                   
    if (frames != NULL) {                                                   
        s3_decode_process(&decoder, frames, n_frames);                      
        ckd_free_2d((void **)frames);                                       
    }                                                                       
    ptmr_stop(&(st->tm));

    if (S3_DECODE_SUCCESS ==                                                
        s3_decode_hypothesis(&decoder, NULL, &hypstr, NULL))                
        if (decoder.phypdump)                                               
            E_INFO("PARTIAL_HYP: %s\n", hypstr);                            
    len = fread(samples, sizeof(short), SAMPLE_BUFFER_LENGTH, rawfd);       
}

Here s3_decode_hypothesis returns you result in hypstr. I suppose you can adopt that utt_livepretend without ctl_process since you have control files and use it directly.

[Arslan]

Thank's man,

wouldn't it be possible to find the recognize string using decode.c?
anyway, i'm gonna try what you adviced me, I let you know how it works.

Best regards

[Nickolay V. Shmyrev]

Hm, it shouldn't be worse unless you've made a mistake somewhere. So please just check everything once again. Probably if you'll paste sources you've wrote we can try to fix it. For example are you sure you correctly passing feature files instead of raw data?

[Arslan]

Hi,

Indeed it works, but the recognizion is a lot worst, I need to stay with decode.c
If you've got any idea how to proceed to get the recognize string let me know.
thank's a lot.
Best regards
cyril

[Arslan]

Hi,

I might have made a mistake, but anyway, I have manage to make it work using decode.
I've modify the libs3decoder library.
For those interested, I've modify the match_write function in the srch_output.c file. And replace fprintf(fp, "%s ", dict_wordstr(dict, dict_basewid(dict, h->id)));
by strcat(mystr, dict_wordstr(dict, dict_basewid(dict, h->id)));

This way I'm getting a string of my recognized utterance that I can reusein the program.
The lib need to be rebuilt afterwards.

I still have a problem with -utt parameters. The recognition works fine, but the excecution doesn't end. it gives me this:

FWDXCT: num_tel3.mfc_00000000 S 15289829 T 9987784 A 9980211 L 75730 673966 -8101 <sil>27 23462 -2438 zero59 152281 -12405 six106 182625 -8101 <sil>113 -215373 -12606 two131 -282111 -12606 zero175 841434 -8101 <sil>194 322347 -12193 nine230 297171 -15236 zero273 1748407 -8101 <sil>309 678698 -12405 seven361 211441 -12606 three400 692483 -8101 <sil>418 -291949 -12606 three450 16135 -8101 <sil>455 513083 -12606 seven504 1011945 -8101 <sil>538 3404166 -8101 <sil>618 0 -14555 </s>618

INFO: stat.c(154): 618 frm; 15 cdsen/fr, 102 cisen/fr, 123 cdgau/fr, 816 cigau/fr, Sen 0.01, CPU 0.01 Clk [Ovrhd 0.01 CPU 0.01 Clk]; 24 hmm/fr, 1 wd/fr, Search: 0.00 CPU 0.00 Clk (num_tel3.mfc_00000000)
INFO: fast_algo_struct.c(398): HMMHist0..0: 618(100)
carslani 17043 0.0 0.0 11796 5064 pts/21 S+ 10:05 0:00 ./decode
INFO: lm.c(944): 619 tg(), 606 tgcache, 12 bo; 10 fills, 17 in mem (8.9%)
INFO: lm.c(948): 56 bg(), 1 bo; 7 fills, 61 in mem (59.8%)
INFO: corpus.c(721): num_tel3.mfc_00000000: 0.1 sec CPU, 0.3 sec Clk; TOT: 0.1 sec CPU, 0.3 sec Clk

INFO: corpus.c(703): Waiting for ./num_tel3.mfc, count 1000000000, c 1

I have no clue where could that come from. When I'm using the -ctl option, specifing wich file I want to recognized in a separate file, everything is working just fine.

I'm really looking forward to see your answer.

Thank's to you Nickolay and to all the sphinx team.

Best regards

[Nickolay V. Shmyrev]

Well, ctl_process_utt is not the thing you really want. It waits for file changes and processes them. See function documentation:

/
* Like ctl_process, but process the single filename given (uttfile), count times. After each
* processing, wait for the time of modification on the given file to change. In this mode,
* the decoder can be used to process a dynamically generated sequence of utterances. To avoid
* race conditions, each new instance of the file should be created "in an instant": by creating
* it under a temporary name and finally renaming it to the given filename atomically.
* @return: ptmr_t structure containing cpu/elapsed time stats for the run.
/
ptmr_t ctl_process_utt (char uttfile, /< In: Filename to be process (in its entirety) */
int32 count, /< In: No. of iterations to process uttfile /
void (func) (void kb, utt_res_t ur, int32 sf, int32 ef, char *uttid),/< A function pointer that

                    void *kb);

Do you really need to pass a single file to decoder?

[Arslan]

Hi,
I still have a problem with the -utt option.
Indeed when I use -ctl, and I specify the file to decode in a .ctl file, it works fine.
but with utt parameter, the recognizer works as well, but the execution blocks, it never quit the program. (the log is the previous message).

I have no clue how to proceed.

here is the configuration I use, maybe it is coming from here:

-mdef ./tidigits/wd_dependent_phone.500.mdef -fdict ./tidigits/fillerdict
-dict ./tidigits/dictionary -mean ./tidigits/means.LittleEndian
-var ./tidigits/variances.LittleEndian -mixw ./tidigits/mixture_weights.LittleEndian
-tmat ./tidigits/transition_matrices.LittleEndian
-utt ./num_tel3.mfc -subvqbeam 1e-02 -epl 4 -fillprob 0.02 -feat 1s_c_d_dd -lw 9.5
-maxwpf 1 -beam 1e-40 -pbeam 1e-30 -wbeam 1e-20 -maxhmmpf 1500
-wend_beam 1e-1 -ci_pbeam 1e-5 -lm ./tidigits/tidigits.lm.DMP

Thank's for reading me.
Best regards.

[Arslan]

Hi,

Yes I need to pass a single file, and I would prefer to pass it directly to the decoder,
without writing it on a ctl file.
I need that to make my sytem all automatic and synchronous.

I don't know if this is possible, but it would be great.

Thank's again.
Best regards.

[Nickolay V. Shmyrev]

Sure it's possible, you just need to use sphinx API in a different way.

If you'll look into the sources of ctl_process src/s3decode/libcommon/corpus.c you'll find it does almost nothing except reading your utterances and calling utt_decode. You can do the same, it shouldn't create a performance degradation:

    ptmr_start(&tm);   // it's just a timer, not required                                                   
    if (func) {                                                             
        if (ctlmllrfile)                                                    
            kb_setmllr(regmatfile, cb2mllrfile, kb);                        
        (*func) (kb, uttfile, sf, ef, uttid);                               
    }                                                                       
    ptmr_stop(&tm);

kb_setmllr is not required too since you don't use mllr. So instead of running

    st->tm = ctl_process(cmd_ln_str("-ctl"),                                
                         cmd_ln_str("-ctl_lm"),                             
                         cmd_ln_str("-ctl_mllr"),                           
                         cmd_ln_int32("-ctloffset"),                        
                         cmd_ln_int32("-ctlcount"), utt_decode, &kb);

You can just call

utt_decode (&kb, uttfile, source_frame, end_frame, uttid);

Later you can modify utt_decode to return result in a string without dirty hacks.

[Arslan]

Hi Nickolay,

Sorry I didn't answer your post, I've been away lately.
I'm going to try what you advised me.
but look:
void utt_decode (void data, /< A kb /
utt_res_t ur, /< Utterance resource structure /
int32 sf, /< Starting frame of the decoding */
int32 ef, /< Ending frame of the decoding /
char uttid /*< Utterance ID /
);

what should I give as utt_res_t (should i give a file name?)
concerning starting frame and eding frame of the decoding what that represent. I want the recognizer to perform a recognition from the begin to the end of the audio file. Even uttid, I don't have any idea what the parameter needed.

I'm really looking forward to read you back.
Thank for you help.
Best regards.

[Arslan]

Hi guys,
Are you all in vacation?
Hoping one of you will answer.
Best regards.