About saving PCM data when using pocketsphinx_batch

[stevenyslin]

Hello,

I want to save PCM data to convert wav data, so I set fe_interface.c 's fe_end_utt() to let "store_pcm" open,

int32 fe_end_utt(fe_t * fe, mfcc_t * cepvector, int32 * nframes)
{
    /* Process any remaining data, not very accurate for the VAD */
    *nframes = 0;
    if (fe->num_overflow_samps > 0) {
        fe_read_frame(fe, fe->overflow_samps, fe->num_overflow_samps);
        //fe_write_frame(fe, cepvector, FALSE);

        /* New add */
        fe_write_frame(fe, cepvector, TRUE);

        if (fe->vad_data->in_speech)
            *nframes = 1;
    }

    /* reset overflow buffers... */
    fe->num_overflow_samps = 0;

    return 0;
}

and then, I run my command,

pocketsphinx_batch -adcin yes -cepdir wav -cepext .wav -ctl etc/your_db_test.fileids -lm etc/your_db.lm.DMP -dict etc/your_db.dict -hmm tdt_sc_8k -hyp test.hyp

but the system show as follows in fe_prespch_buf.c：

Floating point exception (core dumped)

So I set the fe_prespch_buf.c as follows：

fe_prespch_init(int num_frames, int num_cepstra, int num_samples)
{
    prespch_buf_t *prespch_buf;

    prespch_buf = (prespch_buf_t *) ckd_calloc(1, sizeof(prespch_buf_t));

    prespch_buf->num_cepstra = num_cepstra;
    prespch_buf->num_frames_cep = num_frames;
    prespch_buf->num_samples = num_samples;
    //prespch_buf->num_frames_pcm = 0;

    /* New add */
    prespch_buf->num_frames_pcm = num_samples;

......

    return prespch_buf;
}

My question：
I add some code in fe_prespch_write_pcm() to save "prespch_buf->pcm_buf" ,
but when I convert to wav file, it seems like doesn't work,
Does my code have any problem ?

Thanks for your help.

void fe_prespch_write_pcm(prespch_buf_t * prespch_buf, int16 * samples)
{
    int32 sample_ptr;

    FILE *fid1;
    char SaveTxtFile[256]="";
    int i;

    sample_ptr = prespch_buf->pcm_write_ptr * prespch_buf->num_samples;
    memcpy(&prespch_buf->pcm_buf[sample_ptr], samples,
           prespch_buf->num_samples * sizeof(int16));

    prespch_buf->pcm_write_ptr = (prespch_buf->pcm_write_ptr + 1) % prespch_buf->num_frames_pcm;
    if (prespch_buf->npcm < prespch_buf->num_frames_pcm) {
        prespch_buf->npcm++;    
    } else {
        prespch_buf->pcm_read_ptr = (prespch_buf->pcm_read_ptr + 1) % prespch_buf->num_frames_pcm;
    }

    /* New add */
    sprintf(SaveTxtFile, "prespch_buf");
    fid1 = fopen(SaveTxtFile, "wt");
    for (i=0; i < *sample; i++)
    {
        fprintf(fid1, %d\n, prespch_buf->pcm_buf[i]);
    }
    fcolse(fid1);
}

[Nickolay V. Shmyrev]

It is not clear what are you asking about, try to follow this plan when you ask questions:

1) What was the initial state, what versions did you take, etc.
2) What have you done, what code changes have you done, what command do you run. Share the code you modified as a whole project, do not provide single lines.
3) What do you see
4) What do you expect to see

Also please try to use more straightforward English please. When you do not put effort in your question, there is no reasons for us to put effort in an answer.

[stevenyslin]

Dear Nickolay,

I really apologize for my behavior and poor English,
I reorganize my questions as follows：

4) What do you expect to see

Because in this discussion, it have mentioned that pocketsphinx_batch can leaves some silence or noise before decoding,
So I want to get PCM data after deal with by pocketsphinx_batch, and convert PCM data to wave data.

---

1) What was the initial state, what versions did you take, etc.

I downloaded the project from the github link
sphinxbase：
https://github.com/cmusphinx/sphinxbase

pocketsphinx：
https://github.com/cmusphinx/pocketsphinx

---

2) What have you done, what code changes have you done, what command do you run. Share the code you modified as a whole project, do not provide single lines.

The First Step：
I trace code of sphinxbase, and I found sphinxbase/src/libsphinxbase/fe/fe_noise.c can store PCM data in line 464,
but the variable "store_pcm" is set FALSE, so I modify the code (sphinxbase/src/libsphinxbase/fe/feinterface.c) to let it work, the code is in line 607~623.

fe_noise.c

void
fe_vad_hangover(fe_t * fe, mfcc_t * feat, int32 is_speech, int32 store_pcm)
{
    if (!fe->vad_data->in_speech) {
        fe_prespch_write_cep(fe->vad_data->prespch_buf, feat);
        if (store_pcm)
            fe_prespch_write_pcm(fe->vad_data->prespch_buf, fe->spch);
    }

    /* track vad state and deal with cepstrum prespeech buffer */
    if (is_speech) {
        fe->vad_data->post_speech_frames = 0;
        if (!fe->vad_data->in_speech) {
            fe->vad_data->pre_speech_frames++;
            /* check for transition sil->speech */
            if (fe->vad_data->pre_speech_frames >= fe->start_speech) {
                fe->vad_data->pre_speech_frames = 0;
                fe->vad_data->in_speech = 1;
            }
        }
    } else {
        fe->vad_data->pre_speech_frames = 0;
        if (fe->vad_data->in_speech) {
            fe->vad_data->post_speech_frames++;
            /* check for transition speech->sil */
            if (fe->vad_data->post_speech_frames >= fe->post_speech) {
                fe->vad_data->post_speech_frames = 0;
                fe->vad_data->in_speech = 0;
                fe_prespch_reset_cep(fe->vad_data->prespch_buf);
            fe_prespch_reset_pcm(fe->vad_data->prespch_buf);
            }
        }
    }
}

fe_interface.c

int32
fe_end_utt(fe_t * fe, mfcc_t * cepvector, int32 * nframes)
{
    /* Process any remaining data, not very accurate for the VAD */
    *nframes = 0;
    if (fe->num_overflow_samps > 0) {
        fe_read_frame(fe, fe->overflow_samps, fe->num_overflow_samps);

    /* Original code */
        //fe_write_frame(fe, cepvector, FALSE);

    /* New add by Steven */
        fe_write_frame(fe, cepvector, TRUE);

        if (fe->vad_data->in_speech)
            *nframes = 1;
    }

    /* reset overflow buffers... */
    fe->num_overflow_samps = 0;

    return 0;
}

The Second Step：
Then I rebuild the sphinxbase library, and run pocketsphinx_batch,

$ pocketsphinx_batch -adcin yes -cepdir wav -cepext .wav -ctl etc/your_db_test.fileids -lm etc/your_db.lm.DMP -dict etc/your_db.dict -hmm tdt_sc_8k -hyp test.hyp

but the system show error as follows：
http://imgur.com/a/hosMZ

Then, I trace the code to solve error, I found the sphinxbase/src/libsphinxbase/fe/feprespchbuf.c => fe_prespch_init() => prespch_buf->num_frames_pcm,
the variable was set zero, then I change it as follows, ( prespch_buf->num_frames_pcm = num_samples;)
The code is modified in (sphinxbase/src/libsphinxbase/fe/feprespchbuf.c), and the code is in line 82~111.

fe_prespch_buf.c

prespch_buf_t *
fe_prespch_init(int num_frames, int num_cepstra, int num_samples)
{
    prespch_buf_t *prespch_buf;

    prespch_buf = (prespch_buf_t *) ckd_calloc(1, sizeof(prespch_buf_t));

    prespch_buf->num_cepstra = num_cepstra;
    prespch_buf->num_frames_cep = num_frames;
    prespch_buf->num_samples = num_samples;

    /* Original code */
    //prespch_buf->num_frames_pcm = 0;

    /* New add by Steven */
    prespch_buf->num_frames_pcm = num_samples;    

    prespch_buf->cep_write_ptr = 0;
    prespch_buf->cep_read_ptr = 0;
    prespch_buf->ncep = 0;

    prespch_buf->pcm_write_ptr = 0;
    prespch_buf->pcm_read_ptr = 0;
    prespch_buf->npcm = 0;

    prespch_buf->cep_buf = (mfcc_t **)
        ckd_calloc_2d(num_frames, num_cepstra,
                      sizeof(**prespch_buf->cep_buf));

    prespch_buf->pcm_buf = (int16 *)
        ckd_calloc(prespch_buf->num_frames_pcm * prespch_buf->num_samples,
                   sizeof(int16));

    return prespch_buf;
}

The Third Step：

The sphinxbase/src/libsphinxbase/fe/fenoise.c => fe_vad_hangover() will call function "fe_prespch_write_pcm()", if "store_pcm" was set TRUE.
Then, I add some code in function "fe_prespch_write_pcm()" to save "prespch_buf->pcm_buf" variable,
The code is modified in (sphinxbase/src/libsphinxbase/fe/feprespchbuf.c), and the code is in line 157~172.

fe_prespch_buf.c

void
fe_prespch_write_pcm(prespch_buf_t * prespch_buf, int16 * samples)
{
    int32 sample_ptr;

    /* New add start by Steven */
    FILE *fid1;
    char SaveTxtFile[256]="";
    int i;
    /* New add end by Steven */

    sample_ptr = prespch_buf->pcm_write_ptr * prespch_buf->num_samples;
    memcpy(&prespch_buf->pcm_buf[sample_ptr], samples,
           prespch_buf->num_samples * sizeof(int16));

    prespch_buf->pcm_write_ptr = (prespch_buf->pcm_write_ptr + 1) % prespch_buf->num_frames_pcm;
    if (prespch_buf->npcm < prespch_buf->num_frames_pcm) {
        prespch_buf->npcm++;    
    } else {
        prespch_buf->pcm_read_ptr = (prespch_buf->pcm_read_ptr + 1) % prespch_buf->num_frames_pcm;
    }

    /* New add start by Steven */
    sprintf(SaveTxtFile, "prespch_buf");
    fid1 = fopen(SaveTxtFile, "wt");
    for (i=0; i < *sample; i++)
    {
        fprintf(fid1, %d\n, prespch_buf->pcm_buf[i]);
    }
    fcolse(fid1);
    /* New add end by Steven */
}

---

3) What do you see

I rename prespch_buf to prespch_buf.raw, and use sox library to transform wave data,

$ sox -r 16000 -e signed-integer -b 16 -c 1 prespch_buf.raw prespch_buf.wav

but it seems like doesn't work.

My question：
1.Is the variable "prespch_buf->pcm_buf" in function "fe_prespch_write_pcm()" represents the data which has been handled with noise and silence?
2.In accordance with step one to step three, is there something wrong with my code?

Really thanks a lot.

My code is as follows link：
fe_interface.c
fe_prespch_buf.c

[Nickolay V. Shmyrev]

1.Is the variable "prespch_buf->pcm_buf" in function "fe_prespch_write_pcm()" represents the data which has been handled with noise and silence?

Well, sort of, you'd better access processed data in higher levels, for example, in ps_process_raw function.

2.In accordance with step one to step three, is there something wrong with my code?

Your code is not quite correct

   fid1 = fopen(SaveTxtFile,"wt");
    for( i=0; i < *samples ; i++)
    {
            fprintf(fid1,"%d\n", prespch_buf->pcm_buf[i]);
    }
    fclose(fid1);

samples here is a buffer of samples, you are using it as size, it should be something like

   fid1 = fopen(SaveTxtFile,"wt");
    for( i=0; i < prespch_buf->num_samples ; i++)
    {
            fprintf(fid1,"%d\n", samples[i]);
    }
    fclose(fid1);

instead