Accessing the lattice via gstreamer plugin

[chris]

hi,

I'm using the gstreamer Demo app as a starting point for a program idea for
which I need the n-best list. The lattice seems to be the way to get the
n-best list and confidence scores (correct if I'm wrong), so I extended the
demo code to read the lattice from the 'asr' element.

Here's the problem: I can print out the reference to the lattice object (I get
something like "<pocketsphinx.lattice object="" at="" 0x1de3dd0="">"), so I think it's
really there, but when I try to do something meaningful with it (like calling
.bestpath() or .write() on it), the window with the "speak" button just
disappears, and there is no error I could catch. </pocketsphinx.lattice>

Has anyone got an idea what the cause for this might be? I'm not too
experienced with python or GUI programming, and that's particularly true for
the combination of the two, so maybe I'm missing something obvious and not PS-
related here...

Regards,
Chris

[Nickolay V. Shmyrev]

The lattice seems to be the way to get the n-best list and confidence scores

To access confidence you can use ps_seg_iter/ps_seg_prob functions. To access
nbest you can use ps_nbest/ps_nbest_next. All of them do not require any
lattice

but when I try to do something meaningful with it (like calling .bestpath()
or .write() on it), the window with the "speak" button just disappears, and
there is no error I could catch.

It crashes. You need to attach to the process with a debugger and collect a
stacktrace.

[chris]

hi,
thanks for the reply

To access confidence you can use ps_seg_iter/ps_seg_prob functions. To
access nbest you can use ps_nbest/ps_nbest_next.

how are they available when using PS via gstreamer? I thought I'm limited to
the "Element Properties" gst-inspect shows for the pocketsphinx plugin.
I can't just call ps_seg_iter() on the "asr" object I get from calling

asr = self.pipeline.get_by_name('asr')

can I?

I'm afraid you're right about the debugger...

Regards,
Chris

[chris]

let me phrase it a bit more generally:

how does one access the n-best list and confidence scores when using
pocketsphinx via the gstreamer pocketsphinx plugin and using the python
gstreamer bindings?

Regards,
Chris

[Nickolay V. Shmyrev]

No, it's not implemented now but you could implement it.

[chris]

hi,

I gave it a shot but since my C is even worse than my Python, I can't figure
out what the problem with the following code might be:

        int * best_score;
        int * seg_prob;
        if (ps->ps){
            ps_seg_t * seg_iter = ps_seg_iter(ps->ps,&best_score);
            seg_prob = ps_seg_prob(seg_iter, NULL, NULL, NULL);
            float seg_prob_flt = logmath_exp(ps_get_logmath(ps->ps),&seg_prob);
                //...
        }
        // ...

neither "best_score" nor "seg_prob_flt" contain sensible confidence values.
"best_score" oscilates around -10000 while "seg_prob_flt" gives 0.

Is this a C pointer reference problem or am I using the API wrongly?

Regards,
Chris

[Nickolay V. Shmyrev]

Hi

You get zero probs because the gstreamer plugin doesn't enable bestpath.
Change the following lines in gstpocketsphinx.c:

    "-fwdflat", "yes",
    "-bestpath", "yes",

[chris]

hi,
I was already setting those two options programmatically on the gstreamer
pocketsphinx element.

It seems that I was just handling the float output of logmath_exp() wrongly. I
now multipy it by 100 before casting to int and I get probabilities that seem
to make some sense. I'm using ps_get_prob(), though. Using ps_seg_prob()
instead always gave me a result of 100. Not sure why, I might be using it
wrongly, but I'm going to concentrate on the n-best list now, anyway.

Thanks for your help!

Chris

[Nickolay V. Shmyrev]

Hello Chris

Actually you should multiply internal scores by 1024 or shift them by 10 bits
but it is an internal score representation. Which version are you using? You
shoulnd't have access to internal scores through public API.

[chris]

Right now I'm working on the HEAD, but ps_get_prob() has been there since 0.6
I think. Is the output of ps_get_prob() an internal score? It says "posterior
probability" here:
http://cmusphinx.sourceforge.net/api/pocketsphinx/pocketsphinx_8h.html#afa663
1ae5c30a30eec08c7bf1ef41ef5

pocketsphinx.h is the public API, isn't it`?

[Thomas Harris]

Can you make a patch available for this addition?