Switching language models on the fly with JSG

[Halle]

Hi Nickolay,

As an upcoming feature of OpenEars I'm adding in the ability to change
grammars on the fly in an already-started continuous recognition loop. I
already have a tested and working method for doing this with Arpa language
models, which is that I call this method at the top of the outer loop in the
continuous function using the already-existing ps_decoder_t, cmd_ln_t and the
logmath_t I've extracted from the decoder:

- (void) changePS:(ps_decoder_t *)pocketsphinxDecoder usingConfig:(cmd_ln_t *)pocketsphinxConfiguration andLogmath:(logmath_t*)logmath {

    ngram_model_t *lmset, *lm;

    lm = ngram_model_read(pocketsphinxConfiguration, (char *)[self.languageModelFileToChangeTo UTF8String], NGRAM_AUTO, logmath);

    lmset = ps_get_lmset(pocketsphinxDecoder);

    NSString *languageModelID = [NSString stringWithFormat:@"newlm_%d",newlmnumber];
    ngram_model_set_add(lmset, lm, (char *)languageModelID, 1.0, TRUE);
    ngram_model_set_select(lmset, (char *)languageModelID);

    ps_update_lmset(pocketsphinxDecoder, lmset);    
}

This is working fine at the moment; the grammar is changed the next time that
Pocketsphinx gets to the top of that loop in continuous. My question for you
is that I would like to also offer this ability for JSGF grammars but I don't
see the parallel functions that would let me insert new JSGF config
information into an already-running config. Is this possible? If you have a
moment could you let me know how I should be trying to do this with a JSGF?

Thank you,

Halle

[Nickolay V. Shmyrev]

But I don't see the parallel functions that would let me insert new JSGF
config information into an already-running config. Is this possible?

For finite state grammar you can use

POCKETSPHINX_EXPORT
fsg_set_t *ps_update_fsgset(ps_decoder_t *ps);

And related functions. Just the same as lmset. To convert JSGF grammar into
fsg to insert into fsgset use

SPHINXBASE_EXPORT
fsg_model_t *jsgf_build_fsg(jsgf_t *grammar, jsgf_rule_t *rule,
                            logmath_t *lmath, float32 lw);

It would be nice to get more consistent API of course, patch for that thing is
welcome.

[Halle]

Thank you, that really helped me get started. I also went and looked at the
fsg search init code to get some help with how to generate the "rule" argument
that has to be passed to jsgf_build_fsg(). I think I'm getting there but there
is one thing I don't understand yet since it isn't analogous to the ARPA lm
API (as you warned me). Here is my method so far:

- (void) changePS:(ps_decoder_t *)pocketsphinxDecoder withLogmath:(logmath_t*)logmath languageModelIsJSGF:(BOOL)languageModelIsJSGF {

    if(languageModelIsJSGF) {

        jsgf_t *jsgf;
        fsg_model_t *fsg;
        jsgf_rule_t *rule;
        char const *path = (char *)[self.languageModelFileToChangeTo UTF8String];

        if ((jsgf = jsgf_parse_file(path, NULL)) == NULL) {
            NSLog(@"Error: no JSGF file at path.");     
        }

        rule = NULL;

        jsgf_rule_iter_t *itor;

        for (itor = jsgf_rule_iter(jsgf); itor;
        itor = jsgf_rule_iter_next(itor)) {
            rule = jsgf_rule_iter_rule(itor);
            if (jsgf_rule_public(rule)) {
                break;
            }
            if (rule == NULL) {
                NSLog(@"Error: No public rules found in %s", path);
            }
        }

        fsg = jsgf_build_fsg(jsgf, rule, logmath, 1.0);
        if (fsg_set_add(????, fsg_model_name(fsg), fsg) != fsg) {
            fsg_model_free(fsg);
            NSLog(@"Error: could not add finite state grammar to set.");        
        }

        if (fsg_set_select(????, fsg_model_name(fsg)) == NULL) {
            NSLog(@"Error: could not select new grammar.");     
        }

        ps_update_fsgset(pocketsphinxDecoder);

    }

}

If you see my arguments where I'm passing "????", that is something I'm
unclear about what it should be. In the ARPA version I would do this to get
the pointer to the old model to add the new model to:

        lmset_from_current_decoder = ps_get_lmset(pocketsphinxDecoder);
        ngram_model_set_add(lmset_from_current_decoder, new_lm, "my_id", 1.0, TRUE);

In the FSG version the equivalent looks like it would be:

       fsg_set_get_fsg(fsg_search_t *fsgs, const char *name);

Except that I don't get what the relationship is between a fsg_search_t and my
current decoder, so I don't know how to get a fsg_search_t that I can pass
into fsg_set_get_fsg() and then into fsg_set_add() that will end up making
changes to the language model of my current decoder as intended. Thanks for
your further assistance.

[Nickolay V. Shmyrev]

Hello

You can get fst_set with

POCKETSPHINX_EXPORT
fsg_set_t *ps_get_fsgset(ps_decoder_t *ps);

fsg_search_t and fsg_set_t are the same

typedef struct fsg_search_s fsg_set_t;

But I think it would be nice to unify this name to be consistent.

[Halle]

Ah, OK, I overlooked that. I think it's working now, thanks!