pocketsphinx gstreamer plugin initialization

[Anonymous]

I've installed pocketsphinx-0.7 and sphinxbase-0.7.
pocketsphinx_continuous runs and can pick up my voice.
I followed the hello world example and wrote a demo cpp program, which
compiled and worked fine.

However, when I tried to use the pocketsphinx gstreamer plugin, I had some
problem with initializing gstreamer properly.
The tutorial I followed for this is found at: http://cmusphinx.sourceforge.ne
t/wiki/gstreamer

The program seems to complete the initialization:
INFO: fsg_search.c(364): Added 38 alternate word transitions
INFO: fsg_lextree.c(108): Allocated 21930 bytes (21 KiB) for left and right
context phones
INFO: fsg_lextree.c(251): 4022 HMM nodes in lextree (3462 leaves)
INFO: fsg_lextree.c(253): Allocated 514816 bytes (502 KiB) for all lextree
nodes
INFO: fsg_lextree.c(256): Allocated 443136 bytes (432 KiB) for lextree
leafnodes

I tried speaking after pressing the "Speak" button. However, the program does
not recognize my speech. Neither does it throw any error.

I've ensured that I added the directory to which the pocketsphinx gstreamer
plugin is installed:
export GST_PLUGIN_PATH=/usr/local/lib/gstreamer-0.10

As a work-around, I can run audacity, which seems to initializes gstreamer
and/or the pocketsphinx plugin properly.
After audacity at least once, the aforementioned pocksphinx-gstreamer demo
works as expected, until the next reboot.

I tried writing an equivalent C++ program to use the pocketsphinx-gstreamer
plugin, which had the some problem of incomplete initialization.
See attached file.

The relevant part:

 g_thread_init(NULL);

 GError *err;

 // Initialize gstreamer
 if (!gst_init_check(&argc, &argv, &err)) {
 g_print("Error: Failed to initialize Gstreamer. Error %d: %s", err->code, err->message);
 g_error_free(err);
 }

 GstElement *pipeline;
 GstBus *bus;
 GError *error = NULL;

 loop = g_main_loop_new(NULL, false);

 // Create pipeline based on command line syntax
 // audio source -> conversion -> resampling -> voice activity detection -> recognition -> fakesink
 // VADER = Voice Activity DEtectoR 
 // pocketsphinx element requires 8kHz, 16-bit PCM audio
 // fakesink discards the final output
 pipeline = gst_parse_launch("alsasrc ! audioconvert ! audioresample"
 "! vader name=vad auto_threshold=true "
 "! pocketsphinx name=asr ! fakesink",
 &error);

 if (!pipeline) g_error("%s", error->message);

 //GstElement *vad;
 //vad = gst_bin_get_by_name(GST_BIN(pipeline), "vad");
 //g_object_set(G_OBJECT(vad), "silent", true, NULL);

 GstElement *asr;
 asr = gst_bin_get_by_name(GST_BIN(pipeline), "asr");
 // N.B. jsgf parameter is not supported by the pocketsphinx plugin for gstreamer!
 g_object_set(G_OBJECT(asr), "fsg", "navigation.fsg", NULL);
 //g_object_set(G_OBJECT(asr), "dict", MODEL_DIR "/lm/en/turtle.dic", NULL);
 //g_object_set(G_OBJECT(asr), "lm", MODEL_DIR "/lm/en/turtle.DMP", NULL);

 // Connect signal from asr to call-back functions
 // s.t. functions are called whenever a partial or complete utterance is decoded
 g_signal_connect(asr, "partial_result", G_CALLBACK(on_asr_partial_result), NULL);
 g_signal_connect(asr, "result", G_CALLBACK(on_asr_result), NULL);
 // force ars to be initialized
 g_object_set(G_OBJECT(asr), "configured", true, NULL);

 // Add call-back function to handle messages
 bus = gst_pipeline_get_bus(GST_PIPELINE(pipeline));
 gst_bus_add_watch(bus, bus_call, NULL);
 gst_object_unref(bus);

 // Start playing
 gst_element_set_state(GST_ELEMENT(pipeline), GST_STATE_PLAYING);

 g_main_loop_run(loop);

 // Stop pipeline and free it
 gst_element_set_state(GST_ELEMENT(pipeline), GST_STATE_NULL);
 gst_object_unref(GST_OBJECT(pipeline));

What initialization step am I missing?

 $uname -a
 Linux samurai 2.6.39-ARCH #1 SMP PREEMPT Mon Jun 6 22:37:55 CEST 2011 x86_64 Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz GenuineIntel GNU/Linux

 alsa-lib 1.0.24.1-1
 alsa-plugins 1.0.24-2
 alsa-utils 1.0.24.2-1

[Nickolay V. Shmyrev]

Hi

Can you properly record audio with this gstreamer pipeline ? Can you add a
branch and dump stream to a file?

[Anonymous]

Nothing works after a fresh restart before launching audacity.

I tried recording to a file with:
gst-launch-0.10 alsasrc ! audioconvert ! audioresample ! vorbisenc ! oggmux !
filesink location=test.ogg

As above, this command records a silent test.ogg (of the right length).

Everything works fine after I launch audacity at least once.

I tried looking into the Audacity source file to see what initialization step
I am missing, though I couldn't find anything in the gstreamer initialization
code.
Perhaps it is some initialization step before gstreamer initialization?

Additionally, I am using KDE instead of Gnome, and I am using alsa. Does this
have something to do with my problem?

[Nickolay V. Shmyrev]

Maybe recording volume is muted or set to 0 after restart. Dump mixer state
before and after audacity start. Just run "amixer" to dump to stdout or to
file.