python Pocketsphinx speech recognition

[Andrea B]

Hi Guys,
I'm trying to use pocketsphinx to create an application able to undestand few words in order to link that to an action. I'm using this code:

from pocketsphinx import *
import pyaudio
import sys

hmm = ("/home/user/AudioProject/pocketsphinx-5prealpha/model/en-us/en-us")
dic =("/home/user/Desktop/4525.dict")
lm =("/home/user/AudioProject/pocketsphinx-5prealpha/model/en-us/en-us.lm.dmp")

config = Decoder.default_config()
config.set_string('-hmm', hmm)
config.set_string('-lm', lm)
config.set_string('-dict', dic)
config.set_string('-logfn', '/dev/null')

decoder = Decoder(config)

p = pyaudio.PyAudio()

stream = p.open(format=pyaudio.paInt16, channels=1, rate=16000, input=True, frames_per_buffer=1024)
stream.start_stream()
in_speech_bf = True
decoder.start_utt()
while True:
buf = stream.read(1024)
if buf:
decoder.process_raw(buf, False, False)
try:
if decoder.hyp().hypstr != '':
print 'Partial decoding result:', decoder.hyp().hypstr
except AttributeError:
pass
if decoder.get_in_speech():
sys.stdout.write('.')
sys.stdout.flush()
if decoder.get_in_speech() != in_speech_bf:
in_speech_bf = decoder.get_in_speech()
if not in_speech_bf:
decoder.end_utt()
try:
if decoder.hyp().hypstr != '':
print 'Stream decoding result:', decoder.hyp().hypstr
except AttributeError:
pass
decoder.start_utt()
else:
break
decoder.end_utt()
print 'An Error occured:', decoder.hyp().hypstr

my problem is that if I say another word the decoder force the solution linking any word to the few words in the dictionary.
Anyone have some suggets to me?
thanks to all

[Nickolay V. Shmyrev]

You can use keyword spotting mode to look for specific keyphrases.

[Andrea B]

it will solve my problem?
thanks for your suggets :)

[Nickolay V. Shmyrev]

Yes

[Andrea B]

doesn't work:(
if I say "NO" the decoder reply with "play"(one of the command) or with " skip"(another command)....last week I did something similar with Google API...this time I want to do it with an offline API...much more difficult....

[Nickolay V. Shmyrev]

It is very hard to look for short words in continuous stream reliably. Try use longer commands or single activation keyword like "ok google" to activate the command. Once keyword is recognized you can start recognition with a grammar.

[Andrea B]

ok thanks :) i'll try soon

[Matt]

Hello Nickolay,

I read many threads where you advise switching from keyword to grammar search and I found Java and C codes using this method but how about Python ? I went through the Swig source : I can see set_search and get_search but no function for declaring a "named search"... it seems I can only set the global config with set_string. Am I missing the obvious ? Is there any python example out there showing how to switch between searches I could use as a boilerplate ?

Regards,
matt

[Nickolay V. Shmyrev]

This is an example for you, searches are created with set_lm_file and set_keyphrase and activated with ps_set_search:

from os import path

from pocketsphinx.pocketsphinx import *
from sphinxbase.sphinxbase import *

MODELDIR = "../../../model"
DATADIR = "../../../test/data"

# Create a decoder with certain model
config = Decoder.default_config()
config.set_string('-hmm', path.join(MODELDIR, 'en-us/en-us'))
config.set_string('-dict', path.join(MODELDIR, 'en-us/cmudict-en-us.dict'))

decoder = Decoder(config)

decoder.set_lm_file("lm", path.join(MODELDIR, 'en-us/en-us.lm.bin'))
decoder.set_keyphrase("kws", "go forward")

decoder.set_search("kws")

decoder.start_utt()
stream = open(path.join(DATADIR, 'goforward.raw'), 'rb')
while True:
  buf = stream.read(1024)
  if buf:
    decoder.process_raw(buf, False, False)
  else:
    break
decoder.end_utt()
print decoder.hyp().hypstr

decoder.set_search("lm")

decoder.start_utt()
stream = open(path.join(DATADIR, 'goforward.raw'), 'rb')
while True:
  buf = stream.read(1024)
  if buf:
    decoder.process_raw(buf, False, False)
  else:
    break
decoder.end_utt()
print decoder.hyp().hypstr

[Matt]

Thank you for your quick answer Nickolay. This is exactly what I needed to keep me going.
I must have been blind while I was reading pocketsphinx.py, now that I see the functions in use ! Can't wait to test them... so back to my code.
I don't know at which level you are involved in PocketSphinx... but I want to say : keep up the good work on this beautiful project !
Regards. M.

[Yurii]

Hello. Nickolay, I have tried to use your example with Russian language key phrase, but nothing is working (it look like the phrase is not recognized). Can you please see what is wrong:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62

#!/usr/bin/env python # -*- coding: utf-8 -*- import os import sys from ctypes import * from contextlib import contextmanager import pyaudio from pocketsphinx.pocketsphinx import * from sphinxbase.sphinxbase import * script_dir = os.path.dirname(os.path.realpath(__file__)) model_dir = "/home/yura/practics/" hmm = os.path.join(model_dir, "zero_ru_cont_8k_v3/zero_ru.cd_cont_4000") lm = os.path.join(model_dir, "speech_reconginition_project/lmbase.lm.DMP") dict = os.path.join(model_dir, "speech_reconginition_project/words.dict") #sys.stderr = open(os.path.join(script_dir, "stderr.log"), "a") ERROR_HANDLER_FUNC = CFUNCTYPE(None, c_char_p, c_int, c_char_p, c_int, c_char_p) def py_error_handler(filename, line, function, err, fmt): pass c_error_handler = ERROR_HANDLER_FUNC(py_error_handler) @contextmanager def noalsaerr(): asound = cdll.LoadLibrary('libasound.so') asound.snd_lib_error_set_handler(c_error_handler) yield asound.snd_lib_error_set_handler(None) config = Decoder.default_config() config.set_string('-hmm', hmm) #config.set_string('-lm', lm) config.set_string('-dict', dict) config.set_string('-logfn', 'log.log') decoder = Decoder(config) decoder.set_lm_file("lm", lm) decoder.set_keyphrase("kws", "включить тест") decoder.set_search("kws") decoder.start_utt() with noalsaerr(): p = pyaudio.PyAudio() stream = p.open(format=pyaudio.paInt16, channels=1, rate=16000, input=True, frames_per_buffer=1024) stream.start_stream() in_speech_bf = True print 'Start.' while True: buf = stream.read(1024) if buf: decoder.process_raw(buf, False, False) else: break decoder.end_utt() print "End." print decoder.hyp().hypstr

[Nickolay V. Shmyrev]

You need to explain what do you mean by "nothing is working".

[Yurii]

I starting the program, waiting until "Start." is shown, saying "включить тест", but that is all - I can't see any response from the program after saying the phrase.

[Nickolay V. Shmyrev]

You need to configure keyword spotting threshold as described in our tutorial

http://cmusphinx.sourceforge.net/wiki/tutoriallm

[Yurii]

But how I can do it in python code? Can you give some example, please? Is there any code wiki with methods of pocketsphinx in python?

[Nickolay V. Shmyrev]

https://github.com/cmusphinx/pocketsphinx/blob/master/swig/python/test/kws_test.py

[Yurii]

Thank you. But how I can specify the list of key words? And also what is this threshold? Is there some theory of using it in speech recognition? I mean not only experimeting with it but understanding it too.

[Nickolay V. Shmyrev]

Is there some theory of using it in speech recognition? I mean not only experimeting with it but understanding it too.

You can get some introduction from wikipedia

https://en.wikipedia.org/wiki/Detection_theory

[derinam chere]

Hi guys how you???
please anyone help me
when i run this code on pythone spider invironment
it displays error which says
"
File "C:\Users\CRM\AppData\Local\Continuum\anaconda3\Lib\site-packages\pocketsphinx\pocketsphinx.py", line 273, in init
this = _pocketsphinx.new_Decoder(*args)

RuntimeError: new_Decoder returned -1"
the code i run is
from os import path

from pocketsphinx.pocketsphinx import *
from sphinxbase.sphinxbase import *

MODELDIR = "../../../model"
DATADIR = "../../../test/data"

Create a decoder with certain model

config = Decoder.default_config()
config.set_string('-hmm', path.join(MODELDIR, 'en-us/en-us'))
config.set_string('-dict', path.join(MODELDIR, 'en-us/cmudict-en-us.dict'))

decoder = Decoder(config)

decoder.set_lm_file("lm", path.join(MODELDIR, 'en-us/en-us.lm.bin'))
decoder.set_keyphrase("kws", "go forward")

decoder.set_search("kws")

decoder.start_utt()
stream = open(path.join(DATADIR, 'goforward.raw'), 'rb')
while True:
buf = stream.read(1024)
if buf:
decoder.process_raw(buf, False, False)
else:
break
decoder.end_utt()
print decoder.hyp().hypstr

decoder.set_search("lm")

decoder.start_utt()
stream = open(path.join(DATADIR, 'goforward.raw'), 'rb')
while True:
buf = stream.read(1024)
if buf:
decoder.process_raw(buf, False, False)
else:
break
decoder.end_utt()
print decoder.hyp().hypstr

please i have no idea about python and eager to know this language and help me

best regards
deriman
thanks