Menu

PocketSphinx Android dies when left listening

Help
Anonymous
2010-12-22
2012-09-22
  • Anonymous

    Anonymous - 2010-12-22

    Hi Everyone -

    I'm trying to use PocketSphinx on Android for semi-continuous voice
    recognition. By semi-continuous I mean an application that will be opened and
    left open and listening for at most 6 to 10 minutes. I will be using the
    partial hypothesis to do some processing. I know these hypothesis aren't
    always as good as the final but they accomplish our goals.

    What I've noticed is that if I leave my application (Which is just a modified
    version of the demo to listen on button click and stop on a second button
    click) listening for more then 2 - 3 minutes the application dies with the
    following information being sent to LogCat -

    12-22 15:00:25.114: INFO/ActivityManager(14242): Process
    edu.cmu.pocketsphinx.demo (pid 18438) has died.
    12-22 15:00:25.122: DEBUG/AudioHardwareMot(14229): Input 0x1dd58 entering
    standby
    12-22 15:00:25.130: DEBUG/AudioHardwareMot(14229):
    AudioStreamInMot::setParameters() routing=0
    12-22 15:00:25.130: DEBUG/AudioHardwareMot(14229): set input routing 0
    12-22 15:00:25.130: DEBUG/AudioHardwareMot(14229): doRouting called
    12-22 15:00:25.130: DEBUG/AudioHardwareMot(14229): Stereo output exists; using
    0x1
    12-22 15:00:25.130: DEBUG/AudioHardwareMot(14229): Routing to earpiece
    12-22 15:00:25.130: DEBUG/AudioHardwareMot(14229): doRouting: setup input path
    12-22 15:00:25.130: DEBUG/AudioPostProcessor(14229): slider opened, coeffs
    device for handset =0
    12-22 15:00:25.130: DEBUG/AudioHardwareMot(14229): Gain for Input accy = 9 is
    1c1c
    12-22 15:00:25.130: INFO/WindowManager(14242): WIN DEATH: Window{45ca41b0
    edu.cmu.pocketsphinx.demo/edu.cmu.pocketsphinx.demo.ExploriaTest paused=false}
    12-22 15:00:25.137: DEBUG/Zygote(14230): Process 18438 exited cleanly (255)
    12-22 15:00:25.145: INFO/UsageStats(14242): Unexpected resume of
    com.fede.launcher while already resumed in edu.cmu.pocketsphinx.demo

    Here is the total output in the PocketSphinx log file for the session. Doesn't
    seem like there's much of interest here but I'll post it anyways :)

    INFO: cmd_ln.c(512): Parsing command line:

    Current configuration:

    -agc none none
    -agcthresh 2.0 2.000000e+00
    -alpha 0.97 9.700000e-01
    -ascale 20.0 2.000000e+01
    -aw 1 1
    -backtrace no no
    -beam 1e-48 1.000000e-48
    -bestpath yes yes
    -bestpathlw 9.5 9.500000e+00
    -bghist no no
    -ceplen 13 13
    -cmn current current
    -cmninit 8.0 8.0
    -compallsen no no
    -debug 0
    -dict
    -dictcase no no
    -dither no no
    -doublebw no no
    -ds 1 1
    -fdict
    -feat 1s_c_d_dd 1s_c_d_dd
    -featparams
    -fillprob 1e-8 1.000000e-08
    -frate 100 100
    -fsg
    -fsgusealtpron yes yes
    -fsgusefiller yes yes
    -fwdflat yes yes
    -fwdflatbeam 1e-64 1.000000e-64
    -fwdflatefwid 4 4
    -fwdflatlw 8.5 8.500000e+00
    -fwdflatsfwin 25 25
    -fwdflatwbeam 7e-29 7.000000e-29
    -fwdtree yes yes
    -hmm
    -input_endian little little
    -jsgf
    -kdmaxbbi -1 -1
    -kdmaxdepth 0 0
    -kdtree
    -latsize 5000 5000
    -lda
    -ldadim 0 0
    -lextreedump 0 0
    -lifter 0 0
    -lm
    -lmctl
    -lmname default default
    -logbase 1.0001 1.000100e+00
    -logfn
    -logspec no no
    -lowerf 133.33334 1.333333e+02
    -lpbeam 1e-40 1.000000e-40
    -lponlybeam 7e-29 7.000000e-29
    -lw 6.5 6.500000e+00
    -maxhmmpf -1 -1
    -maxnewoov 20 20
    -maxwpf -1 -1
    -mdef
    -mean
    -mfclogdir
    -min_endfr 0 0
    -mixw
    -mixwfloor 0.0000001 1.000000e-07
    -mllr
    -mmap yes yes
    -ncep 13 13
    -nfft 512 512
    -nfilt 40 40
    -nwpen 1.0 1.000000e+00
    -pbeam 1e-48 1.000000e-48
    -pip 1.0 1.000000e+00
    -pl_beam 1e-10 1.000000e-10
    -pl_pbeam 1e-5 1.000000e-05
    -pl_window 0 0
    -rawlogdir
    -remove_dc no no
    -round_filters yes yes
    -samprate 16000 1.600000e+04
    -seed -1 -1
    -sendump
    -senlogdir
    -senmgau
    -silprob 0.005 5.000000e-03
    -smoothspec no no
    -svspec
    -tmat
    -tmatfloor 0.0001 1.000000e-04
    -topn 4 4
    -topn_beam 0 0
    -toprule
    -transform legacy legacy
    -unit_area yes yes
    -upperf 6855.4976 6.855498e+03
    -usewdphones no no
    -uw 1.0 1.000000e+00
    -var
    -varfloor 0.0001 1.000000e-04
    -varnorm no no
    -verbose no no
    -warp_params
    -warp_type inverse_linear inverse_linear
    -wbeam 7e-29 7.000000e-29
    -wip 0.65 6.500000e-01
    -wlen 0.025625 2.562500e-02

    INFO: cmd_ln.c(512): Parsing command line:
    \
    -nfilt 20 \
    -lowerf 1 \
    -upperf 4000 \
    -wlen 0.025 \
    -transform dct \
    -round_filters no \
    -remove_dc yes \
    -svspec 0-12/13-25/26-38 \
    -feat 1s_c_d_dd \
    -agc none \
    -cmn current \
    -cmninit 56,-3,1 \
    -varnorm no

    Current configuration:

    -agc none none
    -agcthresh 2.0 2.000000e+00
    -alpha 0.97 9.700000e-01
    -ceplen 13 13
    -cmn current current
    -cmninit 8.0 56,-3,1
    -dither no no
    -doublebw no no
    -feat 1s_c_d_dd 1s_c_d_dd
    -frate 100 100
    -input_endian little little
    -lda
    -ldadim 0 0
    -lifter 0 0
    -logspec no no
    -lowerf 133.33334 1.000000e+00
    -ncep 13 13
    -nfft 512 512
    -nfilt 40 20
    -remove_dc no yes
    -round_filters yes no
    -samprate 16000 8.000000e+03
    -seed -1 -1
    -smoothspec no no
    -svspec 0-12/13-25/26-38
    -transform legacy dct
    -unit_area yes yes
    -upperf 6855.4976 4.000000e+03
    -varnorm no no
    -verbose no no
    -warp_params
    -warp_type inverse_linear inverse_linear
    -wlen 0.025625 2.500000e-02

    INFO: acmod.c(238): Parsed model-specific feature parameters from /sdcard/Andr
    oid/data/edu.cmu.pocketsphinx.demo/hmm/en_US/hub4wsj_sc_8k/feat.params
    INFO: feat.c(860): Initializing feature stream to type: '1s_c_d_dd',
    ceplen=13, CMN='current', VARNORM='no', AGC='none'
    INFO: cmn.c(142): mean= 12.00, mean= 0.0
    INFO: acmod.c(163): Using subvector specification 0-12/13-25/26-38
    INFO: mdef.c(520): Reading model definition:
    /sdcard/Android/data/edu.cmu.pocketsphinx.demo/hmm/en_US/hub4wsj_sc_8k/mdef
    INFO: mdef.c(531): Found byte-order mark BMDF, assuming this is a binary mdef
    file
    INFO: bin_mdef.c(330): Reading binary model definition:
    /sdcard/Android/data/edu.cmu.pocketsphinx.demo/hmm/en_US/hub4wsj_sc_8k/mdef
    INFO: bin_mdef.c(507): 50 CI-phone, 143047 CD-phone, 3 emitstate/phone, 150
    CI-sen, 5150 Sen, 27135 Sen-Seq
    INFO: tmat.c(205): Reading HMM transition probability matrices: /sdcard/Androi
    d/data/edu.cmu.pocketsphinx.demo/hmm/en_US/hub4wsj_sc_8k/transition_matrices
    INFO: acmod.c(117): Attempting to use SCHMM computation module
    INFO: ms_gauden.c(198): Reading mixture gaussian parameter:
    /sdcard/Android/data/edu.cmu.pocketsphinx.demo/hmm/en_US/hub4wsj_sc_8k/means
    INFO: ms_gauden.c(292): 1 codebook, 3 feature, size:
    INFO: ms_gauden.c(294): 256x13
    INFO: ms_gauden.c(294): 256x13
    INFO: ms_gauden.c(294): 256x13
    INFO: ms_gauden.c(198): Reading mixture gaussian parameter: /sdcard/Android/da
    ta/edu.cmu.pocketsphinx.demo/hmm/en_US/hub4wsj_sc_8k/variances
    INFO: ms_gauden.c(292): 1 codebook, 3 feature, size:
    INFO: ms_gauden.c(294): 256x13
    INFO: ms_gauden.c(294): 256x13
    INFO: ms_gauden.c(294): 256x13
    INFO: ms_gauden.c(354): 0 variance values floored
    INFO: s2_semi_mgau.c(908): Loading senones from dump file
    /sdcard/Android/data/edu.cmu.pocketsphinx.demo/hmm/en_US/hub4wsj_sc_8k/sendump
    INFO: s2_semi_mgau.c(932): BEGIN FILE FORMAT DESCRIPTION
    INFO: s2_semi_mgau.c(1027): Using memory-mapped I/O for senones
    INFO: s2_semi_mgau.c(1304): Maximum top-N: 4 Top-N beams: 0 0 0
    INFO: phone_loop_search.c(105): State beam -230231 Phone exit beam -115115
    Insertion penalty 0
    INFO: dict.c(306): Allocating 4862 * 20 bytes (94 KiB) for word entries
    INFO: dict.c(321): Reading main dictionary:
    /sdcard/Android/data/edu.cmu.pocketsphinx.demo/lm/en_US/3956.dic
    INFO: dict.c(212): Allocated 5 KiB for strings, 9 KiB for phones
    INFO: dict.c(324): 755 words read
    INFO: dict.c(330): Reading filler dictionary: /sdcard/Android/data/edu.cmu.poc
    ketsphinx.demo/hmm/en_US/hub4wsj_sc_8k/noisedict
    INFO: dict.c(212): Allocated 0 KiB for strings, 0 KiB for phones
    INFO: dict.c(333): 11 words read
    INFO: dict2pid.c(396): Building PID tables for dictionary
    INFO: dict2pid.c(404): Allocating 50^3 * 2 bytes (244 KiB) for word-initial
    triphones
    INFO: dict2pid.c(131): Allocated 30200 bytes (29 KiB) for word-final triphones
    INFO: dict2pid.c(195): Allocated 30200 bytes (29 KiB) for single-phone word
    triphones
    INFO: ngram_model_arpa.c(477): ngrams 1=654, 2=1304, 3=652
    INFO: ngram_model_arpa.c(135): Reading unigrams
    INFO: ngram_model_arpa.c(516): 654 = #unigrams created
    INFO: ngram_model_arpa.c(195): Reading bigrams
    INFO: ngram_model_arpa.c(533): 1304 = #bigrams created
    INFO: ngram_model_arpa.c(534): 3 = #prob2 entries
    INFO: ngram_model_arpa.c(542): 3 = #bo_wt2 entries
    INFO: ngram_model_arpa.c(292): Reading trigrams
    INFO: ngram_model_arpa.c(555): 652 = #trigrams created
    INFO: ngram_model_arpa.c(556): 2 = #prob3 entries
    INFO: ngram_search_fwdtree.c(99): 248 unique initial diphones
    INFO: ngram_search_fwdtree.c(147): 0 root, 0 non-root channels, 16 single-
    phone words
    INFO: ngram_search_fwdtree.c(186): Creating search tree
    INFO: ngram_search_fwdtree.c(191): before: 0 root, 0 non-root channels, 16
    single-phone words
    INFO: ngram_search_fwdtree.c(326): after: max nonroot chan increased to 2905
    INFO: ngram_search_fwdtree.c(338): after: 248 root, 2777 non-root channels, 15
    single-phone words
    INFO: ngram_search_fwdflat.c(156): fwdflat: min_ef_width = 4, max_sf_win = 25

    Any idea why it's only running for a few minutes and then the whole app is
    exiting out? It's not like it just stops listening or something, the whole app
    just crashes without a force close or anything.

    Thanks in advance :)

     
  • Nickolay V. Shmyrev

    I think it just goes out of memory. it make sense to try with smaller model.

    Anyway, I suggest you to learn how to use gdbserver on android and how to
    collect backtraces. That greatly simplifies bug finding

     
  • Anonymous

    Anonymous - 2010-12-23

    Thanks for the tip nshmyrev :) Is gdbserver part of the PocketSphinx or
    something different all together? Do you know of any resources I can check
    into about it?

     
  • Anonymous

    Anonymous - 2010-12-23

    Working with your "Out of Memory" theory I was able to get the demo to listen
    "continuously". This is not the BEST solution but it works for my prototype.

    What I do is I start a timer that fires every minute. Within the timer I kill
    the running recognition listener (stop() and shutdown()) and also the thread
    it's using to listen on. I then recreate the recognition listener and thread
    and restart both of them. Watching the LogCat I can see this is allowing the
    GC to free up any memory used by the PocketSphinx library and/or the
    AudioManager.

    There is a slight loss of recognition due to shutting down and restarting but
    my tests have shown it's less than 5 seconds, so nothing too crazy. My tests
    have also shown that using this approach I can keep PocketSphinx listening for
    upwards of 20 minutes (I stopped after 18 minutes as our need is only a max of
    10 minutes).

    I would love a more eloquent solution but this works :)

     
  • Nickolay V. Shmyrev

    Is gdbserver part of the PocketSphinx or something different all together?

    http://www.google.com/search?q=android+debug+gdbserver

    I would love a more eloquent solution but this works :

    Maybe you want to read pocketsphinx_continuous sources in file continuous.c to
    learn to to implement continuous listening using pocketsphinx API.

     
  • Anonymous

    Anonymous - 2010-12-28

    Thanks for the reply nshmyrev! I looked into the continuous.c code and it
    seems I would have to port over a bunch more code to Android. Unfortunately
    I'm not versed enough in C and Native Android development to feel comfortable
    doing such a thing.

    Do you know who created the Android port of PocketSphinx? I would like to see
    if they'd let me pick their brain for some advice on getting started porting
    over the continuous stuff to Android.

    Thanks :)

     

Log in to post a comment.