Re: [Kaldi-developers] Word insertion penalty in the decoder

[Paul D. <pau...@gm...>]

You can probably just modify the token expansion code in any of the
decoders. Find the code that iterates over the outgoing arcs and
if an arc has output label add an insertion penalty. The below modified
fragment is from faster-decoder.cc. The emitting and non-emitting expansion
will both need to changed. In the fst the output labels won't be
synchronised
and therefore the insertion penalty will not come at the start or end a
word, but
there should be a correct number of insertions along a path. This probably
doesn't matter
in practise.

Paul

for (fst::ArcIterator<fst::Fst<Arc> > aiter(fst_, state);
        !aiter.Done(); aiter.Next()) {
         Arc arc = aiter.Value();
          if (arc.ilabel != 0) {
           Weight ac_weight(- decodable->LogLikelihood(frame, arc.ilabel));
        BaseFloat new_weight = arc.weight.Value() + tok->weight_.Value()
            + ac_weight.Value();

        if (arc.olabel) //Word label so add insertion penalty
          new_weight += insertion penalty;
       ...
       ...
       ...
       ...


On 7 March 2014 13:45, Augusto Henrique Hentz <au...@li...> wrote:

> Hello everyone,
>
> I would like to have a word insertion penalty option in the decoders. I am
> aware of the `lattice-add-inspen' program, but I think it would be nicer if
> this option was inside the decoder core.
>
> In a recent discussion on the forum, Daniel mentioned that
>
> > If you really wanted to bake the insertion penalty into the decoding
> graph,
> > you could just add something to the graph cost for any arcs that have
> word
> > labels on them.
> (
> https://sourceforge.net/p/kaldi/discussion/1355348/thread/b07e145d/#2d84/a3b4
> )
>
> This interests me. Would anyone care to elaborate? I'm thinking of the
> BiglmFasterDecoder class. Is the PropagateLm method a good place to add
> this graph cost?
>
> Thanks
>
> --
> Augusto Henrique Hentz
> au...@li...
>
>
>
>
>
>
> ------------------------------------------------------------------------------
> Subversion Kills Productivity. Get off Subversion & Make the Move to
> Perforce.
> With Perforce, you get hassle-free workflows. Merge that actually works.
> Faster operations. Version large binaries.  Built-in WAN optimization and
> the
> freedom to use Git, Perforce or both. Make the move to Perforce.
>
> http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
> _______________________________________________
> Kaldi-developers mailing list
> Kal...@li...
> https://lists.sourceforge.net/lists/listinfo/kaldi-developers
>