Menu

In which condition backoff weight can be positive?

Biswajit
2014-07-28
2014-08-16
  • Biswajit

    Biswajit - 2014-07-28

    example:

    I know that if prob. is greater than "1", it can be positive value in log domain.

    ~~~~~~~~~~~~~~~~
    1gram:
    -4.1258 above 0.0169

    2gram:
    -1.6579 above
    -1.6579 above all
    -1.6579 above him
    -1.6579 above his
    -1.6579 above the
    ~~~~~~~~~~~~~~~~~~~~

    Total count of above is "5". Total vocabulary size: 6171
    How toolkit is calculating BW here with Good Turing?
    Freq of those bigrams are 1.
    How it will estimate new count for 1 by GT because it is the highest frequency. I'll not get count for 2 as well.

     

    Last edit: Nickolay V. Shmyrev 2014-08-16
  • Biswajit

    Biswajit - 2014-07-29

    Any clarification on this ??

     
    • Nickolay V. Shmyrev

      For the most precise and exact explanation of the calculation process you can read the code.

       

Log in to post a comment.