[Mosesdecoder-commits] SF.net SVN: mosesdecoder:[1870] trunk/moses/src/LanguageModelIRST.cpp

[<mfe...@us...>]

Revision: 1870
          http://mosesdecoder.svn.sourceforge.net/mosesdecoder/?rev=1870&view=rev
Author:   mfederico
Date:     2008-08-04 13:06:52 +0000 (Mon, 04 Aug 2008)

Log Message:
-----------
Fixed bug concerning the handling of the oov penalty with IRSTLM
Now, the penalty for out-of-vocabulary words is specified 
by the parameter 

-lmodel-dub: dictionary upper bounds of language models

For instance, if you set it lmodel-dub to 1000000 (1M) and your actual 
vocabulary is let me say 200000 (200K), then the LM probabilty  of the
OOV word-class is divided by 800000 (800K), i.e. 1M-200K

You have to make sure that lmodel-dub is always larger than the LM 
dictionary.

Modified Paths:
--------------
    trunk/moses/src/LanguageModelIRST.cpp


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.