[moses-smt/mosesdecoder] 5ed734: branch for mtm 11 project

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

  Branch: refs/heads/mtm_lm
  Home:   https://github.com/moses-smt/mosesdecoder
  Commit: 5ed73450dd4baa60d7916e07e74cc30a4d985abf
      https://github.com/moses-smt/mosesdecoder/commit/5ed73450dd4baa60d7916e07e74cc30a4d985abf
  Author: hieuhoang1972 <hieuhoang1972@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-05 (Mon, 05 Sep 2011)

  Changed paths:
    M config.h.in

  Log Message:
  -----------
  branch for mtm 11 project

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4171 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: 65cb7eb040f6edc4ba29e9cff9dafe9e67558931
      https://github.com/moses-smt/mosesdecoder/commit/65cb7eb040f6edc4ba29e9cff9dafe9e67558931
  Author: hieuhoang1972 <hieuhoang1972@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-06 (Tue, 06 Sep 2011)

  Changed paths:
    M kenlm/kenlm.xcodeproj/project.pbxproj
    M moses-chart-cmd/moses-chart-cmd.xcodeproj/project.pbxproj
    M moses/moses.xcodeproj/project.pbxproj

  Log Message:
  -----------
  xcode macros for large file support

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4174 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: 3d1ba8ffd6f8052294c2fd56ed5384525f685b9c
      https://github.com/moses-smt/mosesdecoder/commit/3d1ba8ffd6f8052294c2fd56ed5384525f685b9c
  Author: heafield <heafield@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-06 (Tue, 06 Sep 2011)

  Changed paths:
    M kenlm/lm/model.cc
    M kenlm/lm/model.hh

  Log Message:
  -----------
  Log every call


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4176 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: 96e98e9c8571cac54ac3ff60e8cc46e8afc17c3c
      https://github.com/moses-smt/mosesdecoder/commit/96e98e9c8571cac54ac3ff60e8cc46e8afc17c3c
  Author: hieuhoang1972 <hieuhoang1972@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-06 (Tue, 06 Sep 2011)

  Changed paths:
    M kenlm/lm/model.cc
    M kenlm/lm/model.hh

  Log Message:
  -----------
  logging object is global variable. for xcode compile

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4177 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: e20d87fc92b98b3d16b09a39e8f647d4f6e4f9b5
      https://github.com/moses-smt/mosesdecoder/commit/e20d87fc92b98b3d16b09a39e8f647d4f6e4f9b5
  Author: heafield <heafield@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-06 (Tue, 06 Sep 2011)

  Changed paths:
    M kenlm/lm/model.cc

  Log Message:
  -----------
  Also dump n-gram length


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4178 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: 2580da7ffd9ec622c9d2228e4d251764f2aa7d0c
      https://github.com/moses-smt/mosesdecoder/commit/2580da7ffd9ec622c9d2228e4d251764f2aa7d0c
  Author: hieuhoang1972 <hieuhoang1972@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-06 (Tue, 06 Sep 2011)

  Changed paths:
    M moses/moses.xcodeproj/project.pbxproj
    A moses/src/LanguageModelChartState.cpp
    M moses/src/LanguageModelChartState.h
    M moses/src/Makefile.am

  Log Message:
  -----------
  LM left state class

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4180 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: 64bcd9813000551163a78b00fb3945f5f7911db8
      https://github.com/moses-smt/mosesdecoder/commit/64bcd9813000551163a78b00fb3945f5f7911db8
  Author: hieuhoang1972 <hieuhoang1972@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-06 (Tue, 06 Sep 2011)

  Changed paths:
    M moses/src/LanguageModel.cpp

  Log Message:
  -----------
  LM left state class

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4181 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: 86aeb4d064fcecbb4534c140fb32c9d520d13b1f
      https://github.com/moses-smt/mosesdecoder/commit/86aeb4d064fcecbb4534c140fb32c9d520d13b1f
  Author: heafield <heafield@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-08 (Thu, 08 Sep 2011)

  Changed paths:
    A kenlm/lm/left.hh

  Log Message:
  -----------
  Haven't tested it.  First try at chart state automaton


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4187 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: 474f1f170c5206df0c575f081fd4aac4e6b7cd53
      https://github.com/moses-smt/mosesdecoder/commit/474f1f170c5206df0c575f081fd4aac4e6b7cd53
  Author: heafield <heafield@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-08 (Thu, 08 Sep 2011)

  Changed paths:
    M kenlm/lm/left.hh

  Log Message:
  -----------
  Underscore


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4188 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: 07741df60498e1168506794540fc330edbea84dd
      https://github.com/moses-smt/mosesdecoder/commit/07741df60498e1168506794540fc330edbea84dd
  Author: hieuhoang1972 <hieuhoang1972@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-08 (Thu, 08 Sep 2011)

  Changed paths:
    M kenlm/lm/left.hh

  Log Message:
  -----------
  LM left state class

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4189 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: 6b9305445d48868d7ab013b21da35303945d59ce
      https://github.com/moses-smt/mosesdecoder/commit/6b9305445d48868d7ab013b21da35303945d59ce
  Author: heafield <heafield@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-08 (Thu, 08 Sep 2011)

  Changed paths:
    M kenlm/lm/left.hh

  Log Message:
  -----------
  Coding convention


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4190 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: db5c85a5c6ff83e22f5da01c4d19f09b45a1b845
      https://github.com/moses-smt/mosesdecoder/commit/db5c85a5c6ff83e22f5da01c4d19f09b45a1b845
  Author: heafield <heafield@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-08 (Thu, 08 Sep 2011)

  Changed paths:
    M kenlm/lm/left.hh

  Log Message:
  -----------
  Change some members to remove _


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4191 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: 8e91e79f723eeff146082f4c8830ff2272a4c01c
      https://github.com/moses-smt/mosesdecoder/commit/8e91e79f723eeff146082f4c8830ff2272a4c01c
  Author: hieuhoang1972 <hieuhoang1972@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-08 (Thu, 08 Sep 2011)

  Changed paths:
    M kenlm/kenlm.xcodeproj/project.pbxproj
    M kenlm/lm/left.hh
    M moses/moses.xcodeproj/project.pbxproj
    M moses/src/LanguageModel.cpp
    M moses/src/LanguageModelKen.cpp
    M moses/src/LanguageModelKen.h

  Log Message:
  -----------
  LM left state class

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4192 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: 0371ac043c5086e6933acc11f41424da71d3f7cf
      https://github.com/moses-smt/mosesdecoder/commit/0371ac043c5086e6933acc11f41424da71d3f7cf
  Author: hieuhoang1972 <hieuhoang1972@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-08 (Thu, 08 Sep 2011)

  Changed paths:
    M kenlm/lm/left.hh
    M moses/src/LanguageModel.cpp
    M moses/src/LanguageModelKen.cpp

  Log Message:
  -----------
  LM left state class

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4196 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: 085f7f02510dc5a8eda1692dc2778529d6f6c8c5
      https://github.com/moses-smt/mosesdecoder/commit/085f7f02510dc5a8eda1692dc2778529d6f6c8c5
  Author: hieuhoang1972 <hieuhoang1972@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-08 (Thu, 08 Sep 2011)

  Changed paths:
    M moses/src/LanguageModel.cpp

  Log Message:
  -----------
  LM left state class

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4197 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: 44452eba71d7bad7e56b2c813bfa9535edc039ed
      https://github.com/moses-smt/mosesdecoder/commit/44452eba71d7bad7e56b2c813bfa9535edc039ed
  Author: heafield <heafield@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-09 (Fri, 09 Sep 2011)

  Changed paths:
    M kenlm/lm/left.hh
    M moses/src/LanguageModel.cpp
    M moses/src/LanguageModel.h
    M moses/src/LanguageModelChartState.cpp
    M moses/src/LanguageModelChartState.h
    M moses/src/LanguageModelImplementation.cpp
    M moses/src/LanguageModelImplementation.h
    M moses/src/LanguageModelKen.cpp
    M moses/src/LanguageModelKen.h

  Log Message:
  -----------
  Move ChartScore to LanguageModelImplementation so we can use different ones.  Actually call RullScore.  Unsure if it works.  


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4198 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: 732b6b6a8f979c1e4e01a48abd43c7622334dcca
      https://github.com/moses-smt/mosesdecoder/commit/732b6b6a8f979c1e4e01a48abd43c7622334dcca
  Author: heafield <heafield@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-09 (Fri, 09 Sep 2011)

  Changed paths:
    M kenlm/lm/left.hh

  Log Message:
  -----------
  Fix BOS handling and state handling


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4199 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: 0bd8f3f0ee840222ad10eb9cc4fde2b72b4db45e
      https://github.com/moses-smt/mosesdecoder/commit/0bd8f3f0ee840222ad10eb9cc4fde2b72b4db45e
  Author: tetsuok <tetsuok@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-09 (Fri, 09 Sep 2011)

  Changed paths:
    M moses/src/LanguageModelChartState.h
    M moses/src/LanguageModelKen.cpp

  Log Message:
  -----------
  Fix indentation

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4201 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: ac090caae857ddd5a79371221c20be491233a934
      https://github.com/moses-smt/mosesdecoder/commit/ac090caae857ddd5a79371221c20be491233a934
  Author: tetsuok <tetsuok@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-09 (Fri, 09 Sep 2011)

  Changed paths:
    M moses/src/LanguageModel.cpp
    M moses/src/LanguageModelChartState.cpp
    M moses/src/LanguageModelKen.cpp

  Log Message:
  -----------
  Fix indentation and whitespace

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4202 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: aaf6d141b36d75991ff9cf430a53f13ab2276565
      https://github.com/moses-smt/mosesdecoder/commit/aaf6d141b36d75991ff9cf430a53f13ab2276565
  Author: heafield <heafield@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-09 (Fri, 09 Sep 2011)

  Changed paths:
    M kenlm/lm/left.hh

  Log Message:
  -----------
  Now with less segfaults


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4203 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: 9663286b42cd15db5b7556ff1c69840bafc9ae65
      https://github.com/moses-smt/mosesdecoder/commit/9663286b42cd15db5b7556ff1c69840bafc9ae65
  Author: heafield <heafield@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-09 (Fri, 09 Sep 2011)

  Changed paths:
    M moses/src/LanguageModelKen.cpp

  Log Message:
  -----------
  TransformLMScore


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4204 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: 3fa32ef950a2e00001d43c231651380cdd30ee03
      https://github.com/moses-smt/mosesdecoder/commit/3fa32ef950a2e00001d43c231651380cdd30ee03
  Author: heafield <heafield@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-09 (Fri, 09 Sep 2011)

  Changed paths:
    M moses/src/LanguageModelKen.cpp

  Log Message:
  -----------
  Apparently you don't TransformLMScore coming out. . . 


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4207 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: cdf8b0b9751073dd2ce6846e9045be3831100e74
      https://github.com/moses-smt/mosesdecoder/commit/cdf8b0b9751073dd2ce6846e9045be3831100e74
  Author: hieuhoang1972 <hieuhoang1972@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-09 (Fri, 09 Sep 2011)

  Changed paths:
    M moses-chart-cmd/src/IOWrapper.cpp

  Log Message:
  -----------
  fix n-best label for word penalty

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4208 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: 64ccfaddba6f13189d4dc87656096201015371a0
      https://github.com/moses-smt/mosesdecoder/commit/64ccfaddba6f13189d4dc87656096201015371a0
  Author: heafield <heafield@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-09 (Fri, 09 Sep 2011)

  Changed paths:
    M kenlm/lm/binary_format.cc
    M kenlm/lm/model.cc
    M kenlm/lm/model.hh
    M kenlm/lm/model_test.cc
    M kenlm/lm/search_hashed.cc
    M kenlm/lm/search_hashed.hh
    M kenlm/lm/search_trie.cc
    M kenlm/lm/search_trie.hh
    M kenlm/lm/virtual_interface.hh
    M kenlm/util/bit_packing.hh
    M kenlm/util/scoped.hh
    A kenlm/util/sized_iterator.hh

  Log Message:
  -----------
  KenLM that returns the right flag


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4212 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: 11f6f022ca4af7f6475a9d85fbd06ff3ae7b16de
      https://github.com/moses-smt/mosesdecoder/commit/11f6f022ca4af7f6475a9d85fbd06ff3ae7b16de
  Author: heafield <heafield@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-09 (Fri, 09 Sep 2011)

  Changed paths:
    M kenlm/lm/virtual_interface.hh

  Log Message:
  -----------
  Oops forgot this change


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4213 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: d0dd72aa171d7fd6f088d20a64f687ad4df6fcb0
      https://github.com/moses-smt/mosesdecoder/commit/d0dd72aa171d7fd6f088d20a64f687ad4df6fcb0
  Author: heafield <heafield@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-09 (Fri, 09 Sep 2011)

  Changed paths:
    M kenlm/lm/left.hh
    M moses/src/LanguageModelKen.cpp

  Log Message:
  -----------
  Still not recombining, but it's still working


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4214 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: a740dab61cb823019891aebd577e63afc16c076e
      https://github.com/moses-smt/mosesdecoder/commit/a740dab61cb823019891aebd577e63afc16c076e
  Author: heafield <heafield@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-12 (Mon, 12 Sep 2011)

  Changed paths:
    M kenlm/lm/left.hh
    A kenlm/lm/left_test.cc
    M kenlm/test.sh

  Log Message:
  -----------
  Fix left state bug.  First sentence now decodes correctly.  


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4215 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: eceb07b0ccb96cd0693861d73be5a44938d19872
      https://github.com/moses-smt/mosesdecoder/commit/eceb07b0ccb96cd0693861d73be5a44938d19872
  Author: heafield <heafield@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-12 (Mon, 12 Sep 2011)

  Changed paths:
    M kenlm/lm/left.hh
    M moses/src/LanguageModelKen.cpp

  Log Message:
  -----------
  Faster handling when X is on the left of a rule


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4217 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: 0110b93e13af226b94d00585f316f0c5896a63c8
      https://github.com/moses-smt/mosesdecoder/commit/0110b93e13af226b94d00585f316f0c5896a63c8
  Author: heafield <heafield@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-18 (Sun, 18 Sep 2011)

  Changed paths:
    M kenlm/lm/bhiksha.hh
    M kenlm/lm/binary_format.cc
    M kenlm/lm/binary_format.hh
    M kenlm/lm/left.hh
    M kenlm/lm/left_test.cc
    M kenlm/lm/model.cc
    M kenlm/lm/model.hh
    M kenlm/lm/model_test.cc
    A kenlm/lm/model_type.hh
    M kenlm/lm/quantize.cc
    M kenlm/lm/quantize.hh
    A kenlm/lm/return.hh
    M kenlm/lm/search_hashed.cc
    M kenlm/lm/search_hashed.hh
    M kenlm/lm/search_trie.cc
    M kenlm/lm/search_trie.hh
    M kenlm/lm/trie.cc
    M kenlm/lm/trie.hh
    M kenlm/lm/virtual_interface.hh
    M kenlm/lm/vocab.cc
    M kenlm/lm/vocab.hh
    M kenlm/util/exception.cc
    M kenlm/util/exception.hh
    M kenlm/util/file_piece.cc
    M kenlm/util/file_piece.hh
    M moses/src/LanguageModelKen.cpp

  Log Message:
  -----------
  Update kenlm.   Trie is currently broken for purposes of left combination.  Probing may have changed binary file formats.  


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4231 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: 7a5b96f1297bced3eef009a6fbec202c4d91da3d
      https://github.com/moses-smt/mosesdecoder/commit/7a5b96f1297bced3eef009a6fbec202c4d91da3d
  Author: heafield <heafield@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-19 (Mon, 19 Sep 2011)

  Changed paths:
    M kenlm/Makefile.am
    M kenlm/compile.sh
    M kenlm/lm/blank.hh
    M kenlm/lm/model.cc
    M kenlm/lm/quantize.cc
    M kenlm/lm/quantize.hh
    M kenlm/lm/search_hashed.cc
    M kenlm/lm/search_trie.cc
    A kenlm/lm/trie_sort.cc
    A kenlm/lm/trie_sort.hh
    M kenlm/lm/vocab.cc
    A kenlm/util/file.cc
    A kenlm/util/file.hh
    M kenlm/util/file_piece.cc
    M kenlm/util/file_piece.hh
    M kenlm/util/mmap.cc
    M kenlm/util/mmap.hh
    R kenlm/util/scoped.cc
    M kenlm/util/scoped.hh
    M kenlm/util/sized_iterator.hh

  Log Message:
  -----------
  Rewrite the trie builder to eliminate kBlankProb.  


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4234 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: 5db31f234c89cb5428cd70499551518e1649b72e
      https://github.com/moses-smt/mosesdecoder/commit/5db31f234c89cb5428cd70499551518e1649b72e
  Author: heafield <heafield@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-19 (Mon, 19 Sep 2011)

  Changed paths:
    M kenlm/lm/left.hh
    M kenlm/lm/left_test.cc

  Log Message:
  -----------
  Thought I fixed the issue by recombining more aggressively, but now I have an offending hypo


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4235 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: 1143bf1ea9203dd300f8b2f4930a26977515d26c
      https://github.com/moses-smt/mosesdecoder/commit/1143bf1ea9203dd300f8b2f4930a26977515d26c
  Author: heafield <heafield@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-19 (Mon, 19 Sep 2011)

  Changed paths:
    M kenlm/lm/left.hh

  Log Message:
  -----------
  Kludge comparison


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4236 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: 8cb9c3f6eb77bbf64cc4ef57b34f73d4ed7e4231
      https://github.com/moses-smt/mosesdecoder/commit/8cb9c3f6eb77bbf64cc4ef57b34f73d4ed7e4231
  Author: heafield <heafield@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-19 (Mon, 19 Sep 2011)

  Changed paths:
    M kenlm/lm/search_trie.cc

  Log Message:
  -----------
  Unsed parameter


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4237 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: 55d2f135237753a894d9801f8a01b1d8ca55c85f
      https://github.com/moses-smt/mosesdecoder/commit/55d2f135237753a894d9801f8a01b1d8ca55c85f
  Author: heafield <heafield@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-19 (Mon, 19 Sep 2011)

  Changed paths:
    M kenlm/lm/model_test.cc
    M kenlm/lm/trie_sort.cc

  Log Message:
  -----------
  Fix trie WriteOrThrow error.  Fix signed/unsigned comparison warning.  


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4238 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: f1206340a3621e5821f40f9344cbaa8dd2d2f7d3
      https://github.com/moses-smt/mosesdecoder/commit/f1206340a3621e5821f40f9344cbaa8dd2d2f7d3
  Author: heafield <heafield@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-20 (Tue, 20 Sep 2011)

  Changed paths:
    M kenlm/lm/left_test.cc
    M kenlm/lm/trie_sort.cc

  Log Message:
  -----------
  More tests, less unused parameters


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4239 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: 2dfd8011274b1abf39fd85f270a76095b959b1ae
      https://github.com/moses-smt/mosesdecoder/commit/2dfd8011274b1abf39fd85f270a76095b959b1ae
  Author: hieuhoang1972 <hieuhoang1972@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-23 (Fri, 23 Sep 2011)

  Changed paths:
    A kenlm/util/tokenize_piece.hh
    M mert/Data.cpp
    M mert/Data.h
    M mert/FeatureArray.cpp
    M mert/FeatureArray.h
    M mert/FeatureData.cpp
    M mert/FeatureData.h
    M mert/FeatureStats.cpp
    M mert/FeatureStats.h
    M mert/Makefile.am
    M mert/Point.cpp
    M mert/Point.h
    M mert/ScoreStats.cpp
    R mert/TERsrc/documentStructure.cpp
    R mert/TERsrc/documentStructure.h
    M mert/TERsrc/hashMap.h
    M mert/TERsrc/hashMapInfos.h
    M mert/TERsrc/hashMapStringInfos.h
    R mert/TERsrc/multiEvaluation.cpp
    R mert/TERsrc/multiEvaluation.h
    R mert/TERsrc/multiTxtDocument.cpp
    R mert/TERsrc/multiTxtDocument.h
    R mert/TERsrc/segmentStructure.cpp
    R mert/TERsrc/segmentStructure.h
    R mert/TERsrc/sgmlDocument.cpp
    R mert/TERsrc/sgmlDocument.h
    R mert/TERsrc/terEvaluation.h
    R mert/TERsrc/tinystr.cpp
    R mert/TERsrc/tinystr.h
    R mert/TERsrc/tinyxml.cpp
    R mert/TERsrc/tinyxml.h
    R mert/TERsrc/tinyxmlerror.cpp
    R mert/TERsrc/tinyxmlparser.cpp
    M mert/TERsrc/tools.cpp
    M mert/TERsrc/tools.h
    R mert/TERsrc/xmlStructure.cpp
    R mert/TERsrc/xmlStructure.h
    M mert/TerScorer.cpp
    M mert/TerScorer.h
    M mert/Types.h
    M mert/mert.cpp
    R mert/regression-testing/MertRegressionTesting.pm
    R mert/regression-testing/compare-results.pl
    R mert/regression-testing/run-single-test.pl
    R mert/regression-testing/run-test-suite
    R mert/regression-testing/tests/extractor-bin/command
    R mert/regression-testing/tests/extractor-bin/data/nbest.1.gz
    R mert/regression-testing/tests/extractor-bin/data/nbest.2.gz
    R mert/regression-testing/tests/extractor-bin/data/nbest.3.gz
    R mert/regression-testing/tests/extractor-bin/data/nbest.4.gz
    R mert/regression-testing/tests/extractor-bin/data/nbest.5.gz
    R mert/regression-testing/tests/extractor-bin/data/reference
    R mert/regression-testing/tests/extractor-bin/filter-stderr
    R mert/regression-testing/tests/extractor-bin/filter-stdout
    R mert/regression-testing/tests/extractor-bin/truth/results.dat
    R mert/regression-testing/tests/extractor-txt/command
    R mert/regression-testing/tests/extractor-txt/data/nbest.1.gz
    R mert/regression-testing/tests/extractor-txt/data/nbest.2.gz
    R mert/regression-testing/tests/extractor-txt/data/nbest.3.gz
    R mert/regression-testing/tests/extractor-txt/data/nbest.4.gz
    R mert/regression-testing/tests/extractor-txt/data/nbest.5.gz
    R mert/regression-testing/tests/extractor-txt/data/reference
    R mert/regression-testing/tests/extractor-txt/filter-stderr
    R mert/regression-testing/tests/extractor-txt/filter-stdout
    R mert/regression-testing/tests/extractor-txt/truth/results.dat
    R mert/regression-testing/tests/mert-basic/command
    R mert/regression-testing/tests/mert-basic/data/FEATSTAT.bin
    R mert/regression-testing/tests/mert-basic/data/FEATSTAT.txt
    R mert/regression-testing/tests/mert-basic/data/INIT
    R mert/regression-testing/tests/mert-basic/data/SCORESTAT.bin
    R mert/regression-testing/tests/mert-basic/data/SCORESTAT.txt
    R mert/regression-testing/tests/mert-basic/filter-stderr
    R mert/regression-testing/tests/mert-basic/filter-stdout
    R mert/regression-testing/tests/mert-basic/truth/results.dat
    R mert/tests/testmert.py
    M moses-chart-cmd/src/Main.cpp
    M moses-cmd/src/Main.cpp
    M moses/moses.vcxproj
    M moses/src/ChartCell.cpp
    M moses/src/ChartHypothesisCollection.cpp
    M moses/src/ChartRuleLookupManagerOnDisk.cpp
    M moses/src/Factor.cpp
    M moses/src/Factor.h
    M moses/src/FactorCollection.cpp
    M moses/src/FactorCollection.h
    M moses/src/GenerationDictionary.cpp
    M moses/src/LMList.cpp
    M moses/src/LMList.h
    M moses/src/LanguageModel.cpp
    M moses/src/LanguageModel.h
    M moses/src/LanguageModelIRST.cpp
    M moses/src/LanguageModelKen.cpp
    M moses/src/Parameter.cpp
    M moses/src/StaticData.cpp
    M moses/src/StaticData.h
    M moses/src/TargetPhrase.cpp
    M moses/src/ThreadPool.cpp
    M moses/src/ThreadPool.h
    M moses/src/TranslationOption.cpp
    A regression-testing/run-test-mert.perl
    M regression-testing/run-test-suite.perl
    A regression-testing/tests/mert.basic/command
    A regression-testing/tests/mert.basic/data/FEATSTAT.bin
    A regression-testing/tests/mert.basic/data/FEATSTAT.txt
    A regression-testing/tests/mert.basic/data/INIT
    A regression-testing/tests/mert.basic/data/SCORESTAT.bin
    A regression-testing/tests/mert.basic/data/SCORESTAT.txt
    A regression-testing/tests/mert.basic/filter-stderr
    A regression-testing/tests/mert.basic/filter-stdout
    A regression-testing/tests/mert.basic/truth/results.txt
    A regression-testing/tests/mert.extractor-bin/command
    A regression-testing/tests/mert.extractor-bin/data/nbest.1.gz
    A regression-testing/tests/mert.extractor-bin/data/nbest.2.gz
    A regression-testing/tests/mert.extractor-bin/data/nbest.3.gz
    A regression-testing/tests/mert.extractor-bin/data/nbest.4.gz
    A regression-testing/tests/mert.extractor-bin/data/nbest.5.gz
    A regression-testing/tests/mert.extractor-bin/data/reference
    A regression-testing/tests/mert.extractor-bin/filter-stderr
    A regression-testing/tests/mert.extractor-bin/filter-stdout
    A regression-testing/tests/mert.extractor-bin/truth/results.txt
    A regression-testing/tests/mert.extractor-txt/command
    A regression-testing/tests/mert.extractor-txt/data/nbest.1.gz
    A regression-testing/tests/mert.extractor-txt/data/nbest.2.gz
    A regression-testing/tests/mert.extractor-txt/data/nbest.3.gz
    A regression-testing/tests/mert.extractor-txt/data/nbest.4.gz
    A regression-testing/tests/mert.extractor-txt/data/nbest.5.gz
    A regression-testing/tests/mert.extractor-txt/data/reference
    A regression-testing/tests/mert.extractor-txt/filter-stderr
    A regression-testing/tests/mert.extractor-txt/filter-stdout
    A regression-testing/tests/mert.extractor-txt/truth/results.txt
    A regression-testing/tests/phrase.basic-lm-oov/filter-stderr.pl
    A regression-testing/tests/phrase.basic-lm-oov/filter-stdout.pl
    A regression-testing/tests/phrase.basic-lm-oov/moses.ini
    A regression-testing/tests/phrase.basic-lm-oov/to-translate.txt
    A regression-testing/tests/phrase.basic-lm-oov/truth/results.txt
    M regression-testing/tests/score.hierarchical-inv/args.txt
    M regression-testing/tests/score.hierarchical-inv/truth/results.txt
    M regression-testing/tests/score.hierarchical/args.txt
    M regression-testing/tests/score.hierarchical/truth/results.txt
    M regression-testing/tests/score.phrase-based-inv/args.txt
    M regression-testing/tests/score.phrase-based-inv/truth/results.txt
    M regression-testing/tests/score.phrase-based-with-alignment-inv/args.txt
    M regression-testing/tests/score.phrase-based-with-alignment-inv/truth/results.txt
    M regression-testing/tests/score.phrase-based-with-alignment/args.txt
    M regression-testing/tests/score.phrase-based-with-alignment/truth/results.txt
    M regression-testing/tests/score.phrase-based/args.txt
    M regression-testing/tests/score.phrase-based/truth/results.txt
    M scripts/Makefile
    A scripts/analysis/extract-target-trees.py
    M scripts/training/mert-moses.pl
    A scripts/training/phrase-extract/ExtractedRule.cpp
    M scripts/training/phrase-extract/ExtractedRule.h
    M scripts/training/phrase-extract/Hole.h
    M scripts/training/phrase-extract/Makefile
    M scripts/training/phrase-extract/PhraseAlignment.cpp
    M scripts/training/phrase-extract/PhraseAlignment.h
    M scripts/training/phrase-extract/RuleExtractionOptions.h
    M scripts/training/phrase-extract/SentenceAlignment.cpp
    M scripts/training/phrase-extract/SentenceAlignment.h
    M scripts/training/phrase-extract/consolidate.cpp
    M scripts/training/phrase-extract/extract-rules.cpp
    M scripts/training/phrase-extract/extract-rules.vcxproj
    M scripts/training/phrase-extract/extract.cpp
    M scripts/training/phrase-extract/extract.xcodeproj/project.pbxproj
    M scripts/training/phrase-extract/score.cpp
    M scripts/training/phrase-extract/tables-core.cpp
    M scripts/training/train-model.perl

  Log Message:
  -----------
  recreate branch from trunk

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4254 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: 33585c567b3b35ab98473ff525dbe5bad232406d
      https://github.com/moses-smt/mosesdecoder/commit/33585c567b3b35ab98473ff525dbe5bad232406d
  Author: hieuhoang1972 <hieuhoang1972@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-23 (Fri, 23 Sep 2011)

  Changed paths:
    M moses/src/ChartHypothesis.cpp
    M moses/src/ChartHypothesis.h
    M moses/src/LanguageModelChartState.cpp
    M moses/src/LanguageModelImplementation.cpp

  Log Message:
  -----------
  recreate branch from trunk

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4255 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: 5e5fd7f7ce81da3344cbd4db1eab6e46f1cb688f
      https://github.com/moses-smt/mosesdecoder/commit/5e5fd7f7ce81da3344cbd4db1eab6e46f1cb688f
  Author: hieuhoang1972 <hieuhoang1972@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-23 (Fri, 23 Sep 2011)

  Changed paths:
    M kenlm/Makefile.am
    M kenlm/kenlm.xcodeproj/project.pbxproj
    M kenlm/lm/left.hh
    M moses/src/LanguageModelKen.cpp

  Log Message:
  -----------
  recombination ratio

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4256 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: 4f5c891e7ac08ba9f052c35378af6c958d52328b
      https://github.com/moses-smt/mosesdecoder/commit/4f5c891e7ac08ba9f052c35378af6c958d52328b
  Author: hieuhoang1972 <hieuhoang1972@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-23 (Fri, 23 Sep 2011)

  Changed paths:
    M kenlm/lm/left.hh
    M moses/src/ChartHypothesis.cpp
    M moses/src/ChartHypothesis.h

  Log Message:
  -----------
  recombination ratio

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4257 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: dc369971f854148fccc35e530453da69ebf6e1eb
      https://github.com/moses-smt/mosesdecoder/commit/dc369971f854148fccc35e530453da69ebf6e1eb
  Author: hieuhoang1972 <hieuhoang1972@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-23 (Fri, 23 Sep 2011)

  Changed paths:
    A kenlm/lm/left.cc

  Log Message:
  -----------
  recombination ratio

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4258 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: ba92ad662d5e7cc38712d74e28e6721dbd6e75ef
      https://github.com/moses-smt/mosesdecoder/commit/ba92ad662d5e7cc38712d74e28e6721dbd6e75ef
  Author: hieuhoang1972 <hieuhoang1972@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-24 (Sat, 24 Sep 2011)

  Changed paths:
    M kenlm/kenlm.xcodeproj/project.pbxproj
    M kenlm/lm/left.cc
    M kenlm/lm/left.hh
    M moses-chart-cmd/src/Main.cpp

  Log Message:
  -----------
  recombination ratio

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4260 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: 17c81c57c829058685e745a9df14e6033d2e7386
      https://github.com/moses-smt/mosesdecoder/commit/17c81c57c829058685e745a9df14e6033d2e7386
  Author: hieuhoang1972 <hieuhoang1972@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-29 (Thu, 29 Sep 2011)

  Changed paths:
    M moses-chart-cmd/src/Main.cpp

  Log Message:
  -----------
  recomb stats

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4281 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: 3f3729c8e54c2d574812ece2387d473b13a58e61
      https://github.com/moses-smt/mosesdecoder/commit/3f3729c8e54c2d574812ece2387d473b13a58e61
  Author: hieuhoang1972 <hieuhoang1972@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-09-30 (Fri, 30 Sep 2011)

  Changed paths:
    M kenlm/lm/left.cc

  Log Message:
  -----------
  total recomb

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4283 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: 64bfa6bc803f62a5c9f889716e61f8d689e5bd07
      https://github.com/moses-smt/mosesdecoder/commit/64bfa6bc803f62a5c9f889716e61f8d689e5bd07
  Author: heafield <heafield@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-10-01 (Sat, 01 Oct 2011)

  Changed paths:
    M kenlm/lm/left.hh

  Log Message:
  -----------
  Merge cdec changes, bugfixes


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4288 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: bf9d8dd0df6de43712e0cd075ad5bd5012b9f6a7
      https://github.com/moses-smt/mosesdecoder/commit/bf9d8dd0df6de43712e0cd075ad5bd5012b9f6a7
  Author: heafield <heafield@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-10-02 (Sun, 02 Oct 2011)

  Changed paths:
    M kenlm/lm/left.cc
    M kenlm/lm/left.hh
    M moses/src/LanguageModelKen.cpp

  Log Message:
  -----------
  Fix full left/full right definition to be zero length


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4289 1f5c12ca-751b-0410-a591-d2e778427230


  Commit: ff1b1cf6382e52fc67d48aa587de14461125ebaf
      https://github.com/moses-smt/mosesdecoder/commit/ff1b1cf6382e52fc67d48aa587de14461125ebaf
  Author: heafield <heafield@1f5c12ca-751b-0410-a591-d2e778427230>
  Date:   2011-10-04 (Tue, 04 Oct 2011)

  Changed paths:
    M kenlm/lm/left.cc
    M kenlm/lm/left.hh
    M moses-chart-cmd/src/Main.cpp

  Log Message:
  -----------
  Lots more stats collection


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mtm_lm@4294 1f5c12ca-751b-0410-a591-d2e778427230


Compare: https://github.com/moses-smt/mosesdecoder/compare/5ed73450dd4b^...ff1b1cf6382e