[moses-smt/mosesdecoder] 2a3c9f: Further optimization for extractor.

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

  Branch: refs/heads/weight-new
  Home:   https://github.com/moses-smt/mosesdecoder
  Commit: 2a3c9fc6797c97595f5f6bda7dab0b17ce758afc
      https://github.com/moses-smt/mosesdecoder/commit/2a3c9fc6797c97595f5f6bda7dab0b17ce758afc
  Author: Tetsuo Kiso <tet...@is...>
  Date:   2012-12-06 (Thu, 06 Dec 2012)

  Changed paths:
    M mert/BleuScorer.cpp
    M mert/Ngram.h

  Log Message:
  -----------
  Further optimization for extractor.

Fixes inefficient updating N-gram counts.

NOTE: Using '--binary' option (this option is not enabled by default yet)
for saving outputs would lead to significant speed up.

  Commit: 466b502ae01e8d1d91011197c1179686ef83b27a
      https://github.com/moses-smt/mosesdecoder/commit/466b502ae01e8d1d91011197c1179686ef83b27a
  Author: phikoehn <pk...@in...>
  Date:   2012-12-09 (Sun, 09 Dec 2012)

  Changed paths:
    M scripts/ems/example/config.basic
    M scripts/ems/example/config.factored
    M scripts/ems/example/config.hierarchical
    M scripts/ems/example/config.syntax
    M scripts/ems/example/config.toy
    M scripts/ems/experiment.machines
    M scripts/ems/experiment.meta
    M scripts/ems/support/build-sparse-lexical-features.perl
    M scripts/ems/support/mml-train.perl

  Log Message:
  -----------
  minor bug fixes with MML

  Commit: ccf9e13d8e8531f4055a6833ef939b253f5e2ea8
      https://github.com/moses-smt/mosesdecoder/commit/ccf9e13d8e8531f4055a6833ef939b253f5e2ea8
  Author: phikoehn <pk...@in...>
  Date:   2012-12-09 (Sun, 09 Dec 2012)

  Changed paths:
    M scripts/ems/experiment.perl

  Log Message:
  -----------
  bug fix with multicore parallelizer

  Commit: ed2d1918217233aec230734aed0bcb6177641f50
      https://github.com/moses-smt/mosesdecoder/commit/ed2d1918217233aec230734aed0bcb6177641f50
  Author: phikoehn <pk...@in...>
  Date:   2012-12-09 (Sun, 09 Dec 2012)

  Changed paths:
    M scripts/ems/experiment.perl

  Log Message:
  -----------
  allow specification of end point for experiment.perl

  Commit: 598d65bcfd0ce84d153335c60ed75f5213a5ed54
      https://github.com/moses-smt/mosesdecoder/commit/598d65bcfd0ce84d153335c60ed75f5213a5ed54
  Author: Ales Tamchyna <a.t...@gm...>
  Date:   2012-12-10 (Mon, 10 Dec 2012)

  Changed paths:
    M mert/Jamfile
    A mert/sentence-bleu.cpp

  Log Message:
  -----------
  adding a simple command-line utility for computing sentence-level BLEU (+1)

  Commit: 16ea68f55f6ffc80ceb433a1db8ce9d9aa47f8fb
      https://github.com/moses-smt/mosesdecoder/commit/16ea68f55f6ffc80ceb433a1db8ce9d9aa47f8fb
  Author: Barry Haddow <bar...@gm...>
  Date:   2012-12-10 (Mon, 10 Dec 2012)

  Changed paths:
    M scripts/ems/support/mml-score.perl

  Log Message:
  -----------
  Fix bug in mml scoring

Line length calculation was out of step with LM scoring.

  Commit: 438dcb1a34e5707a418ad1de55916eb9dcc5a17d
      https://github.com/moses-smt/mosesdecoder/commit/438dcb1a34e5707a418ad1de55916eb9dcc5a17d
  Author: phikoehn <pk...@in...>
  Date:   2012-12-10 (Mon, 10 Dec 2012)

  Changed paths:
    M scripts/ems/experiment.perl

  Log Message:
  -----------
  bug fix in experiment.perl wrt. get-corpus-script

  Commit: 24e1df7520dfe5dae8f74d28204914a9df2d451f
      https://github.com/moses-smt/mosesdecoder/commit/24e1df7520dfe5dae8f74d28204914a9df2d451f
  Author: phikoehn <pk...@in...>
  Date:   2012-12-11 (Tue, 11 Dec 2012)

  Changed paths:
    M scripts/ems/example/config.basic
    M scripts/ems/example/config.factored
    M scripts/ems/example/config.hierarchical
    M scripts/ems/example/config.syntax
    M scripts/ems/example/config.toy
    M scripts/ems/experiment.meta
    M scripts/ems/experiment.perl
    M scripts/training/train-model.perl

  Log Message:
  -----------
  support for use of baseline alignment model

  Commit: b275c94dbf690d492a763f03e70939bf86d24b84
      https://github.com/moses-smt/mosesdecoder/commit/b275c94dbf690d492a763f03e70939bf86d24b84
  Author: phikoehn <pk...@in...>
  Date:   2012-12-11 (Tue, 11 Dec 2012)

  Changed paths:
    M scripts/ems/example/config.basic
    M scripts/ems/example/config.factored
    M scripts/ems/example/config.hierarchical
    M scripts/ems/example/config.syntax
    M scripts/ems/example/config.toy
    M scripts/ems/experiment.meta
    M scripts/ems/experiment.perl
    M scripts/generic/extract-parallel.perl
    M scripts/training/LexicalTranslationModel.pm
    M scripts/training/train-model.perl

  Log Message:
  -----------
  allow for inclusion of extract from previous run

  Commit: 667b8a495f4e9a2bd677340919a6545d7e8a8c55
      https://github.com/moses-smt/mosesdecoder/commit/667b8a495f4e9a2bd677340919a6545d7e8a8c55
  Author: Hieu Hoang <fis...@gm...>
  Date:   2012-12-17 (Mon, 17 Dec 2012)

  Changed paths:
    M contrib/other-builds/moses-chart-cmd/.cproject
    M contrib/other-builds/moses-cmd/.cproject
    M moses-chart-cmd/Main.cpp
    M moses-cmd/Main.cpp
    M scripts/ems/support/reuse-weights.perl
    M scripts/training/mert-moses.pl

  Log Message:
  -----------
  decoder -show-weight to output consistent new format

  Commit: 06081f7ddb050696c759441fdb081655d233c749
      https://github.com/moses-smt/mosesdecoder/commit/06081f7ddb050696c759441fdb081655d233c749
  Author: Phil Williams <phi...@ma...>
  Date:   2012-12-17 (Mon, 17 Dec 2012)

  Changed paths:
    M scripts/analysis/extract-target-trees.py

  Log Message:
  -----------
  extract-target-trees.py: minor fixes, code style

  Commit: fb8d20a22fd3dc61df250393ccc18131b35acfc7
      https://github.com/moses-smt/mosesdecoder/commit/fb8d20a22fd3dc61df250393ccc18131b35acfc7
  Author: Phil Williams <phi...@ma...>
  Date:   2012-12-17 (Mon, 17 Dec 2012)

  Changed paths:
    M phrase-extract/extract-ghkm/ExtractGHKM.cpp
    M phrase-extract/extract-ghkm/ExtractGHKM.h
    M phrase-extract/extract-ghkm/Options.h

  Log Message:
  -----------
  extract-ghkm: --UnknownWordMinRelFreq, --UnknownWordUniform

  Commit: 0ca5b8932a7d15ab725cac72ee077b98d4fa6c6f
      https://github.com/moses-smt/mosesdecoder/commit/0ca5b8932a7d15ab725cac72ee077b98d4fa6c6f
  Author: Phil Williams <phi...@ma...>
  Date:   2012-12-17 (Mon, 17 Dec 2012)

  Changed paths:
    M phrase-extract/extract-ghkm/ExtractGHKM.cpp
    M phrase-extract/extract-ghkm/ExtractGHKM.h

  Log Message:
  -----------
  extract-ghkm: tweak label collection for unknown words

Produce a better label set when unary rule elimination is enabled.

  Commit: 139148bc8fd5e77c95157bc82c459a0612e64890
      https://github.com/moses-smt/mosesdecoder/commit/139148bc8fd5e77c95157bc82c459a0612e64890
  Author: Phil Williams <phi...@ma...>
  Date:   2012-12-17 (Mon, 17 Dec 2012)

  Changed paths:
    M phrase-extract/SentenceAlignmentWithSyntax.cpp
    M phrase-extract/XmlTree.cpp
    M phrase-extract/XmlTree.h
    M phrase-extract/extract-ghkm/XmlTreeParser.cpp
    M phrase-extract/relax-parse-main.cpp

  Log Message:
  -----------
  extract-ghkm and friends: don't unescape special characters

Don't unescape special characters when reading XML parse trees in
extract-ghkm, extract-rules, and relax-parse.

  Commit: aa00aebee6c39db05c5a5067aa15d23c38105c0d
      https://github.com/moses-smt/mosesdecoder/commit/aa00aebee6c39db05c5a5067aa15d23c38105c0d
  Author: Hieu Hoang <fis...@gm...>
  Date:   2012-12-18 (Tue, 18 Dec 2012)

  Changed paths:
    M misc/Jamfile
    A misc/reuse-weights.cpp
    M scripts/ems/experiment.meta

  Log Message:
  -----------
  rewrite reuse-weight

  Commit: 7ed52368a8c337b13d07dccaa6be7840e82ffe7b
      https://github.com/moses-smt/mosesdecoder/commit/7ed52368a8c337b13d07dccaa6be7840e82ffe7b
  Author: Hieu Hoang <fis...@gm...>
  Date:   2012-12-18 (Tue, 18 Dec 2012)

  Changed paths:
    M misc/reuse-weights.cpp
    M scripts/training/mert-moses.pl

  Log Message:
  -----------
  more minor changes in scripts for new format

  Commit: bc615bdac86edafe372ffa6b1a1c8ad43fe16701
      https://github.com/moses-smt/mosesdecoder/commit/bc615bdac86edafe372ffa6b1a1c8ad43fe16701
  Author: Hieu Hoang <fis...@gm...>
  Date:   2012-12-18 (Tue, 18 Dec 2012)

  Changed paths:
    M mert/BleuScorer.cpp
    M mert/Jamfile
    M mert/Ngram.h
    A mert/sentence-bleu.cpp
    M phrase-extract/SentenceAlignmentWithSyntax.cpp
    M phrase-extract/XmlTree.cpp
    M phrase-extract/XmlTree.h
    M phrase-extract/extract-ghkm/ExtractGHKM.cpp
    M phrase-extract/extract-ghkm/ExtractGHKM.h
    M phrase-extract/extract-ghkm/Options.h
    M phrase-extract/extract-ghkm/XmlTreeParser.cpp
    M phrase-extract/relax-parse-main.cpp
    M scripts/analysis/extract-target-trees.py
    M scripts/ems/example/config.basic
    M scripts/ems/example/config.factored
    M scripts/ems/example/config.hierarchical
    M scripts/ems/example/config.syntax
    M scripts/ems/example/config.toy
    M scripts/ems/experiment.machines
    M scripts/ems/experiment.meta
    M scripts/ems/experiment.perl
    M scripts/ems/support/build-sparse-lexical-features.perl
    M scripts/ems/support/mml-score.perl
    M scripts/ems/support/mml-train.perl
    M scripts/generic/extract-parallel.perl
    M scripts/training/LexicalTranslationModel.pm
    M scripts/training/train-model.perl

  Log Message:
  -----------
  Merge branch 'master' into weight-new

Compare: https://github.com/moses-smt/mosesdecoder/compare/05045d574cff...bc615bdac86e