Helsinki Finite-State Technology / News: Recent posts

** HFST has been migrated to Github: https://hfst.github.io/. **

• License for the library code and python bindings (under libhfst and python directories) is now LGPL 3

• Make python bindings compatible also with python version 2

• Add time cutoff option to hfst-pmatch and hfst-proc2

• Rename hfst-proc2 to hfst-tokenize, with hfst-proc2 and hfst-tokenise as aliases

• Use standard c++0x when compiling unless a higher standard is required with -std

• Check automake version during configuration to determine whether header files generated by yacc/bison should use extension .h (automake < 1.12) or .hh (automake >= 1.12)... read more

• Implement a new Python API

• Add option --time-cutoff to hfst-lookup

• Remove option --minimizeFlags from hfst-lexc and minimize flags by default, add option --alignStrings

• Add support for controlling input and output streams in xre, xfst and lexc compilers

• Update foma back-end

• Improve handling of negative weights, check for epsilon cycles with negative weights and issue a warning, if needed

• Improvements and bug fixes to hfst-optimized-lookup, hfst-proc2, hfst-proc and hfst-pmatch... read more

• Bugfixes and improvements to many tools, including hfst-xfst, hfst-proc(2),
  hfst-lexc, hfst-pmatch(2fst), hfst-reweight, hfst-pair-test, hfst-compose-intersect
• Better support for command line tools on windows

• Runtime speed improvements in pmatch
• Speedup in conversion to optimized lookup format
• Merge and compile-replace operations supported in hfst-xfst
• Added option --print-symbol-pair-statistis(=N) to hfst-summarize

• (the release of version 3.8.0 was delayed, its changes are also listed here)

• changes to flag diacritic handling

• now hfst-lexc has options --withFlags, --minimizeFlags and --renameFlags to control
  hyperminimization

• hfst-xfst has corresponding variables --lexc-with-flags, --lexc-minimize-flags and
  --lexc-rename-flags

• hfst-xfst has variable xerox-composition (default ON) and hfst-lexc option
  --xerox-composition (default ON) to control if flags are matched with unknowns and
  identities in composition (as Xerox does)... read more

• improvements to lexc, pmatch and twolc tools

• same mappings with different weights are now handled right in
  parallel replace rules.

• improvements to optimized lookup format

• better support for weighted parallel replace rules
• allowing empty contexts in replace rules
• bugfixes and improvements to xfst and pmatch tools
• bugfixes to handling of optimized lookup transducers
• support for weighted containment operator
• tentatively added option and variable 'encode-weights' to regexp
  and xfst parsers to control determinization of weighted transducers,
  the default value being false (encoding weights prevents non-ending
  determinization in some cases)

• improvements to xfst, lexc, pmatch and lookup tools

• hfst-lexc uses native hfst functions
• improvements to pmatch tool
• limited support for optimized lookup transducers in hfst-xfst
• better support for Mac and Windows
• replace rule longest match bug fixed

• improvements and bugfixes to lexc, pmatch and xfst tools
• added an option --enable-all-tools in configure

• a new tool hfst-xfst for compiling xfst scripts and using xfst commands
  in interactive mode (NOTE: hfst-xfst was earlier a wrapper for foma, so this
  change breaks backwards compatibility)
• wrapper for foma command line tool is renamed to hfst-foma
• changes to the api
  • made HfstTransducer::harmonize() a public member
  • added a boolean argument to function 'set_final_weights' which defines
    whether existing weights are summed with the new weight or overwritten
  • updated cross product and apply boundary mark in replace rules
• improvements to pmatch tool
  • --verbose output gives neater and more fine-grained information
  • some fixes to ?-expansion
  • more checking and better error diagnostics in case of broken tags
  • time individual compilations
  • fixes to leftmost greediness and overlong input
• changes to other tools
  • added support for weights in prolog format, also added an option
    --prolog to hfst-fst2txt
  • added option --eliminate-flags to hfst-compare
  • added option --harmonize-flags to hfst-regex2fst
  • lexc parser handles epsilon '0' correctly
  • better support for quoted literals and marker '.#.' in xre parser
  • added warning messages about hfst-special symbols of form
    '@.*@' to xre parser that are printed in verbose mode

• Improvements to pmatch functionalities
• Some fixes to twolc, proc and lexc tools
• New operators added to the regexp parser
• Reading in prolog format supported in hfst-txt2fst

• Several improvements to pmatch tools
• Function calls supported in xre parser
• A bug in replace rules with term complement in context fixed
• Some fixes to the lexc parser
• Now option combination --obey-flags --random works in hfst-fst2strings

This is a bug fix release.

• fixed a bug that caused certain types of paths with flag diacritics not to be accepted.

No changes to interface or features.

Noteworthy changes:

• foma and sfst backends are enabled by default in configure
• Updated to foma version 0.9.17
• New functions added to HfstTransitionGraph, e.g. longest_string_size
  and top_sort
• pmatch tools updated
• Bugs fixed in xre parser and hfst-twolc
• A boolean argument added to binary transducer functions that
  controls whether symbol harmonization is carried out

Noteworthy changes:

• Better support for compilation on Cygwin.
• Tool hfst-prune-alphabet added.
• A function 'hfst_get_exception()' added to the swig interface that
  can be used to handle exceptions on Python 3.
• Bugfixes in hfst-info, hfst-twolc, hfst-regexp2fst and tagger tools.
• Flag diacritic harmonization problem fixed, now identity and
  unknown symbols are not expanded by flag diacritics.

Noteworthy changes:

• Added regexp substitution and left replace epenthesis to regexp parser.

• A bug in including zlib in foma fixed.

• Now ncurses library is used instead if termcap is not found when
  building hfst-xfst.

Noteworthy changes:

• Updates to regexp parser and optimized lookup transducer

• SWIG bindings now work with both Python version 2 and 3. As FILE* conversions
  are no longer supported in Python 3, more wrapper functions are added that use
  an HfstFile argument instead.

• Some of the warnings in back-end libraries fixed or suppressed

• A bug in hfst-xfst linking fixed

• Obsolete command line tools removed

Noteworthy changes:

• Experimental version of the SWIG-generated Python interface is
  included in the distribution. See swig/README for more info.

• All back-ends (SFST, OpenFst and foma) are bundled with HFST.
  There is no more need to install them separately.

Noteworthy changes:

• Small bugs fixed in function insert_freely, flag diacritic handling
  as well as lexc and pmatch tools
• GNU and bash dependencies removed from scripts to make them more portable
• The epsilon symbol argument in HfstTransducer(FILE*, ...) is made obligatory,
  also a new obligatory argument added that keeps track of the line number
  in the AT&T file

Noteworthy changes:

• Bug fixes in tools pmatch2fst, strings2fst, substitute and guessify
• Bug fixes in replace rules
• More functionalities added to the Python interface

Noteworthy changes:

• Better support for Windows compilation
• Some bugs in replace rules and hfst-regexp2fst fixed
• Bug fixes and updates in command line tools including hfst-tagger, hfst-info, optimized-lookup, hfst-regexp2fst and hfst-guessify
• PCKIMMO output mode for hfst-fst2txt
• Now option --norm precedes --log in hfst-strings2fst

Noteworthy changes:

• Support for default symbol @DEFAULT_SYMBOL@
• A tagger tool hfst-tagger
• Improvements to hfst-lexc
• Removed warning messages and XFAIL tests that may confuse users
• Updated Doxygen documentation.