This project, currently in its early stages, aims to implement N-gram counting, smoothing, and interpolation. The project may eventually incorporate a number of smoothing methods, but initially it will incorporate just one, which is not a standard method. It's similar in spirit but modified, interpolated Kneser-Ney, but uses the principle that the amount you discount should always be the amount that gets given to the backoff state (which makes the traditional count-of-counts stuff and the associated formulas for obtaining smoothing constants unusuable).
This gives slightly better results than KN for standard LM building, but has larger advantages for interpolation and pruning. See <a href=freelm.sf.net> here </a> for documentation.

Project Activity

See All Activity >

Categories

Linguistics

License

Apache License V2.0

Follow FreeLM

FreeLM Web Site

Other Useful Business Software
Build Securely on AWS with Proven Frameworks Icon
Build Securely on AWS with Proven Frameworks

Lay a foundation for success with Tested Reference Architectures developed by Fortinet’s experts. Learn more in this white paper.

Moving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
Download Now
Rate This Project
Login To Rate This Project

User Reviews

Be the first to post a review of FreeLM!

Additional Project Details

Operating Systems

Linux

Intended Audience

Science/Research

User Interface

Command-line

Programming Language

C++

Related Categories

C++ Linguistics Software

Registered

2012-02-25