Hermes Natural Language Processing Files

A repository of software, documentation and data for NLP

Brought to you by: tresoldi

The interactive file manager requires Javascript. Please enable it or use sftp or scp.
You may still browse the files here.

Name	Modified	Size	InfoDownloads / Week
Parent folder
SingleLineTokens.py	2012-07-07	273 Bytes	0
Lowercase.py	2012-07-07	156 Bytes	0
README	2012-07-07	557 Bytes	0
WikiExtractor.py	2012-07-05	21.3 kB	0
TrainPunkt.py	2012-07-05	1.6 kB	0
RawFilter.py	2012-07-05	1.3 kB	0
Punkt.py	2012-07-05	1.1 kB	0
Totals: 7 Items		26.3 kB	0

wget -c http://dumps.wikimedia.org/glwiki/20120625/glwiki-20120625-pages-articles.xml.bz2
mkdir glwiki
bzcat glwiki-20120625-pages-articles.xml.bz2 | ./WikiExtractor.py -o glwiki
find glwiki -exec cat {} \; > corpus/gl.raw
rm -rf glwiki
cat corpus/gl.raw | ./RawFilter.py > corpus/gl.pars
cat corpus/gl.pars | ./TrainPunkt.py > corpus/gl.punkt
cat corpus/gl.pars | ./Punkt.py corpus/gl.punkt > corpus/gl.sent
cd corpus
bzip2 gl.*
cat corpus/gl.tokens | ./SingleLineTokens.py | ./Lowercase.py | /usr/local/irstlm/add-start-end.sh > corpus/gl-20120625.tokens

Source: README, updated 2012-07-07

Other Useful Business Software

MongoDB Atlas runs apps anywhere Icon

MongoDB Atlas runs apps anywhere

Deploy in 115+ regions with the modern database for every enterprise.

MongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.

Start Free

Build Securely on Azure with Proven Frameworks Icon

Build Securely on Azure with Proven Frameworks

Lay a foundation for success with Tested Reference Architectures developed by Fortinet’s experts. Learn more in this white paper.

Moving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.

Download Now

Recommended Projects

TEES
Turku Event Extraction System (TEES) is a free and open source natural language processing system developed for the extraction of events and relations from biomedical text. It is written mostly in Python, and should work in generic Unix/Linux environments. Currently, the TEES source code...
BioC
A minimalist approach to share text documents and data annotations. Allows a large number of different annotations to be represented. Project files contain: - simple code to hold/read/write data and perform sample processing. - BioC-formatted corpora - BioC tools that work with BioC...
Virastyar
Virastyar is a free and open-source (FOSS) spell checker. It stands upon the shoulders of many free/libre/open-source (FLOSS) libraries developed for processing low-resource languages, especially Persian and RTL languages Publications: Kashefi, O., Nasri, M., & Kanani, K. (2010). Towards...
Arabic Corpus
The Arabic Corpus {compiled by Dr. Mourad Abbas ( http://sites.google.com/site/mouradabbas9/corpora ) The corpus Khaleej-2004 contains 5690 documents. It is divided to 4 topics (categories). The corpus Watan-2004 contains 20291 documents organized in 6 topics (categories). Researchers who use...
NLP software, uses for MT
A unique natural-language processing software, called Discovery, created on the CA Visual Objects/Vulcan.NET environment, which also has potential for effective "shallow approach" machine translation.