Open Source Python Linguistics Software - Page 2

Redundancy due to cut-paste operations in text creates bias in machine learning for NLP. This module takes a directory and produces a subset of the files in that directory (in a list) with an upper bound on similarity between two files.

CycleGAN and pix2pix in PyTorch repository is a PyTorch implementation of two influential image-to-image translation frameworks: CycleGAN (for unpaired translation) and pix2pix (for paired translation). This repo gives developers and researchers a convenient, modern (PyTorch-based) platform to train and test these methods — supporting both paired datasets (input to output) and unpaired datasets (domain-to-domain) with minimal changes. The code supports standard training and inference pipelines, and as of recent updates, compatibility with the latest Python and PyTorch versions (e.g. Python 3.11, PyTorch 2.4) as well as support for distributed/multi-GPU training for scalable workflows. Because of its flexibility, users can apply it to many tasks: e.g. style transfer between domains (e.g. season changes, art-to-photo, etc.), mapping sketches/edges to real images, image colorization, day-to-night, photo enhancement, and more.

Donatus is an on-going project consisting of Python, NLTK-based tools and grammars for deep parsing and syntactical annotation of Brazilian Portuguese corpora. It includes a user-friendly graphical user interface for building syntactic parsers with the NLTK, providing some additional functionalities.

The Helsinki Finite-State Transducer toolkit is intended for processing natural language morphologies. The toolkit is demonstrated by wide-coverage implementations of a number of languages of varying morphological complexity.

An agent-based situated language learning simulation that focuses on lexical learning and grounding, featuring a unigram syntax structure and a CFG-based semantic grammar. Created as a MSc thesis project, using python.

KAF2Tiger2 is a KAF (KYOTO annotation format) to <tiger2/> (Tiger2 XML) converter.

Kana no quiz is a little educational tool to memorize the transcription and pronunciation of Japanese kana (katakana & hiragana), presented as a quiz. It is written in Python and uses a GTK+ interface for a nice cross-paltform rendering!

With this tool you can easily design a new language, digitize an existing one or incrementally reconstruct an ancient language. It allows for free experimentation of all aspects of the language, so it does not have to be made consistent on paper first. You can edit script, syntax, grammar, morphology, lexicon and phonology, as well as write documents in the language, as it might be too complex to be handled by current font technology. The information is stored in xml format for easy integration with other software.

Little Cohesion Helper (LCH), alias TraglWeck, semi-automates the annotation of lexical-cohesion in a given text. Input is a raw text file and this software generates a bunch of XML files which can be used with MMAX2.

The MITRE Annotation Toolkit (MAT) is a suite of tools which can be used for automated and human tagging of annotations. Annotation is a process, used mostly by researchers in natural language processing, of enhancing documents with information about the various phrase types the documents contain. MAT supports both UI interaction and command-line interaction, and provides various levels of control over the overall annotation process. It can be customized for specific tasks (e.g., named entity identification, de-identification of medical records). The goal of MAT is not to help you configure your training engine (in the default case, the Carafe CRF system) to achieve the best possible performance on your data. MAT is for "everything else": all the tools you end up wishing you had.

A public repository of open source scripts and small programs related to linguistics and language.

The Visual Annotator from Health Information Technologies Research Laboratory at The University of Sydney. http://hitrl.it.usyd.edu.au/

Pootle is written in Python using the Django framework and therefore can be installed on any web server that supports serving WSGI applications. Key localization file formats are supported, including Gettext PO, XLIFF, Java .properties, PHP arrays and many more supported by the Translate Toolkit. Ensures the best quality localizations by automatically detecting common errors made by translators. Integrates user suggestions, terminology, Translation Memory, Machine Translation, and more. A number of translation projects for a number of languages can be hosted on Pootle. Teams can manage their files, permissions, projects, and translate on-line. Files can be downloaded for offline translation. Delegate to your translators and allow them to commit directly to your version control systems. Pootle can easily be adapted and integrated into your existing web look and feel.

PyAnnotation is a Python Library to access and manipulate linguistically annotated corpus files. Supported file formats are Kura XML, Elan XML and Toolbox files. A Corpus Reader API is provided to support statistical analysis within the NLTK.

Pylero is an open-source Python-based text generator.

RDRPOSTagger is a robust, easy-to-use and language-independent rule-based toolkit for Part-of-Speech (POS) and morphological tagging. RDRPOSTagger obtains fast performance in both learning and tagging process. RDRPOSTagger also achieves a very competitive accuracy in comparison to the state-of-the-art results. RDRPOSTagger now supports pre-trained POS and morphological tagging models for Bulgarian, Czech, Dutch, English, French, German, Hindi, Italian, Portuguese, Spanish, Swedish, Thai and Vietnamese. Additionally, RDRPOSTagger supports the pre-trained Universal POS tagging models for 40 languages. See the full usage of RDRPOSTagger at: http://rdrpostagger.sourceforge.net/

This project concerns the development of human language technology resources, based on the approach to share or recycle resources between closely related language. http://gerhard.pro/closely-related-languages/

The Rudify tools are a collection of tools for ontology tagging.

Russian morphology tagger. Parses text(s) and output xml representation of text(s) with grammatical annotation.

This is a fast C implementation of Arturo Camacho's SWIPE' pitch extraction algorithm. See the project homepage for more about the advantages of the SWIPE' algorithm. swipe-1.0.tar.gz contains the current source, which should compile quite neatly.

Phalanx - Deidentify Safe Harbor Deidentification Mode of Phalanx is an abridged pipeline of NLP annotators culminating in NER annotators which write output of text offsets. It uses the Safe Harbor deidentification method.

Shift Word Helper is a scrabble-like games helper, which includes an online dictionary client. Shift Word Helper est un bon assistant pour les jeux de mots tels que le scrabble, avec un dictionnaire en-ligne integré. English & french supported

A free Spanish - English Translator for Linux. It will translate a phrase (via internet) or single word (built-in dictionary.) Has capability to learn new words and is smart enough to find plural and feminine words. Written using Python/GTK under GPL

SpeakingInNumbers converts numbers into pronounceable pseudo words.

Software for speech research. It includes programs and libraries for signal processing, along with general purpose scientific libraries. Most of the code is in Python, with C/C++ supporting code. Also, contains code releases corresponding to publishe