Download
CSAw is an NLP framework for low-resource languages with a focus on machine translation. The primary goal is to build language models automatically from bilingual text (e.g., front and back translations) using a deep transfer rule-based approach. The core of this strategy is the Concept Specification and Abstraction semantic representation which is specially designed with machine translation in mind. See the preprint article here:
https://arxiv.org/abs/1807.02226
The current framework includes transduction algorithms (i.e., from text to semantic representation and back again) and some components needed for automatic language model building (lexical alignment and grammar rule generation). Recently, we have added transliteration functionality. The project is currently incomplete. More to come.
Features
- Transduction algorithms (text-to-concept-network and back again)
- USFM-based Sentence Alignment
- Lexical Alignment (enhanced for low occurrence words)
- Automatic Rule Generation (syntactic and morphological)
- Tokenization/Lemmatization
- POS and Ontological Tagging
- Transliteration
