Search Results for "corpora"
Sort By:
Showing 33 open source projects for "corpora"
View related business solutions-
Streamline Azure Security with Palo Alto Networks VM-SeriesImprove your security posture and reduce incident response time. Use the VM-Series to natively analyze Azure traffic and dynamically drive policy updates based on workload changes.
-
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
1
IMS Open Corpus Workbench
Indexing and query tools for very large text corpora
The IMS Open Corpus Workbench is a collection of tools for managing and querying large text corpora (100 M words and more) with linguistic annotations. Its central component is the flexible and efficient query processor CQP, which can be used interactively in a terminal session, as a backend e.g. from a Perl script, or through the Web-based GUI CQPweb. -
2
Tokenized Text Aligner
Aligns tokens in two versions of a text with differing tokenization.
This tool performs token-by-token alignment of two versions of a text with differing tokenization by interpreting the results of a file diff (https://docs.python.org/3/library/difflib.html). It is intended for use in the preparation of annotated linguistic corpora, where differences in tokenization may arise (i) following corrections or modifications to the source text or (ii) through the creation of different layers of annotation (part-of-speech, treebank) requiring different tokenization. In its default implementation, it produces a human-readable CSV table associating tokens in text A with tokens in text B, and can also inject token-level annotation from text B to text A. ... -
3
TXM
Unicode XML TEI text analysis platform
TXM is a free and open-source cross-platform Unicode & XML based text analysis environment and graphical client, supporting Windows, Linux and Mac OS X. It can also be used online as a J2EE standard compliant web portal (GWT based) with access control built in. DOWNLOAD LATEST VERSION OF TXM : http://textometrie.ens-lyon.fr/spip.php?rubrique61&lang=en TXM offers a comprehensive range of analysis tools (concordances, collocate search, frequency lists, etc.) based on the powerfull CQP... -
4
JoBimText
Linking Language to Knowledge with Distributional Semantics
JobimText is a software solution for automatic text expansion using contextualized distributional similarity. It provides text analysis tools for large corpora and has capabilities to create distributional semantic models (JoBimText models) and multi-word expressions. -
Forever Free Full-Stack Observability | Grafana CloudBuilt on open standards like Prometheus and OpenTelemetry, Grafana Cloud includes Kubernetes Monitoring, Application Observability, Incident Response, plus the AI-powered Grafana Assistant. Get started with our generous free tier today.
-
5
Linguistic Analyzer
The Linguistic Analyzer is a tool for corpus analysis and comparison
The Linguistic Analyzer (Almuhalil Alloghawy) is a free tool designed by a team from Al-Imam Muhammad bin Saud islamic university that can be used for corpus analysis and comparison in terms of the several linguistic characteristics, such as frequency lists generation, concordances, collocation extraction, the difference between two words, and keyword identification. -
6
UnsupervisedMT
Phrase-Based & Neural Unsupervised Machine Translation
Unsupervised Machine Translation is a research repository that implements both phrase-based SMT and neural MT approaches for translation without parallel corpora. The neural component supports multiple architectures—seq2seq, biLSTM with attention, and Transformer—and allows extensive parameter sharing across languages to improve data efficiency. Training relies on denoising auto-encoding and back-translation, with on-the-fly, multithreaded generation of synthetic parallel data to continually refresh supervision signals. ... -
7
Arabic Corpus
Text categorization, arabic language processing, language modeling
...More useful references to check: ------------------------------------------- https://sites.google.com/site/mouradabbas9/corpora -
8
concordia
Powerful search library, best suited for computer-aided translation
...This project now contains fully functional Concordia search library. In the near future, it will be extended by concordia-server: ligthweight, robust web server providing corpora search functionalities -
9
Ghawwas_V4
An open source system for Arabic corpora processing
Ghawwas (previously known as Khawas) is an open source system for Arabic corpora processing. Ghawwas V4.0 provides the following main functions: a. Frequency list for single word and N-Grams b. Concordance c. Collocation (MI, CHI Squared, LL, T-Score, Z Score, Dice, Log Dice, Weirdness Coefficient) d. Lexical patterns search e. Two corpora frequency profile comparison based on MI, CHI, LL, T-Score, Z Score, Dice, Log Dice, Weirdness Coefficient f. -
Secure File Transfer for Windows with Cerberus by RedwoodCerberus supports unlimited users and connections on a single IP, with built-in encryption, 2FA, and a browser-based web client — all deployable in under 15 minutes with a 25-day free trial.
-
10
rcqp
R interface to the Corpus Query Protocol
Implements the Corpus Query Protocol as a package for the R statistical environment. It allows to query linguistic corpora and manipulate the data as native R objects. It is based on the CWB software. -
11
poliqarp2
natural language corpora search engine
This project aims at building an efficient indexer and search engine for natural language corpora with multilevel annotations. -
12
BioC
We describe a simple XML format to share text documents and annotation
...Allows a large number of different annotations to be represented. Project files contain: - simple code to hold/read/write data and perform sample processing. - BioC-formatted corpora - BioC tools that work with BioC corpora BioC goals - simplicity - interoperability - broad use - reuse There should be little investment required to learn to use a format or a software module to process that format. We are interested in reuse, and we focus on common NLP tasks that are broadly useful for textmining. -
13
diasim
Dialogue Similarity
Tools for calculating similarity (including lexical and syntactic) between speakers in dialogue, across standard and randomised corpora. -
14
eAlign
A parallel corpora (bitext) aligning tool. Create TMX databases
(Full support available under superalign.sourceforge.net) Aligning parallel corpora Creating TMX, csv, Tab Delimited TMs Automatic aligning of text Super fast handling of multiple files Very easy GUI handling of files under Windows CAT tool assistant -
15
SuperAlign was fully updated as of 15 July 2013 and is now released under the name eAlign as well. A parallel corpora (bitext) aligning tool. Create TMX databases and align translations for Translation Memory databases. Use multiple files in multiple formats to align them with their translations. The full workflow is built in with a GUI interface. SuperAlign-eAlign uses the hunalign algorithm.
-
16
Cross-Language Computational Linguistics
cross-languages resources
...It is composed of 40K aligned articles, 91.3M English words, 57.8M French words, 22M Arabic words, 2.8M English unique words, 1.9M French unique words, and 1.5M Arabic unique words. Wikipedia text is available under Creative Commons Attribution-ShareAlike 3.0 License. https://en.wikipedia.org/wiki/Wikipedia:About To cite the corpora: M. Saad, D. Langlois, and K. Smaïli. Extracting Comparable Articles from Wikipedia and Measuring their Comparabilities. Procedia - Social and Behavioral Sciences, 95(0):40 – 47, 2013. ISSN 1877-0428. -
17
mwetoolkit
THIS PROJECT MIGRATED TO https://gitlab.com/mwetoolkit/mwetoolkit3/
THIS PROJECT MIGRATED TO https://gitlab.com/mwetoolkit/mwetoolkit3/ The Multiword Expressions toolkit aids in the automatic identification and extraction of multiword units in running text. These include idioms (kick the bucket), noun compounds (cable car), phrasal verbs (take off, give up), etc. Even though it focuses on multiword expresisons, the framework is quite complete and can also be useful in any corpus-based study in computational linguistics. The mwetoolkit can be... -
18
TextTools is a freeware corpus linguistics tool developed in Python to aid in research. This program analyzes user-created corpora and displays information about word (token) frequency, n-grams, clusters, collocations, keyword in context (KWIC), and keyness. TextTools is designed to be user-friendly and intuitive and will run natively on Mac OS X.
-
19
Aelius Brazilian Portuguese POS-Tagger
Python, NLTK-based package for shallow parsing of Brazilian Portuguese
...It also includes language resources such as language models, sample texts, and gold standards. Presently, Aelius already offers facilities for POS-tagging and chunking corpora and outputting annotations in different formats, such as in XML in the TEI P5 encoding scheme. -
20
DisMo
A POS, disfluency and multi-word unit annotator for spoken language
...It is developed and maintained by George Christodoulides (Centre Valibel, IL&C, University of Louvain, Louvain-la-Neuve, Belgium). Visit www.corpusannotation.org to find out more about DisMo and other annotation tools for language corpora. If you are using DisMo to annotate your corpus, please cite the following paper: Christodoulides, George; Avanzi, Mathieu; Goldman, Jean-Philippe. DisMo: A Morphosyntactic, Disfluency and Multi-Word Unit Annotator. An Evaluation on a Corpus of French Spontaneous and Read Speech. Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC) 2014, Reykjavik, Iceland, 26-31 May 2014, pp. 3902-3907. -
21
Khawas
An Arabic Corpora Processing Tool
The new version is available at https://sourceforge.net/projects/ghawwasv4/ -
22
Platform for Annotated Corpora in XML Integrated tool for corpus linguists built on Eclipse, Vex, Subversive, etc. for creating and editing transcriptions and annotations, querying, managing version controlled data, and building a shippable corpus.
-
23
Donatus is an on-going project consisting of Python, NLTK-based tools and grammars for deep parsing and syntactical annotation of Brazilian Portuguese corpora. It includes a user-friendly graphical user interface for building syntactic parsers with the NLTK, providing some additional functionalities.
-
24
Hermes Natural Language Processing
A repository of software, documentation and data for NLP
Hermes is a repository of software, documentation and data for NLP. I am currently adding corpora extracted from Wikipedia (mostrly in Romance languages). -
25
Various tools for creating annotated parallel corpora including pre-trained tagging and parsing models for various languages, sentence alignment tools and word alignment tools. Uplug also includes a web-based interface for interactive sentence and word alignment and scripts for indexing and querying parallel corpora using the Corpus Work Bench CWB. Download 'uplug-main' first and then add other packages.
