Search Results for "corpus"
Sort By:
Showing 27 open source projects for "corpus"
View related business solutions-
Boost application security and continuity with SKUDONET ADC, our Open Source Load Balancer, that maximizes IT infrastructure flexibility. Additionally, save up to $470 K per incident with AI and SKUDONET solutions, further enhancing your organization’s risk management and cost-efficiency strategies.
-
Deploy Red Hat Ansible Automation Platform on Microsoft Azure for a strategic automation solution that allows you to orchestrate, govern and operationalize your Azure environment.
-
1
Minimal text diffusion
A minimal implementation of diffusion models for text generation
A minimal implementation of diffusion models of text: learns a diffusion model of a given text corpus, allowing to generate text samples from the learned model. The main idea was to retain just enough code to allow training a simple diffusion model and generating samples, remove image-related terms, and make it easier to use. To train a model, run scripts/train.sh. By default, this will train a model on the simple corpus. However, you can change this to any text file using the --train_data... -
2
gensim
Topic Modelling for Humans
Gensim is a Python library for topic modeling, document indexing, and similarity retrieval with large corpora. The target audience is the natural language processing (NLP) and information retrieval (IR) community. -
3
Chronos Forecasting
Pretrained (Language) Models for Probabilistic Time Series Forecasting
Chronos is a family of pretrained time series forecasting models based on language model architectures. A time series is transformed into a sequence of tokens via scaling and quantization, and a language model is trained on these tokens using the cross-entropy loss. Once trained, probabilistic forecasts are obtained by sampling multiple future trajectories given the historical context. Chronos models have been trained on a large corpus of publicly available time series data, as well... -
4
-
Venn is a secure workspace for remote work that isolates and protects work from any personal use on the same computer. Work lives in a secure local enclave that is company controlled, where all data is encrypted and access is managed. Within the enclave – visually indicated by the Blue Border around these applications – business activity is walled off from anything that happens on the personal side. As a result, work and personal uses can now safely coexist on the same computer.
-
5
hebrew-gpt_neo
Hebrew text generation models based on EleutherAI's gpt-neo
Hebrew text generation models based on EleutherAI's gpt-neo. Each was trained on a TPUv3-8 which was made available to me via the TPU Research Cloud Program. The Open Super-large Crawled ALMAnaCH coRpus is a huge multilingual corpus obtained by language classification and filtering of the Common Crawl corpus using the goclassy architecture. -
6
AliceMind
ALIbaba's Collection of Encoder-decoders from MinD
... and sentence levels, respectively. Pre-trained models for natural language generation (NLG). We propose a novel scheme that jointly pre-trains an autoencoding and autoregressive language model on a large unlabeled corpus, specifically designed for generating new text conditioned on context. It achieves new SOTA results in several downstream tasks. -
7
GPT2 for Multiple Languages
GPT2 for Multiple Languages, including pretrained models
With just 2 clicks (not including Colab auth process), the 1.5B pretrained Chinese model demo is ready to go. The contents in this repository are for academic research purpose, and we do not provide any conclusive remarks. Research supported with Cloud TPUs from Google's TensorFlow Research Cloud (TFRC) Simplifed GPT2 train scripts（based on Grover, supporting TPUs). Ported bert tokenizer, multilingual corpus compatible. 1.5B GPT2 pretrained Chinese model (~15G corpus, 10w steps). Batteries... -
8
IRAMUTEQ : Interface de R pour les Analyses Multidimensionnelles de Textes et de Questionnaires. Logiciel de traitement de données pour des corpus texte ou de type individus/caractères. Permet notamment de réaliser des analyses de type "ALCESTE"
-
9
jieba
Stuttering Chinese word segmentation
"Jaba" Chinese word segmentation, do the best Python Chinese word segmentation component. Four word segmentation modes are supported. Precise mode, which tries to cut the sentence most precisely, suitable for text analysis. Full mode, scans all the words that can be formed into words in the sentence, the speed is very fast, but the ambiguity cannot be resolved. The search engine mode, on the basis of the precise mode, divides the long words again to improve the recall rate, which is suitable... -
New customers get $300 in free credits to spend on Speech-to-Text. All customers get 60 minutes for transcribing and analyzing audio free per month, not charged against your credits.
-
10
Dragonfire
The open-source virtual assistant for Ubuntu based Linux distributions
Dragonfire is the open-source virtual assistant project for Ubuntu-based Linux distributions. Her main objective is to serve as a command and control interface to the helmet user. So that you will be able to give orders just by using your voice commands and your eye movements. That makes the helmet handsfree. We are planning to ship Dragonfire as a preinstalled software package on DragonOS Linux Distribution. DragonOS will be a Linux distribution specially designed for the helmet. It will... -
11
PyTorch Natural Language Processing
Basic Utilities for PyTorch Natural Language Processing (NLP)
... this example code for training on the Stanford Natural Language Inference (SNLI) Corpus. Now you've setup your pipeline, you may want to ensure that some functions run deterministically. Wrap any code that's random, with fork_rng and you'll be good to go. Now that you've computed your vocabulary, you may want to make use of pre-trained word vectors to set your embeddings. -
12
CakeChat
CakeChat: Emotional Generative Dialog System
... bidirectional. By default, CuDNNGRU implementation is used for ~25% acceleration during inference. Thought vector is fed into decoder on each decoding step. Decoder can be conditioned on any categorical label, for example, emotion label or persona id. May be initialized using w2v model trained on your corpus. Embedding layer may be either fixed or fine-tuned along with other weights of the network. -
13
Arabic Corpus
Text categorization, arabic language processing, language modeling
The Arabic Corpus {compiled by Dr. Mourad Abbas ( http://sites.google.com/site/mouradabbas9/corpora ) The corpus Khaleej-2004 contains 5690 documents. It is divided to 4 topics (categories). The corpus Watan-2004 contains 20291 documents organized in 6 topics (categories). Researchers who use these two corpora would mention the two main references: (1) For Watan-2004 corpus ---------------------- M. Abbas, K. Smaili, D. Berkani, (2011) Evaluation of Topic Identification Methods... -
14
NeuroNER
Named-entity recognition using neural networks
...-platform, open source, freely available, and straightforward to use. Enables the users to create or modify annotations for a new or existing corpus. Train the neural network that performs the NER. During the training, NeuroNER allows monitoring of the network. Evaluate the quality of the predictions made by NeuroNER. The performance metrics can be calculated and plotted by comparing the predicted labels with the gold labels. -
15
WikiSQL
A large annotated semantic parsing corpus for developing NL interfaces
A large crowd-sourced dataset for developing natural language interfaces for relational databases. WikiSQL is the dataset released along with our work Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning. Regarding tokenization and Stanza, when WikiSQL was written 3-years ago, it relied on Stanza, a CoreNLP python wrapper that has since been deprecated. If you'd still like to use the tokenizer, please use the docker image. We do not anticipate switching... -
16
SmartMuseum
Software for work with Corpus of Everyday life history Sources
... collected in many museums and document archives. In this project, we consider the problem of creating software infrastructure for collaborative semantic annotation, information relation, and personalized access to corpus of everyday life history sources. Project financially supported from Department for Humanities of Russian Fund for Basic Research according to project # 16-01-12033. Authors: Vdovenko A., Marchenkov S., Petrina O., Korzun D. -
17
mwetoolkit
THIS PROJECT MIGRATED TO https://gitlab.com/mwetoolkit/mwetoolkit3/
THIS PROJECT MIGRATED TO https://gitlab.com/mwetoolkit/mwetoolkit3/ The Multiword Expressions toolkit aids in the automatic identification and extraction of multiword units in running text. These include idioms (kick the bucket), noun compounds (cable car), phrasal verbs (take off, give up), etc. Even though it focuses on multiword expresisons, the framework is quite complete and can also be useful in any corpus-based study in computational linguistics. The mwetoolkit can be applied... -
18
TextTools is a freeware corpus linguistics tool developed in Python to aid in research. This program analyzes user-created corpora and displays information about word (token) frequency, n-grams, clusters, collocations, keyword in context (KWIC), and keyness. TextTools is designed to be user-friendly and intuitive and will run natively on Mac OS X.
-
19
Redundancy due to cut-paste operations in text creates bias in machine learning for NLP. This module takes a directory and produces a subset of the files in that directory (in a list) with an upper bound on similarity between two files.
-
20
Chinese-Korean Bilingual Corpus
Construction of Chinese-Korean Bilingual Corpus and Search Technology
Construction of Chinese-Korean Bilingual Corpus and Search Technology. Some auto alignment programs and a search engine based on jung seong are provided. -
21
WebSynonymExtractor
a synonym extractor based on web-corpora and a multilingual translator
This project is an approach for synonym extraction and extending WordNet by the so found synonyms. The python application is realised as a kind of pipe that starts with a web-corpus-reader which is followed by several workers (tokenizers, lemmatizers, ...) and finally completed by a result writer. In contrast to the state of the art approaches, this implementation is based on single words found in the web used as a corpus and translated to other languages. If translations of different... -
22
-
23
Ug is a program to generate pseudo-realistic usernames using a categorised BNC (British National Corpus) wordlist. It combines words using pre-defined strategies such as adjective-noun or adjective-conjunction-adjective.
-
24
PyAnnotation is a Python Library to access and manipulate linguistically annotated corpus files. Supported file formats are Kura XML, Elan XML and Toolbox files. A Corpus Reader API is provided to support statistical analysis within the NLTK.
-
25