Search Results for "corpora"

...It was created to make multilingual vector representations easier to obtain, especially for languages with fewer readily available resources than English. The repository provides models trained using both Word2Vec and fastText. Training corpora are constructed from Wikipedia database dumps using language-specific preprocessing when necessary. Scripts are included for corpus creation and for training new embeddings with either supported algorithm. Available languages include Chinese, Japanese, Korean, Spanish, French, German, Hindi, Russian, Vietnamese, Thai, and many others, with model metadata covering vector, corpus, and vocabulary sizes.

RC-Data is a dataset generation framework created by Google DeepMind to produce large-scale reading comprehension question-answer pairs from CNN and Daily Mail news articles. The dataset, introduced in the 2015 paper “Teaching Machines to Read and Comprehend” (Hermann et al., NIPS 2015), was among the first large corpora designed to train and evaluate machine reading and comprehension models. The repository provides scripts for downloading archived CNN and Daily Mail articles from the Wayback Machine and automatically generating cloze-style questions where entities in the text are replaced with placeholders. Each data instance consists of a news article (context), a generated question, and its corresponding answer, making it suitable for supervised machine learning setups. ...

...Also, it comes with a WordNet integration. If you only intend to use TextBlob’s default models (no model overrides), you can pass the lite argument. This downloads only those corpora needed for basic functionality. TextBlob is also available as a conda package.