Search Results for "multi language"

Stanza is a collection of accurate and efficient tools for the linguistic analysis of many human languages. Starting from raw text to syntactic analysis and entity recognition, Stanza brings state-of-the-art NLP models to languages of your choosing. Stanza is a Python natural language analysis package. It contains tools, which can be used in a pipeline, to convert a string containing human language text into lists of sentences and words, to generate base forms of those words, their parts of...

...Flax emphasizes composability: optimizers, training loops, and checkpointing are provided as examples or utilities rather than monolithic frameworks, encouraging research-friendly customization. The library is widely used in vision, language, and reinforcement learning, often serving as a thin layer atop NumPy-like JAX primitives. Tutorials and examples show patterns for multi-host training, mixed precision, and advanced input pipelines that scale from laptops to TPUs.

Fairseq(-py) is a sequence modeling toolkit that allows researchers and developers to train custom models for translation, summarization, language modeling and other text generation tasks. We provide reference implementations of various sequence modeling papers. Recent work by Microsoft and Google has shown that data parallel training can be made significantly more efficient by sharding the model parameters and optimizer state across data parallel workers. These ideas are encapsulated in the...

char-rnn is a classic codebase for training multi-layer recurrent neural networks on raw text to build character-level language models that learn to predict the next character in a sequence. It supports common recurrent architectures including vanilla RNNs as well as LSTM and GRU variants, letting users compare behavior and output quality across model types. It is straightforward: you provide a single text file, train the model to minimize next-character prediction loss, then sample from the trained network to generate new text one character at a time in the style of the dataset. ...