Search Results for "transformer"

Megatron is a large, powerful transformer developed by the Applied Deep Learning Research team at NVIDIA. This repository is for ongoing research on training large transformer language models at scale. We developed efficient, model-parallel (tensor, sequence, and pipeline), and multi-node pre-training of transformer based models such as GPT, BERT, and T5 using mixed precision.

...We aim to make this repo a centralized and accessible place to gather techniques for training large-scale autoregressive language models, and accelerate research into large-scale training. For those looking for a TPU-centric codebase, we recommend Mesh Transformer JAX. If you are not looking to train models with billions of parameters from scratch, this is likely the wrong library to use. For generic inference needs, we recommend you use the Hugging Face transformers library instead which supports GPT-NeoX models.

Text Classification is a deep learning repository focused on text classification models for NLP. It provides a broad set of baseline architectures that can be used to study, train, compare, and adapt classification approaches. The project supports both single-label and multi-label classification, making it useful for sentence-level and document-level tasks. It includes classic and advanced models such as fastText, TextCNN, BERT, TextRNN, RCNN, hierarchical attention networks, seq2seq...

...The material teaches both model structure and training workflows, with examples built in PyTorch and TensorFlow through Keras. Classification topics range from LeNet and AlexNet to ResNet, EfficientNet, Vision Transformer, Swin Transformer, ConvNeXt, and MobileViT. Additional sections cover object detection, semantic segmentation, instance segmentation, and keypoint detection using widely studied models. The project is designed as a learning resource for students and developers who want readable code and guided comparisons across computer vision tasks.