Download
Transformers.jl is a Julia library that implements Transformer models for natural language processing tasks. Inspired by architectures like BERT, GPT, and T5, the library offers a modular and flexible interface for building, training, and using transformer-based deep learning models. It supports training from scratch and fine-tuning pretrained models, and integrates with Flux.jl for automatic differentiation and optimization.
Features
- Implements standard Transformer architectures (BERT, GPT, etc.)
- Modular design for custom model configuration
- Pretraining and fine-tuning capabilities
- Tokenization and positional encoding support
- Compatible with Flux.jl and automatic differentiation
- Support for GPU acceleration via CUDA.jl
Project Samples
Categories
Natural Language Processing (NLP)License
MIT LicenseFollow Transformers.jl
Other Useful Business Software
Paessler - Monitor Your Whole Network in Minutes
Waiting weeks for a monitoring rollout isn't an option when infrastructure doesn't stop running. PRTG's auto-discovery scans your network and suggests from over 200 pre-configured sensor types, so you're watching servers, applications and devices within minutes, not after a multi-week deployment. Enterprise-strength monitoring, without the enterprise complexity. Start your free trial today.
Rate This Project
Login To Rate This Project
User Reviews
Be the first to post a review of Transformers.jl!
