This is a Pytorch implementation of Reformer. It includes LSH attention, reversible network, and chunking. It has been validated with an auto-regressive task (enwik8).
Features
- Positional Embeddings
- Masking
- Documentation available
- Examples available
- Reformer Encoder Decoder Architecture
- Product Key Memory
- Customize Feedforward
- Helpers for training auto-regressive models
Categories
Machine LearningLicense
MIT LicenseFollow Reformer PyTorch
Other Useful Business Software
Try Google Cloud Risk-Free With $300 in Credit
Use your credit across every product. Compute, storage, AI, analytics. When it runs out, 20+ products stay free. You only pay when you choose to.
Rate This Project
Login To Rate This Project
User Reviews
Be the first to post a review of Reformer PyTorch!