Download
Parameter-Efficient Fine-Tuning (PEFT) methods enable efficient adaptation of pre-trained language models (PLMs) to various downstream applications without fine-tuning all the model's parameters. Fine-tuning large-scale PLMs is often prohibitively costly. In this regard, PEFT methods only fine-tune a small number of (extra) model parameters, thereby greatly decreasing the computational and storage costs. Recent State-of-the-Art PEFT techniques achieve performance comparable to that of full fine-tuning.
Features
- Accelerate for large scale models leveraging DeepSpeed and Big Model Inference
- Get comparable performance to full finetuning by adapting LLMs to downstream tasks using consumer hardware
- GPU memory required for adapting LLMs on the few-shot dataset
- Parameter Efficient Tuning of Diffusion Models
- GPU memory required by different settings
- Parameter Efficient Tuning of LLMs for RLHF components such as Ranker and Policy
Project Samples
License
Apache License V2.0Follow PEFT
Other Useful Business Software
$300 Free Credits for Your Google Cloud Projects
Launch your next project with $300 in free Google Cloud credits—no strings attached. Test, build, and deploy without risk. Use your credits across the entire Google Cloud platform to find what works best for your needs. After your credits are used, continue with always-free tier services. Only pay when you're ready to scale. Sign up in minutes and start exploring.
Rate This Project
Login To Rate This Project
User Reviews
Be the first to post a review of PEFT!
