Download
Parameter-Efficient Fine-Tuning (PEFT) methods enable efficient adaptation of pre-trained language models (PLMs) to various downstream applications without fine-tuning all the model's parameters. Fine-tuning large-scale PLMs is often prohibitively costly. In this regard, PEFT methods only fine-tune a small number of (extra) model parameters, thereby greatly decreasing the computational and storage costs. Recent State-of-the-Art PEFT techniques achieve performance comparable to that of full fine-tuning.
Features
- Accelerate for large scale models leveraging DeepSpeed and Big Model Inference
- Get comparable performance to full finetuning by adapting LLMs to downstream tasks using consumer hardware
- GPU memory required for adapting LLMs on the few-shot dataset
- Parameter Efficient Tuning of Diffusion Models
- GPU memory required by different settings
- Parameter Efficient Tuning of LLMs for RLHF components such as Ranker and Policy
Project Samples
License
Apache License V2.0Follow PEFT
Other Useful Business Software
Train ML Models With SQL You Already Know
Build and deploy ML models using familiar SQL. Automate data prep with built-in Gemini. Query 1 TB and store 10 GB free monthly.
Rate This Project
Login To Rate This Project
User Reviews
Be the first to post a review of PEFT!