Name | Modified | Size | Downloads / Week |
---|---|---|---|
Parent folder | |||
README.md | 2023-10-26 | 812 Bytes | |
Release v1.1.3 (Bring your own data, LoRA training) source code.tar.gz | 2023-10-26 | 13.2 MB | |
Release v1.1.3 (Bring your own data, LoRA training) source code.zip | 2023-10-26 | 13.3 MB | |
Totals: 3 Items | 26.5 MB | 0 |
Updates
- Support LoRA for the instruction tuning stage of LLaVA-1.5 -- comparable performance to full-model finetuning, and reduced requirements on GPU VRAM. (ckpts/logs, script)
- Bring your own data and finetune LLaVA-1.5 to your own task. (instruction)
- Basic support for Windows. (instruction)
- Fix: the training behavior with gradient accumulation is the same as large-batch training.
Notes
- A new LoRA schedule for LLaVA-1.5 is used,
- rank: 128
- alpha: 256
- lr (LoRA): 2e-4
- lr (projector): 2e-5