Download
mistral-finetune is an official lightweight codebase designed for memory-efficient and performant finetuning of Mistral’s open models (e.g. 7B, instruct variants). It builds on techniques like LoRA (Low-Rank Adaptation) to allow customizing models without full parameter updates, which reduces GPU memory footprint and training cost. The repo includes utilities for data preprocessing (e.g. reformat_data.py), validation scripts, and example YAML configs for training variants like 7B base or instruct models. It supports function-calling style datasets (via "messages" keys) as well as plain text formats, with guidelines on formatting, tokenization, and vocabulary extension (e.g. extending vocab to 32768 for some models) before finetuning. The project also provides tutorial notebooks (e.g. mistral_finetune_7b.ipynb) to walk through the steps.
Features
- LoRA-based finetuning to reduce memory usage and enable efficient adaptation
- Support for both plain text (“pretrain”) and “instruct” / conversational datasets
- Utilities to reformat and validate data (including reformat_data.py)
- Example YAML configs for Mistral 7B training variants
- Tutorials / notebooks to guide new users (e.g. 7B finetuning example)
- Guidance on vocabulary extension, tokenization, and model compatibility
Project Samples
