Download Latest Version Less VRAM + bug fixes source code.tar.gz (4.7 MB)
Email in envelope

Get an email when there's a new version of Unsloth

Home / 2025-03
Name Modified Size InfoDownloads / Week
Parent folder
Gemma 3 + FFT Support source code.tar.gz 2025-03-14 2.8 MB
0
Gemma 3 + FFT Support source code.zip 2025-03-14 2.8 MB
0
README.md 2025-03-14 9.8 kB
0
Totals: 3 Items   5.6 MB 0

March Release 🦥

Get the latest stable Unsloth via:

pip install --upgrade --force-reinstall --no-cache-dir unsloth unsloth_zoo

The March release should be stable - you can force the version via:

pip install "unsloth==2025.3.15" "unsloth_zoo==2025.3.13"

New Features

  • Read all details here: https://unsloth.ai/blog/gemma3

  • Gemma 3 1B, 4B, 12B and 27B finetuning all work now! Colab Notebook We fixed some issues which caused Gemma 3 training loss to be very high. This includes some tokenization issues so fine-tuning Gemma 3 will now work correctly if you use Unsloth. image

  • We also encountered many infinite gradients during Gemma 3 (1B to 27B) finetuning. We found float16 mixed precision (Tesla T4, RTX 2080 series) to not function well, and we defaulted to float32 precision. Float16 also failed on A100, so this is a hardware agnostic issue. Bfloat16 is fine though! Unsloth auto selects the best data-type! You do not have to do anything! Colab Notebook to finetune Gemma 3

  • Preliminary support for full-finetuning and 8bit finetuning - set full_finetuning = True or load_in_8bit = True Both will be optimized further in the future! A reminder you will need more powerful GPUs!

    :::python model, tokenizer = FastModel.from_pretrained( model_name = "unsloth/gemma-3-4B-it", max_seq_length = 2048, # Choose any for long context! load_in_4bit = True, # 4 bit quantization to reduce memory load_in_8bit = False, # [NEW!] A bit more accurate, uses 2x memory full_finetuning = False, # [NEW!] We have full finetuning now! # token = "hf_...", # use one if using gated models ) * New Unsloth Auto Model support - nearly all models are now supported! We now supports vision and text models out of the box, without the need for custom implementations (and all are optimized!) * Mixtral (yes finally!), Gemma 3, Granite 3.2, Cohere, OLMo, Reka, and generally any vision or language model! There might be some occasional models which don't work!

    :::python model, tokenizer = FastModel.from_pretrained( model_name = "mistralai/Mixtral-8x7B-Instruct-v0.1", ) * Windows support via pip install unsloth should function now! Utilizes https://pypi.org/project/triton-windows/ which provides a pip installable path for Triton. Use:

    pip install unsloth * Train on completions / responses only for vision models supported! Use it like below:

    :::python data_collator = UnslothVisionDataCollator( model, tokenizer, train_on_responses_only = False, instruction_part = "<|start_header_id|>user<|end_header_id|>\n\n", response_part = "<|start_header_id|>assistant<|end_header_id|>\n\n", ) SFTTrainer(..., data_collator = data_collator) * Conversions to llama.cpp GGUFs for 16bit and 8bit now DO NOT need compiling! This solves many many issues, and this means no need to install GCC, Microsoft Visual Studio etc!

    :::python model.save_pretrained_merged("gemma-3-finetune", tokenizer) model.save_pretrained_gguf( "gemma-3-finetune", quantization_type = "Q8_0", # For now only Q8_0, BF16, F16 supported ) * Vision models now auto resize images which stops OOMs and also allows truncating sequence lengths! * Many multiple optimizations in Unsloth allowing a further +10% less VRAM usage, and >10% speedup boost for 4bit (on top of our original 2x faster, 70% less memory usage). 8bit and full finetuning also benefit! * GRPO in Unsloth now allows non Unsloth uploaded models to be in 4bit as well - reduces VRAM usage a lot! (ie pretend your own finetune of Llama) * New training logs and infos - training parameter counts, total batch size image

  • Vision models now also work for normal text training! This means non vision notebooks can work with vision models!

  • Complete gradient accumulation bug fix coverage for all models!

  • GRPO notebook for Gemma 3 coming soon with Hugging Face's reasoning course!
  • DoRA, Dropout, and other PEFT methods should just work!

Bug fixes

  • Faster and less error prone streamlined finetuning experience! Apologies for the recent issues with constant releases and breaking breaks - the March release should be stable! Ie pip install "unsloth==2025.3.14" "unsloth_zoo==2025.3.12"
  • Pixtral and Llava finetuning are now fixed! In fact nearly all vision models are supported out of the box! Please update transformers for Pixtral: pip install --no-deps git+https://github.com/huggingface/transformers.git
  • Fixed all Colabs not working - cloud instances like Runpod should just work now!
  • Fixed many many bugs - will reply to each issue with updates!

Other items

New Contributors

Full Changelog: https://github.com/unslothai/unsloth/compare/2025-02...2025-03

Source: README.md, updated 2025-03-14