imference-desktop Files

Local generation grows a whole new backend and a much clearer setup flow.

Krea 2, locally

Load Krea 2 Turbo checkpoints from civitai as-is — scaled-fp8, plain fp8, and int8 files all work (engine v0.4.2 dequantizes in memory, nothing is rewritten on disk). Quantized checkpoints stay fp8-resident (~13 GB instead of ~26), and the Turbo recipe comes pre-filled: 8 steps, CFG 0 — which the UI now fully supports, with a tooltip explaining that for Krea 2, 0 means guidance off (the nominal setting). Text-to-image for now.

Runs on 8 GB cards

Offloading is now a UI setting (no more environment variables): backend-aware, with a group-offloading mode that streams the model block-by-block — the 12–20B models (FLUX, Qwen-Image, Krea 2) become runnable on 8 GB GPUs, at the cost of host RAM and some step time.

A single "local readiness" panel

The local flow is centralized in one preflight card above the model picker: weights → shared components → engine, each row with its own status and action, collapsing to a single ✓ line when everything is ready. Downloading components (text encoders, VAEs — including SDXL's fp16-fix VAE, previously fetched silently at first generate) is now an explicit, visible step. The primary button is always Generate.

Faster, safer downloads

Model and component downloads use multipart range requests (6 parallel streams) with graceful fallback. Custom checkpoints get per-backend sampling defaults applied on load, so a Krea 2 file starts at its recipe instead of generic values.

Under the hood

• Engine pinned to imference-engine v0.4.2 (diffusers 0.40.0 · transformers 5.4.0 · peft 0.19.1). The startup version check force-reinstalls a mismatched venv, so existing installs upgrade on first launch.
• Prompts with broken emoji/surrogate characters no longer crash the tokenizer (engine-side fix).
• An explicit CFG 0 (or steps value) is preserved end-to-end — the sidecar no longer substitutes defaults over meaningful zeros.