whichllm 0.6.5

This release adds Abyss to the engines whichllm knows, stops refusing PCs that are exactly the size a pack asks for, and brings seven engine cards up to date.

Abyss

Abyss is a fork of Project Maya that runs DeepSeek V4 and V4.1 Flash, GLM-5.3-Flash, Qwen3.8-Flash-Next, Ornith-1.5 and Qwen3.5-122B-A10B on one GPU, with the experts spread over VRAM, RAM and the SSD. Its card lists the 15 packs its setup installs with one command.

The author's packs are named for the PC they target, such as 16G64G for a 16 GB GPU with 64 GB of RAM, and each one fits in memory on a machine of that size. The two DeepSeek V4.1 packs are the exception, since that model's experts don't fit a 32 GB GPU with 96 GB of RAM and some are read from the SSD. On a 16 GB GPU with 64 GB of RAM, Abyss shows up under Qwen3.8-Flash-Next.

  #2 also runs with Abyss (zephel01 IQ3_M 16G64G, 70.7 tok/s in its docs on RTX 5090 limited to 16 GB, 58 GB memory limit, with its MTP drafter):
    ./setup-deepseek.sh --model qwen38

On a 12 GB GPU with 32 GB, where llama.cpp can't place the model at all, Abyss's 12G32G pack puts it in the ranking as a row of its own.

  Run #7 with Abyss:
    ./setup-deepseek.sh --model qwen38 --quant 12G32G

Three things to know. The speeds are the author's, measured on a Ryzen AI Max+ 395 whose RAM is about three times as fast as a usual DDR5 desktop's, so expect less on one. Abyss doesn't run under WSL, and whichllm engines says so there. And Ornith-1.5 has no benchmark score yet, so its packs count in whichllm engines but don't reach the ranking.

A PC the size a pack asks for

A PC reports a little less than the size it's sold as. An RTX 5070 "12 GB" has 11.9 GiB of VRAM, and a "32 GB" PC has about 31.2 GiB of RAM. whichllm compared those numbers with the round ones in the engines' docs, so two kinds of machine were turned away.

• Strata's packs need a 12 GB GPU. On an RTX 5070 they were refused with "needs a GPU with 12 GB, the largest here has 12 GB".
• Project Maya's packs run from 32 GB of RAM. On a 32 GB PC they were refused with "needs 32 GB of RAM, this machine has 31 GB".

An amount within 4% of the size a pack names now counts as that size, for VRAM, RAM and the two together.

Engine cards, re-read

The weekly check of cards against their upstream ran for the first time on Monday and listed seven. I re-read each one against what changed in its sources.

• Project Maya 1.1.57. Maya's new memory page gives each pack the VRAM and RAM together it recommends, and the card uses those now. Maya-S of GLM-5.3-Flash needs 128 GB, not the 96.5 GB of its file. Maya-S of DeepSeek V4.1 needs 224 GB, not 340, since the Engram tables stay on the SSD. GLM-5.3, the 744B model, is new, and MiniMax-M3.1-Flash has an experimental Mac build.
• Strata 0.2.13. DeepSeek V4.1 on a Mac reads pictures too. Strata also runs seven of Unsloth's re-uploaded Qwen3.8-Flash-Next files, but its docs don't give the memory they need yet, so they aren't packs on the card.
• ik_llama.cpp loads mainline NVFP4 GGUFs now.
• llama.cpp b12247, Ollama 0.40.16, Lemonade 2026.45.3. Nothing the cards state changed.

Abyss and Maya ship a release most days, which would put them on that list every week. Their cards now follow minor versions, and come due again after 30 days.

How it was checked

All 1,980 tests pass, and the wheel carries all 14 engine cards. Every pack on Abyss's card matches its files on Hugging Face, and its setup options exist in Abyss's setup script. I ran the CLI on eleven simulated machines, from a 12 GB GPU with 16 GB of RAM to a 32 GB GPU with 128 GB, a Radeon, a Mac and CPU only, with no crash and the fits its docs describe. I haven't run Abyss itself, and this release wasn't run natively on Windows.

Upgrade

:::bash
uv tool upgrade whichllm
pip install -U whichllm
brew upgrade whichllm

Everything else is in CHANGELOG.md (github.com).