#344 feat(onboarding): provision all three models (llm + reranker + embedder) end-to-end

[Anonymous]

Originally created by: Akarsh-Hegde

Why

The end-to-end pipeline (classification → worklog update) runs three on-device models, but onboarding only provisioned one:

• LLM Qwen3.5-2B — also does classification/matching (there is no separate classifier model).
• Reranker Qwen3-Reranker-0.6B — downloaded lazily on the first worklog run.
• Embedder Qwen3-Embedding-0.6B — downloaded lazily and its library (sentence-transformers/torch) was never shipped in the runtime, so /distill_hour would ImportError and crash the worklog pipeline at stage 1 on a fresh install.

Result: a fresh DMG could reach the dashboard before any model was ready, then stall or crash on its first worklog cycle.

What

Single source of truth + provision all three

• New services/agents/model_registry.py — declares the three models with env-overridable ids (MERIDIAN_LLM_ID / WORKLOG_RERANKER_ID / MERIDIAN_EMBEDDER_ID) and per-model HF download filesets. mlx_classifier, reranker, and session_distiller now read from it.
• routes/prefetch.py + _state.py download the whole set with aggregate progress. The wire contract the tray decodes (state/received/total/error) is unchanged, so no Rust decode changes were needed.

MLX-native embedder (no torch)

• session_distiller.py swapped sentence-transformers/torch for mlx_embeddings — keeps the runtime lean (no ~2.5 GB torch) and on the single MLX backend. Added to the [mlx] extra. Pooling verified at source: mlx_embeddings/models/qwen3.py does last-token pooling + L2 normalize, matching the previous ST behaviour.

Onboarding UX (the ask)

• The Model step now auto-installs the runtime and auto-downloads all models with no clicks, shows aggregate progress, and gates "Open Dashboard" on modelReady so the user can't reach the dashboard until all three are on disk. A Retry button appears on error so the gate is never a dead end.
• com.meridiona.mlx-server.plist pins HF_HOME so the wizard's eager prefetch and the runtime's lazy loads resolve the same cache; install.sh's first-run check now covers all three models.

Verification

• Python: all modules compile; registry resolves all three specs; no stale torch/sentence-transformers refs.
• TypeScript: setup wizard typechecks clean.
• Rust: cargo fmt + cargo clippy + cargo check pass (workspace incl. tray).
• Embedder pooling correctness confirmed at the mlx_embeddings source level.

Follow-up (network-blocked, not code)

A runtime cosine sanity check on the embedder (norms ≈ 1; related ≫ unrelated) is pending — the 8-bit shard repeatedly stalled on this machine's HF connection (the same download the product does on first run). Run on a good connection:

services/.venv/bin/python -c "from huggingface_hub import snapshot_download; snapshot_download('mlx-community/Qwen3-Embedding-0.6B-8bit')"

then exercise agents.session_distiller._embed.

🤖 Generated with Claude Code

[Anonymous]

Originally posted by: coderabbitai[bot]

[!IMPORTANT]

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: d1221eae-8658-484f-9c9b-49c961696577

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:
- [ ] 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

- [ ] Create PR with unit tests - [ ] Commit unit tests in branch `feat/onboarding-model-provisioning`

---

_{Comment @coderabbitai help to get the list of available commands.}

[Anonymous]

Originally posted by: Akarsh-Hegde

Fix summary — all review findings addressed

Commit: 07f3514f pushed to feat/onboarding-model-provisioning.

---

Findings addressed (code-review workflow + @adityaharishch)

#	Severity	Finding	Fix
1	🔴 High	start_mlx_server_cmd failure swallowed (.catch(() => {}))	Changed to .catch((e) => setMlxErr(String(e)))
2	🟡 Medium	canNext: (s) => s.modelReady gate	Intentionally kept — this is the PR's core deliverable; findings 1, 4, and 6 dissolve the trap by properly surfacing errors
3	🟡 Medium	snapshot_download no stall guard	Deferred — OS socket timeout (~75 s) handles genuine stalls; a wall-clock thread timeout would kill slow-but-progressing downloads on bad connections
4	🔴 High	Apple Intelligence guard removed from prefetch_model()	Restored: if app_state.get("mlx_module") is None: return {"state": "done", ...}
5	🟢 Low	_embed() loads embedder for empty input	Added early return guard before _get_embedder() to skip model load when texts = []
6	🟡 Medium	Inner model_prefetch OTEL span missing ERROR status	Added try/except inside span: span.set_status(ERROR) + span.record_exception(exc) + models[i]["state"] = "error"
7	🟡 Medium	pm_worklog_update/config.py hardcoded LLM default	Changed to os.environ.get("MLX_SERVER_MODEL") or model_registry.llm_id()
8	🟡 Medium	Provisioning errors appear on wrong steps	Separate mlxErr state for provisioning errors; only err (step errors) goes to the Footer
9	🟢 Low	Rust speed uses cumulative average	Deferred — cosmetic display issue; not worth touching the streaming loop in this PR
10	🟡 Medium	mlx_classifier._get_model() reads MODEL_ID module-constant	Changed to model_registry.llm_id() per-call; same for _get_tokenizer()
A1	🔴 High	mlx_server.rs returns Ok(()) on any non-2xx from /prefetch_model	Narrowed graceful-degrade to 404-only; 5xx and other errors now return Err so wizard surfaces Retry
A2	🟡 Medium	Per-model state stays "downloading" after failure	Fixed by finding 6 fix above
A3	🟡 Medium	Three sequential HF size probes add 6–15 s pre-download latency	Parallelized with asyncio.gather
A4	🟡 Medium	Stale state="downloading" snapshot in prefetch_status()	Re-checks prefetch_state["state"] under lock after the rglob walk
A5	🟡 Medium	Shared _MLX_ALLOW_PATTERNS list across all three ModelSpec instances	Changed to tuple[str, ...] — immutable, each spec independently safe
A6	🟢 Low	_embed() returns (0, 0) for empty input	Same fix as finding 5 above
A7	🟢 Low	Model-identity idempotency guard removed	Accepted risk — MERIDIAN_LLM_ID runtime change is an edge case with no user-facing path
A8	🟢 Low	Rust speed display inconsistency	Same as finding 9 above

---

All pre-push checks passed (cargo fmt, cargo clippy, cargo test, UI build, UI tests, security audit). PR is ready for re-review.

[Anonymous]

Ticket changed by: Akarsh-Hegde

• status: open --> closed