#414 feat(setup): expected-memory panel + HF caching-proxy for model downloads

[Anonymous]

Originally created by: Akarsh-Hegde

Summary

Two independent, setup-phase improvements (plan: ~/.claude/plans/spicy-splashing-pretzel.md).

Part 1 — Setup step shows status + expected memory (UI-only)

The "Local intelligence" step now surfaces detected hardware (chip · macOS · unified RAM) and the engine's expected resident footprint against the Mac's memory, so users see what it costs before finishing setup. Reuses the existing Bar atom; degrades to a bare estimate line when detect_system_specs is unavailable (non-macOS).

• ui/app/setup/steps.tsx — new SpecMemoryPanel in MLXBody (shown while provisioning and after ready; hidden on error/unavailable).

Visual preview (all states + light/dark): https://claude.ai/code/artifact/1f614ca7-de30-4b2e-87ec-b0d8ccd97310

Part 2 — Model downloads via a Cloudflare Worker HF caching proxy

Users retry stalled model pulls from HF. This adds an optional edge cache in front of huggingface.co — HF stays the source of truth (no self-hosting/re-upload); we just cache the weight blobs at Cloudflare's edge. No Python / runtime republish (huggingface_hub honours HF_ENDPOINT from the env, so every runtime version routes through it automatically).

• infra/hf-proxy/ — new Cloudflare Worker (wrangler.toml + src/index.js): transparent reverse-proxy, full weight GETs edge-cached via cf.cacheEverything + a stable cacheKey (signed CDN query stripped) so Cache Reserve fills; metadata/HEAD/resume pass through; pass-through to HF origin on any error.
• tray/src-tauri/src/mlx_server.rs — hf_endpoint() resolver (debug env → option_env! → baked default) + sets HF_ENDPOINT on the in-process setup server.
• services/scripts/{com.meridiona.mlx-server.plist, install-mlx-server-daemon.sh} — launchd (dev/bundle) parity: injects a whole HF_ENDPOINT key/value block or nothing — never an empty value (which huggingface_hub would take literally and break downloads).

Ships DISABLED by default (HF_PROXY_ENDPOINT = "") — baking a live default before the Worker exists would point every download at a dead host. Enable per channel via option_env!("MERIDIAN_HF_ENDPOINT") once the Worker + Cache Reserve are live.

Notes / deviations from plan

• Plist wiring is in install-mlx-server-daemon.sh (the actual renderer), not backend_install.rs — the DMG supervises MLX in-process via start() (patched) and removes the legacy launchd agent.
• Default is disabled, not the live hf.meridiona.com (safe-by-default).

Requires before enabling (infra)

Cloudflare zone + Worker route + Cache Reserve (R2-backed; the default edge cache caps objects at 512 MB, below the ~1.4 GB llm). Then bake MERIDIAN_HF_ENDPOINT per channel.

Test

• Rust: cargo fmt / clippy -D warnings clean, 46/46 tray tests pass.
• UI: build ✓, bun test 177/177 ✓, tsc clean for changed files.
• Worker: wrangler deploy --dry-run bundles clean.
• Plist lints (plutil) in both empty and set states.

Not done (needs infra/running app): staging Worker deploy + live wizard smoke against an empty HF cache to confirm edge-cache HITs.

Sequencing (confirmed)

Prod runtime 1.68.0 already includes the [#350] hf_xet fix (it landed at 1.63.0), so the retries are not the Xet hang and no runtime republish is needed — this proxy is the actual durable fix. Verified via the live runtime-latest manifest vs the fix commit.

🤖 Generated with Claude Code

[Anonymous]

Originally posted by: coderabbitai[bot]

[!IMPORTANT]

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 4ee226de-9d70-4dfe-8b45-71f78de42b05

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:
- [ ] 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

- [ ] Create PR with unit tests - [ ] Commit unit tests in branch `feat/setup-mlx-status-and-model-cdn`

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

- [X](https://twitter.com/intent/tweet?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A&url=https%3A//coderabbit.ai) - [Mastodon](https://mastodon.social/share?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A%20https%3A%2F%2Fcoderabbit.ai) - [Reddit](https://www.reddit.com/submit?title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&text=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code.%20Check%20it%20out%3A%20https%3A//coderabbit.ai) - [LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcoderabbit.ai&mini=true&title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&summary=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code)

_{Comment @coderabbitai help to get the list of available commands.}

[Anonymous]

Originally posted by: Akarsh-Hegde

Addressed all three points in 1de79528:

Security concern (cache/auth) — fixed. isFullWeightGet now also requires no Authorization header; an authenticated resolve GET falls through to the plain pass-through path instead of cacheEverything, so a gated/private-repo response can never be written into the shared edge cache. Added an explicit SECURITY: comment block at the top of the file documenting the invariant (public repos only) so a future model addition doesn't silently violate it.

Test coverage — added hf_endpoint_disabled_by_default. Mirrors the "no override at any tier → None" invariant this PR's safety story depends on. Scoped to only clear MERIDIAN_HF_ENDPOINT (a var no other test in the suite touches), so it can't race under parallel cargo test execution without pulling in serial_test — consistent with why manifest_url() itself isn't unit-tested the same way (its env-tiered behavior is only exercised end-to-end by the #[ignore]d live_pull_from_runtime_staging).

Sed nit — fixed, and found a second related bug while verifying it. Escaped sed replacement metacharacters (\, &, our | delimiter) in HF_ENDPOINT_ENTRY per your note. While testing that fix against an adversarial URL containing &, I found the injection was also broken at the XML level — a literal & in a URL (e.g. a query string) produced invalid XML (plutil rejected it: "unknown ampersand-escape sequence"), independent of the sed issue. Fixed both: XML-escape the URL before embedding it in the <string> element, then sed-escape the resulting entry before the outer substitution. Verified round-trip with plutil -lint against &, |, \, a clean URL, and the empty/disabled case — all lint OK and extract back to the exact input.

All pre-push checks green on the updated branch (fmt, clippy, cargo test incl. the new test, UI build, bun test, security audit).

[Anonymous]

Ticket changed by: Akarsh-Hegde

• status: open --> closed