#116 feat(classifier): whole-prompt token budget fit to the selected model

[Anonymous]

Originally created by: Akarsh-Hegde

Stacked on [#112] (feat/mlx-dynamic-model-selection). Base is that branch, not main — merge after [#112]. Both edit run_task_linker_mlx.py; stacking avoids the conflict.

TL;DR — read this first

This is a no-op for every model the classifier can select today. OptiQ has a 128k window (→ ~436k-char budget) and every selectable MLX model is ≥16k, so nothing truncates and the rendered prompt is byte-identical to the current static path (verified). No production inputs change → no eval cycle required.

What it delivers is principled hardening + insurance, not a behavior change:

• the previously uncapped candidate-ticket list is now bounded (with a visible "N omitted" note);
• the prompt is now model-aware (fits whatever window the loaded model has, vs a fixed char cap);
• insurance for a future sub-16k catalog model or a pathological huge-ticket deployment.

It does not fix Apple FM for the classifier — its system prompt alone (~4.85k tok) overflows a ~4k window regardless of user-message budgeting (deferred KAN-97 work).

What changed

• run_task_linker_mlx: read the model's max_position_embeddings from its config.json (authoritative, can't drift like a hand-kept map; clamped + conservative fallback), measure the fixed system prompt with the real loaded tokenizer (dominant constant term), and derive user_message_char_budget = (ctx − output_reserve − system − margin) × chars_per_token. Conservative chars/token biases toward truncating slightly early (overflow is fatal; under-fill is a mild quality cost).
• _prompts.build_user_message(…, char_budget=…): keep session_text at its normal cap; absorb overflow by truncating the candidate list first; only shrink session_text toward a floor when even the candidate floor won't fit. Both floored.
• Precedence mirrors _resolve_model_id: an explicit SESSION_TEXT_CAP env (incl. 0) pins the legacy static behaviour and disables the dynamic budget (eval reproducibility). char_budget=None preserves exact prior behaviour for non-MLX callers.

Verification

• OptiQ (128k) budget → rendered prompt byte-identical to static; production has 8 pm_tasks (~1.3k chars), nowhere near the ~436k budget → no-op deployment-wide.
• Tight budgets (8k/3k/1.5k) → in-bounds prompts with graceful candidate→session truncation.
• Live /classify_sessions round-trips; budget log shows the tokenizer-measured system prompt (4790 tok).
• No new unit-test failures (char_budget=None default is backward-compatible).

Caveat: "guaranteed fit" holds except in a ~5–5.5k-token window band too small to host the classifier at all — academic; no selectable model lives there.

Follow-up (not here)

A true small-window path for the classifier (Apple FM etc.) needs a SKILL.md/system-prompt redesign — separate, eval-gated, KAN-97.

🤖 Generated with Claude Code

[Anonymous]

Ticket changed by: adityaharishch

• status: open --> closed