#103 feat: fleet feedback loop — eval-feedback capture + system-health doctor

[Anonymous]

Originally created by: Akarsh-Hegde

Fleet feedback loop: eval-feedback capture + a system-health / fault-attribution doctor

This branch builds the foundation for evaluating Meridian in the wild — making the question "why was this session misclassified?" answerable, instead of silently blaming the model. It has two complementary halves.

---

1. Eval-feedback capture (the label faucet)

The classifier's quality is only measurable if user corrections are captured. The worklog review surface is the natural source — every approve/reject/edit is a free label — but those actions were being thrown away.

• Every review action is now recorded. POST /api/worklogs/:id (approve / reject / unapprove) writes an immutable pm_worklog_feedback row, atomically with the state change (previously only edit was captured; state transitions were lossy).
• Reject now captures the attribution label — the [#1] accuracy signal. Migration 031 adds corrected_task_key + corrected_to_untracked to pm_worklog_feedback; the "Dismiss" button opens a "Where should this time have gone?" picker (candidate tickets from /api/tasks, plus Untracked / Not sure). A rejection now yields a real ground-truth correction, not just a weak negative.

Files: src/migrations/031_worklog_reject_attribution.sql, ui/app/api/worklogs/[id]/route.ts, ui/components/views/WorklogsView.tsx.

---

2. System-health / fault-attribution layer — meridian doctor

The principle: before a misclassification can be blamed on the AI, rule out that the pipeline was broken. Faults form a ladder; attribute to the earliest broken rung:

• L1 capture (screen-recording / accessibility permission, screenpipe alive, frames landing)
• L2 integration (MLX reachable + model loaded, Jira auth, queue health)
• L3 candidate completeness (JQL coverage, ticket-cache freshness)
• L4 model fair-shot (prompt rendered, text caps) — plus config and prerequisites

meridian doctor grew from a flat bash checklist into a comprehensive, colourised, by-daemon health engine covering every dependency the product needs to run. All checks are content-free (counts, timestamps, statuses, reachability — never screen content), so the same engine is safe to run on a design partner's machine.

What it covers (11 groups, ~50 checks)

system · meridian daemon · screenpipe · mlx-server · jira · worklog · coding-agent · ui · mcp · observability · config

Highlights:

• screenpipe — frame freshness, blank-rate (Screen-Recording proxy), and per-app accessibility yield (a best-app oracle, robust to app mix — verified that VSCode/Claude actually report 96–98% a11y, disproving the "Electron = no a11y" assumption)
• meridian daemon — ETL last-run + freshness, frame cursor, summariser/classifier queue depth, subprocess_error sentinel count
• mlx-server — /health reachability + /info model-loaded readiness (liveness ≠ readiness — the gap the old TCP preflight missed)
• jira — /myself auth that distinguishes 401-expired / 403-scope, sync freshness, candidate completeness
• worklog — drafts, stuck hours, Jira post failures
• coding-agent — Claude/Codex CLI + ingest dirs, and the Cursor blind-spot note
• config — the cross-process contracts (DB-path MERIDIAN_DB vs MERIDIAN_DB_PATH, the settings.json split-brain, dead env vars)
• platform — launchd agents, plist lint, process liveness, node/python toolchains, disk

Rendering + UX

• Grouped ▸ daemon table, severity glyphs in colour (gated on IsTerminal), redundant prefixes stripped, → remedy lines under every failure, a ⚠ N items need attention header, and a ✓/·/⊘/✗ counts footer. Exits non-zero on any critical.
• Root-cause diagnosis + escalation: the report ends with a Diagnosis section that correlates scattered warnings into their underlying cause (e.g. "summariser stalled → that's why hours are stuck and sessions sentinelled"), then an escalation footer pointing at meridian doctor --fix, the team, or a claude hand-off.

Two entry points

• Boot preflight — the L1 capture checks run at daemon startup and log warn/error (non-fatal).
• meridian doctor — the on-demand sweep. The meridian CLI wrapper now delegates to the daemon binary's report (perl-alarm-guarded so a stale binary can't hang the terminal), with a minimal bash fallback. A --porcelain TSV mode is available for tooling.

Architecture

A src/health/ module per subsystem (capture, daemon, mlx, jira, worklog, codingagent, observability, contracts, platform, diagnose) returning Vec<Check>; run_all assembles the grouped report. ~1,800 lines.

---

Validation

Running this against a live machine immediately surfaced real issues a flat checklist hid:

• a 293-session summariser backlog → cascading to 5 stuck worklog hours + 10 sentinel errors (correctly chained into one root cause)
• a stale Jira ticket cache (distinguished from a token problem)
• the settings.json split-brain — ~/.meridian/settings.json (UI-written) is not read by the daemon (a real latent config bug, confirmed)

Testing

• 15 health unit tests (rendering: grouping, prefix-strip, remedies, counts, colour gating, porcelain columns; per-app a11y logic; which, derive_healthz, contract expansion; diagnosis correlation).
• Full suite green: 142 lib + integration tests, cargo fmt + clippy -D warnings clean.

Also included

4 pre-existing eval/classifier commits already on the branch (deepeval goldens, ExtractThenClassify experiment config, SKILL_PATH override, real hand-labeled golden runs) — the experimentation side of the same feedback effort.

Follow-ups

• meridian doctor --fix (auto + guided-with-confirm remediation, escalation bundle / claude hand-off) — next increment.
• Wire the doctor engine into the periodic heartbeat that writes fault intervals for the eval-attribution layer.

🤖 Generated with Claude Code

[Anonymous]

Ticket changed by: Akarsh-Hegde

• status: open --> closed