Menu

#326 feat(cli): doberman baseline seed --from <traces> — production cold-seed for the subjective baseline

open
nobody
2026-08-10
2026-08-10
Anonymous
No

Originally created by: fu351

Problem

The subjective layer's per-entity streaming baseline (subjective/baseline.py, warmed via
observe(), read via drift.py::surprise_blended) starts every deployment cold. Until an entity
reaches K_OBSERVATIONS, its surprise term rides the conservative peer/global prior, so a fresh
install gets the least-sharp version of the adaptive layer exactly when an operator is most likely
to be evaluating it.

The #325 diagnostic showed the layer separates injection-induced actions from benign ones once
the baseline is warm
— but it warmed on a benchmark corpus inside the test harness. There is no
production path for an operator to warm their own baseline from traces they already have (a prior
decision log, an allow-listed workflow capture) before live traffic starts.

Proposal

Add a doberman baseline seed --from <path> command that warms the per-entity streaming baseline
from operator-supplied allowed-action traces, through the production observe() path — the
same code the live proxy calls, so seeding and live warming can never diverge.

  • Reads a documented JSONL trace format (algebra/classes + entity key only) or replays the
    local decision log's own PASS/allowed rows for the target entity.
  • Feeds each trace through observe(obj, entity_id=..., repo_root=..., now=...). No new learning
    path — this is a replay of the existing one.
  • Entity-scoped: seeds the entity_id = HMAC(agent_role | repo_root) the live path would use, so a
    seeded baseline is the one live traffic reads.
  • Prints a redaction-safe summary (per-entity n_observations, whether it crossed K_OBSERVATIONS
    / HST_WARMUP), never a payload or path value.

Invariants (hard)

  • Allowed-only learning. A trace marked blocked/denied is refused, never observed — seeding must
    not teach "normal" from a rejected action (same rule as the live path).
  • Raise-only / floor untouched. Seeding only warms the surprise baseline; it cannot lower a
    verdict or touch the objective / lethal-trifecta floor.
  • Redaction. Trace input and the summary carry classes/fingerprints only; fingerprints stay the
    keyed HMAC, never a plain hash of a low-entropy value.
  • Deterministic + local. No network, no clock dependence beyond an injected now.

Suggested tests

  • Seeding N allowed traces raises the entity's total_observations by exactly N.
  • A blocked/denied trace in the input is refused and observes nothing.
  • A synthetic secret in a trace never appears in the summary or any log (redaction).
  • A seeded baseline is read by surprise_blended for the same entity_id (seed-then-score round-trip).

Why this is separate work (from [#325])

#325 is a measurement-only diagnostic that warms on a benchmark corpus inside the test harness.
This is the production seeding path — a real CLI verb operators run against their own traces.
The advisor on the #325 build recommended scoping it as its own slice rather than deriving a seed
from AgentDojo (which would overfit the seed to a benchmark). This issue is that slice.

Touches the subjective baseline + redaction internals, so not a good-first issue.

Related

Tickets: #325

Discussion


Log in to post a comment.