#322 test(benchmarks): harness fidelity — real inference mapping + per-mode sweep (W-A0.1/A0.2)

[Anonymous]

Originally created by: fu351

What this PR does

Two benchmark-harness fidelity fixes (two-hosts W-A0.1 + W-A0.2). Test/tooling-only — no src/ changes.

A0.1 — the mapping runs production inference (tests/benchmarks/mapping.py)
to_security_object now runs the same generic inference the proxy runs in production (infer_algebra → apply_adapters → infer_reversibility) instead of handing the engine a default all-unknown Algebra. Before this, provenance was unknown and destination none on every benchmarked action, so whole classes of algebra-keyed guardrails (the lethal-trifecta floor, capability/reversibility rules) were silently un-exercised by the entire benchmark. Inference never raises — a failure leaves the conservative default.

A0.2 — per-mode sweep (tests/benchmarks/runner.py, run.py)
--mode {light,balanced,strict,paranoid,all} threaded through run_suite/run_profiles/run_before_after and the CLI (default None = the suite's per-case mode). --mode all reproduces the four-mode CLI numbers keyed by mode.

No metric change

Confirmed no movement on current suites: synthetic ASR unchanged; AgentDojo numbers identical (its attacks don't line up the trifecta target/credential legs, so populating the algebra doesn't shift them). This makes the harness faithful, it doesn't tune a number.

Tests added (run in CI)

tests/unit/test_benchmark_harness_fidelity.py — 5 tests:

• red→green: the mapping populates inferred algebra (provenance/destination) instead of the all-unknown default
• the mapping's output mirrors a direct production-inference call exactly
• benign guard: a user-source action reads as trusted_instruction, never mixed/untrusted
• --mode override threads to EvalContext.mode; mode=None uses the case default

Verification

Full tests/unit + benchmark synthetic-gate integration test → 100%, zero failures, exit 0. ruff check / ruff format --check clean; lint-imports 2/2 contracts kept.

[Anonymous]

Ticket changed by: fu351

• status: open --> closed