Menu

#332 Prove parity cell: tool output carrying credentials is blocked from the model, on the MCP proxy

closed
nobody
2026-08-15
2026-08-11
Anonymous
No

Originally created by: fu351
Originally owned by: jasperdingg

Heads up before you start: this one is lighter than it looks. The proving test already exists, it's just missing the marker.

The guarantee is proven on Claude Code by tests/unit/test_hosthook_claude_post.py::test_secret_string_response_is_blocked. On the MCP proxy, the same mechanism (_scan_output_for_secrets, src/doberman/proxy/executor.py, ADR 0062) is already implemented, and already has a real, unstubbed, end-to-end test proving exactly this: tests/unit/test_proxy_secret_output_gating.py::test_secret_output_triggers_block_gate_without_leaking_secret. It drives executor.decide_and_execute with a downstream result carrying a synthetic AWS key and asserts the response is gated to BLOCK, the secret never reaches the response text, and it never lands in the decision log or the raw db file. It just doesn't carry a @pytest.mark.guarantee marker yet, so the matrix doesn't know it exists.

What to do

  1. Read tests/unit/test_proxy_secret_output_gating.py in full. Confirm for yourself the test genuinely proves the guarantee (real engine, no stubbing of the scan itself).
  2. Add the marker: @pytest.mark.guarantee("output-secret-scan", host="mcp-proxy") on test_secret_output_triggers_block_gate_without_leaking_secret.
  3. Mutation-check it anyway (the bar for a ✅, not just a pre-existing green test): temporarily disable the _scan_output_for_secrets call site in decide_and_execute (executor.py, around lines 623 and 722), confirm the test goes red, revert. Note "mutation-checked" in the PR.
  4. Regenerate: python -m tools.parity.generate_parity.

If step 1 turns up a real gap in that test's coverage, say so in the PR instead of marking it. From reading it, it looks solid.

Related

Tickets: #374
Tickets: #376

Discussion

  • Anonymous

    Anonymous - 2026-08-13

    Originally posted by: jasperdingg

    I'll give this a try, will open a PR soon.

     
  • Anonymous

    Anonymous - 2026-08-13
     
  • Anonymous

    Anonymous - 2026-08-15

    Ticket changed by: fu351

    • status: open --> closed
     

Log in to post a comment.