#319 feat(secrets): detect PyPI API tokens

[Anonymous]

Originally created by: tomatotomata

Slice\n- Repo: doberman-core\n- Feature / Slice: issue [#197] - PyPI API-token secret detection\n- Plan reference: issue [#197]\n\n## What this PR does\n\nAdds one conservative _CREDENTIAL_PATTERNS entry for modern PyPI API tokens, anchored to the documented pypi-AgEIcHlwaS5vcmc prefix and a base64url payload. It leaves every existing pattern and the existing verdict logic unchanged.\n\nThe tests add:\n- positive detection and external-exfiltration blocking\n- a benign pypi-something lookalike\n- redaction checks for the verdict and debug log\n\nCloses [#197].\n\n## Tests added (run in CI)\n- python -m pytest -p no:recording tests/unit/test_rule_secrets.py -q (64 passed locally; the environment has incompatible pytest recording plugins, so the recording plugin was disabled)\n- python -m ruff check --select E,F,I,B,S src/doberman/engine/rules/secrets.py tests/unit/test_rule_secrets.py\n- python -m ruff format --check src/doberman/engine/rules/secrets.py tests/unit/test_rule_secrets.py\n- git diff --check\n\n## Public-release safety (doberman-core only)\n- [x] Contains nothing from the not allowed list: no enterprise/hosted code, no proprietary detection, no customer data, no secrets, no commercial-license code\n- [x] Core still builds/tests/runs with NO enterprise package installed\n\n## Security checklist\n- [x] Fails closed on error / uncertainty\n- [x] No secret, full file, or unredacted prompt logged or committed\n- [x] Any guardrail/learning change is raise-only (no silent loosening)\n- [x] Every BLOCK/AUTH carries reason codes + a human explanation\n- [x] doberman-core does not import doberman_enterprise\n\n## Edge cases covered / Deviations from plan / Risks introduced\n- The pattern intentionally does not match legacy PyPI username/password credentials.\n- The test token is assembled from clearly synthetic pieces so no real credential is committed.\n- Full repository validation remains in CI; local Windows shutdown emits an unrelated native-library access-violation message after the focused tests pass.

[Anonymous]

Originally posted by: fu351

Welcome, and thank you — this is a clean first contribution. A few things you got right that matter: the pypi-AgEIcHlwaS5vcmc prefix is the real macaroon constant (the same literal gitleaks and trufflehog key off), not an invented shape, so it can't drift; you parametrized the existing redaction and exfil-block tests over the new token rather than adding shallow ones, so the fingerprint-not-plaintext guarantee is actually checked for PyPI; and the change is purely additive, so it's raise-only by construction. It reviewed clean, mutation check included.

One honest heads-up, and it's not about your code: [#197] was already assigned to another contributor who'd claimed it. Sorting that out is on our side, not yours, and a maintainer will settle the assignment before anything merges. Your work here stands regardless — I don't want the overlap to read as a discouragement.

A small follow-up idea (not a blocker): most scanners bound this token at {50,} rather than {20,}. Real PyPI tokens run 100+ characters, so tightening it slightly trims an already-tiny false-positive surface.

There's a welcoming crowd in the Discord if you'd like to hang out: https://discord.gg/Sfy5XGNqty Glad you're here.

[Anonymous]

Ticket changed by: fu351

• status: open --> closed

[Anonymous]

Originally posted by: fu351

Merged — thank you again, and congrats on your first landed PR here. If you're up for another, #241 (a labeled detection corpus + FPR/TPR benchmark harness, level-5) is a natural next step from secret-pattern work. Want it?