#461 feat(storage): add configurable decision log pruning

[Anonymous]

Originally created by: slegarraga

Closes [#213]

Summary

• Add an operator-initiated decision-log-prune command with optional max-age and max-retained-row policies.
• Delete only resolved decision rows; final verdicts are eligible, while AUTH rows remain protected until they have an explicit approved/denied/executed outcome.
• Keep pruning outside record_decision and leave the append-only policy-drift ledger untouched.
• Preserve redaction-safe CLI output and document the new command and exit codes.
• Pin age boundaries, row-budget retention, pending-AUTH protection, verdict stability, required policy validation, and writer append-only behavior.

Verification

• [x] pytest tests/integration/test_decision_log.py -v — 10 passed
• [x] Targeted suite: test_decision_log.py, test_cli_views.py, test_cli_help.py — 67 passed
• [x] ruff check .
• [x] ruff format --check .
• [x] lint-imports
• [x] Full suite with coverage: 3157 passed, 12 skipped, coverage 90.73% (threshold 80%).
• One unrelated pre-existing/local-environment failure remains isolated to tests/integration/test_real_plugin_install_discovery.py::test_installed_plugin_is_discovered_and_fires_despite_pip_target: the child interpreter cannot import the editable core (ModuleNotFoundError: doberman). It also fails on clean main locally. The remaining full suite is green.

[Anonymous]

Ticket changed by: fu351

• status: open --> closed

[Anonymous]

Ticket changed by: fu351

• status: closed --> open

[Anonymous]

Ticket changed by: fu351

• status: open --> closed

[Anonymous]

Ticket changed by: fu351

• status: closed --> open

[Anonymous]

Originally posted by: fu351

Thanks @slegarraga, the exact-boundary cutoff test is my favourite part, one row at exactly 90 days and one a second older, with the verdict set compared before and after, that's how a prune test should be written. Two things to change before I can land it, both need your tests:

1. decision-log-prune is missing from _DOBERMAN_CONTROL_SUBCOMMANDS in engine/rules/commands.py, so a mediated agent can shell out to doberman decision-log-prune --max-rows 0 and empty the audit log; that rule is an allowlist, not a catch-all. Add it plus a row in test_rule_commands_control_plane.py like the memory prune one.
2. _RESOLVED_DECISIONS_PREDICATE has AND between its two clauses, so a row with final_verdict = 'AUTH' is never eligible even after auth_result is approved or denied, which means the --max-rows budget never holds for anyone using approvals (I seeded a year-old approved AUTH row and it survived --older-than-days 1). It wants OR, plus the mirror of your unresolved-AUTH test asserting a resolved one does go.

After that a rebase onto main clears the one import-line conflict in cli/main.py, and a CHANGELOG bullet under Unreleased would be great. Looking forward to the update!

[Anonymous]

Originally posted by: slegarraga

Thanks for the precise review — updated in f151631.

• Added decision-log-prune to _DOBERMAN_CONTROL_SUBCOMMANDS, with the requested --max-rows 0 control-plane regression.
• Corrected the resolved-row predicate to use OR: non-AUTH verdicts remain final, while AUTH rows become eligible only after an explicit approved, denied, or executed result.
• Added the mirror regression proving a year-old approved AUTH row is deleted; the pending-AUTH preservation test remains green.
• Rebased onto current main and resolved the import-only cli/main.py conflict while retaining both the upstream exclusions import and this PR's prune import.
• Added the requested Unreleased changelog entry and kept the README/CLI docs in sync.

Local verification:

• 162 focused tests passed.
• Ruff check + format, all 4 import-linter contracts, offline links across 31 Markdown files, and git diff --check passed.
• The full suite collected 3,352 tests and reached 91% coverage. Its sole failure was test_real_plugin_install_discovery.py::test_installed_plugin_is_discovered_and_fires_despite_pip_target (temporary environment cannot import doberman); I reproduced the identical ModuleNotFoundError on a clean detached worktree at current origin/main, so it is unrelated to this branch.

Fresh CI is now running on f151631. Thanks again — the approved-AUTH case and the mediated-command gap were both important catches.

[Anonymous]

Originally posted by: vinayjagan03

Seems like all necessary CI tests have passed. Manually inspection of diffs looks fine. Discussed offline with @fu351 and seems okay for merge. Good job :)

[Anonymous]

Ticket changed by: vinayjagan03

• status: open --> closed

[Anonymous]

Ticket changed by: vinayjagan03

• status: closed --> open

[Anonymous]

Ticket changed by: vinayjagan03

• status: open --> closed

[Anonymous]

Originally posted by: fu351

Thanks @slegarraga, this one's in! The boundary test is still my favourite part, one row exactly at the cutoff and one a second older, with the verdict set checked before and after, so an off-by-one or a prune that touches a verdict both go red. One thing I changed after the merge, in [#502]: the resolved predicate listed approved/denied/executed, but the proxy actually persists the auth method name (soft_confirm, local_auth, the approval method) plus blocked and error, so the most common resolved AUTH rows were never prunable. It now treats any non-NULL auth_result as resolved, pending rows still stay, and the --older-than-days help text now says the exact-cutoff row is kept. If you want the next one on the same log, [#201] (level-7) is a live-tail mode for the TUI decision-log browser, it reads the same table you just added pruning to.