Menu

#248 docs: publish an honest OWASP-LLM / NIST-AI-RMF control-coverage matrix

open
nobody
2026-08-04
2026-08-04
Anonymous
No

Originally created by: fu351

Doberman already enforces most of the runtime controls that AI-security frameworks ask for, but nowhere states which ones — so anyone evaluating it has to reverse-engineer coverage from the source. Publish a coverage matrix that maps Doberman's guarantees to the OWASP Top 10 for LLM Applications and the NIST AI RMF (with a column for the NSA/CISA/NCSC baseline), so a buyer or contributor can see what's covered at a glance.

Build

  • One table per framework. Each row is a framework control; each cell carries a status: covered, partial, or gap — never a bare checkmark.
  • Every covered or partial row cites the feature or reason code that backs the claim (e.g. F8 decision log, ReasonCode.encoded_exfil, tool-schema pinning). A partial row names what's missing in one line. A gap row says whether it's out of Doberman's layer (sandbox / TEE / transport crypto) or on the roadmap.
  • Lives in docs/ and links from the README's security section.

The bar — this needs judgment, not a checkbox pass

  • Honest tags only. Doberman's own rule is defense-in-depth, never "airtight"; a matrix that inflates a partial into a covered is security theater and someone will say so. A visibly-gapped matrix reads as more credible, not less, and doubles as an honest public backlog.
  • No new claims. The matrix documents what ships today — it promises nothing. A roadmap item is gap (planned), never covered.

Derived from a council review of RAND's AI Security Guide (TLA4174-1), which maps its own controls to these same frameworks — reuse that mapping as the row skeleton rather than inventing one.

Related

Tickets: #500

Discussion


Log in to post a comment.