#44 Improve Mneme rule retrieval scoping

[Anonymous]

Originally created by: TheoV823

Context

PR [#43] shipped a narrow CI hotfix after site/llms.txt triggered a false-positive deploy_001 governance verdict.

The immediate site issue was resolved by excluding site/llms.txt and site/llms-full.txt from the mneme-check CI file scope. That is acceptable as a hotfix, but it exposed a deeper product-level retrieval limitation.

Root cause

deploy_001 includes a multi-word anti-pattern such as:

"python scripts/deploy_site.py"

The current retrieval behavior appears to tokenize anti-patterns word-by-word. As a result, static content that mentions Python can retrieve a deployment-script rule even when the file is not executable code and has no deploy-script context.

This is noisy because the rule is about running a deployment script from the wrong directory, not about the word Python appearing in documentation or AI-discovery files.

Desired improvements

This issue tracks product improvements to make retrieval more scope-aware and less token-noisy:

1. Phrase matching for multi-word anti-patterns
2. Multi-word anti-patterns should match as exact phrases or ordered n-grams.

3. Example: python scripts/deploy_site.py should only fire when that sequence or equivalent command pattern appears, not when a file merely contains Python.

4. Rule-level applies_to scope

5. Rules should be able to declare governed paths, globs, or file types.

6. Example: deployment-script rules could apply to scripts/*.py, workflow files, or shell commands, but not site/*.txt.

7. Content-type awareness

8. Retrieval should distinguish static content files, docs, HTML, .htaccess, Python, shell, and workflow files.

9. Deployment/code-execution rules should not be retrieved for static content unless explicitly scoped there.

10. Explicit no-scope verdict

11. When no rules apply to a file, output should distinguish this from uncertainty.
12. Prefer something like SKIP — no rules apply rather than an ambiguous UNKNOWN.

Non-goals for this issue

• Do not reopen PR [#43].
• Do not implement this as part of the site hotfix.
• Do not change benchmark behavior casually during the current validation/freeze period.
• Do not weaken enforcement globally to suppress one false positive.

Acceptance criteria

Before implementation, propose an architecture-compatible design that preserves Mneme's deterministic governance model:

• Retrieval remains deterministic and testable.
• Existing benchmark metrics are either preserved or intentionally re-baselined with explanation.
• Scope rules are explicit, reviewable, and auditable.
• Existing freeform governance decisions continue to work.
• New scoping behavior has targeted regression tests, including a fixture where site/llms.txt mentions Python but does not retrieve/fail deploy_001.

Priority

Post-freeze product improvement. Important, but not an emergency fix.