Skill Scanner Files

Skill Scanner 2.3.2

This release makes LLM severities stable: the same skill now gets the same LLM prompt on every scan and under every preset, so a stricter preset no longer rates an issue lower than a less strict one.

Changes

• Send the LLM the same prompt for the same skill on every scan and under every preset (#283):
• The untrusted-content delimiter is derived from the wrapped content instead of drawn at random, so two scans of the same skill send the same prompt. Content still cannot close its own delimiter, because the delimiter is a hash of the text that would have to contain it.
• The structured pre-scan context passed to the LLM carries deterministic findings rated MEDIUM or higher only. LOW and INFO package-hygiene findings (license, name format, description length) have preset-specific thresholds, so strict's 512-character description cap added a lead that balanced and permissive did not send, and the model rated unrelated issues differently. Those findings are still reported as before.
• Strict and permissive use the default LLM context budgets (20,000 / 15,000 / 10,000 / 100,000 characters for instruction body / code file / referenced file / total), as low-noise and quiet already did. A preset now changes which findings are reported, not how much of the skill the LLM reads.
• Update the audited locks for certifi 2026.10.19, filelock 4.2.1, packaging 26.3 and virtualenv 21.14.10 (dev only), clearing the pip-audit advisories reported against the 2.3.1 lock. (#277, [#281])
• Run the release evidence scans in a network namespace so the network denial no longer cuts the runner's own connection. (#280)

Behavior notes

Static findings are unchanged. LLM severities and titles can differ from 2.3.1 on some skills because the model reads a different prompt. With --use-llm, permissive now sends at most the default budgets; content over them is excerpted or skipped with the existing INFO LLM_CONTEXT_BUDGET_EXCEEDED note. Models that do not accept temperature (OpenAI reasoning models, newer Claude models) still vary from run to run with an identical prompt; use --llm-consensus-runs or a temperature-0 model when severity must be repeatable.

Validation

[#283] changes only what the LLM analyzer is sent; the static rules are untouched and the PR passed the base-versus-head detection comparison. On f782465 the local release gate passed pre-commit on all files and 6,065 tests (31 skipped, 1 xfailed), the eval benchmark scored all 29 skills at 100% precision and recall, and pip-audit found no known vulnerabilities. With GPT-4.1 at temperature 0 on the 29 eval skills, the number of skills where a stricter preset got a lower LLM severity went from 6 to 0, with no new LLM findings on the benign skills.