SkillForge Files

SkillForge v6.0.0 - From Reviewing Documents to Proving Behavior

A ground-up rework driven by a deep external audit (now shipped in-repo as SKILLFORGE_AUDIT.md). The defining change: SkillForge now runs the skills it creates. Static review panels and self-assigned scores are gone; behavioral evidence is the gate.

The new pipeline

• RED baseline gate: a fresh subagent attempts the target task WITHOUT the skill before anything is written. No observed failure = no skill.
• Fresh-context generation: the writer subagent receives only the spec and baseline failures.
• GREEN execution testing: with-skill runs must clear the recorded baseline failures; trigger testing with holdout queries; blind A/B for improvements.
• Review = lint + one adversarial reviewer: every falsifiable check (pinned model IDs, word budgets, description shape) moved into validate_skill.py; a single refutation-charged reviewer replaces the former unanimous panel.

New capabilities

• Per-skill regression evals: every scaffolded skill ships evals/ (trigger queries + behavioral scenarios); run_skill_evals.py runs them statically (CI-safe) or live via headless claude -p.
• Ecosystem doctor (skillforge_doctor.py): trigger collisions between skills, duplicates, stale file references, token budgets, pinned models - across your whole roster.
• Cross-runtime compile (compile_skill.py): author once, target claude / codex / agentskills with migration notes.
• Friction mining (mine_skill_friction.py --consent): opt-in, purely local analysis of your session transcripts to find skill gaps.
• Skill CI: a GitHub Actions template validating changed skills on every PR.

Fixed foundations

• One shared typed frontmatter parser (the old hand-rolled copies rejected SkillForge's own SKILL.md); SkillForge now validates itself, guaranteed by regression test.
• Discovery indexes the real Claude Code plugin cache (previously 0 files scanned there), dedupes across runtimes, auto-refreshes.
• Word-boundary matching replaced substring scoring ("ai" no longer matches "email"); triage reports honest strong/moderate/weak bands instead of pseudo-confidence percentages.
• SKILL.md cut from 5,049 to ~1,200 words; platform documentation corrected (all 17 Claude Code frontmatter fields, hooks-on-stdin interface).

Advisor rebuilt on Claude Code hooks, privacy-first

• SessionStart surfaces queued suggestions; UserPromptSubmit scores checkpoints inline (<0.3s, capped per session/day). The launchd daemon is removed (installers clean up old plists).
• Personal Context scanning is opt-in with recorded consent, disabled by default; hardcoded GitHub handles removed; credential-looking lines redacted from evidence; installers never touch global files without explicit flags.

Quality

• 258-test suite (up from 16), all passing.
• Independently verified: a fresh-context verifier executed a 26-criterion rubric derived from the audit roadmap - 26/26 PASS.

Breaking: validate-skill.py is now a shim for validate_skill.py; assets/templates/skill-spec-template.xml replaced by the tiered references/specification-template.md; scheduled launchd advising removed; Personal Context requires explicit opt-in.