Menu

#45 Harden mneme-check package install step

open
nobody
enhancement (5)
2026-05-22
2026-05-12
Anonymous
No

Originally created by: TheoV823

Context

The mneme-check GitHub Actions workflow installs the local Mneme package before running governance checks:

- name: Install local mneme package
  run: pip install -e mneme-project-memory

If this step fails due to a transient network, PyPI, resolver, or environment issue, the workflow fails before Mneme can run. That makes it hard to distinguish a real governance verdict from CI setup failure.

Desired improvement

Make the install step more robust while keeping failures visible.

Possible approach:

  • Wrap pip install -e mneme-project-memory in a small retry loop.
  • Keep the final failure as a real workflow failure or explicit governance-not-run state.
  • Do not silently mark governance as passed.
  • If feasible, post or preserve a PR comment that says Mneme governance could not run because package installation failed.

Non-goals

  • Do not weaken Mneme verdict semantics.
  • Do not convert install failure into a green governance check.
  • Do not change Retriever, Evaluator, benchmark, or enforcement logic.

Acceptance criteria

  • Transient install failures are retried.
  • Persistent install failures are clearly reported as setup failures, not Mneme verdict failures.
  • PR readers can tell whether Mneme actually ran.
  • Existing warn-mode governance behavior remains unchanged.

Priority

Small CI reliability improvement. Separate from product/core retrieval scoping.

Related

Tickets: #45

Discussion

  • Anonymous

    Anonymous - 2026-05-12

    Originally posted by: TheoV823

    Additional CI hardening case found from PR [#43] failure emails.

    The package install step succeeded. The job failed earlier than Mneme execution during Compute changed files (filtered):

    fatal: 547d70fac56d38858120f4505ce7c5ddb6ad42a5...HEAD: no merge base
    

    So this was not a Mneme verdict failure and not a pip install failure. The workflow checked out a PR merge ref and then attempted:

    git diff --name-only --diff-filter=ACMR "$base_sha"...HEAD
    

    but the shallow/ref state did not provide a valid merge base.

    Suggested fix options:

    • Use GitHub PR SHAs directly where possible:
    • github.event.pull_request.base.sha
    • github.event.pull_request.head.sha
    • Or explicitly fetch both base and head SHAs before diffing.
    • Ensure changed-file computation failures are reported as CI setup failures, not governance verdict failures.
    • Ensure the PR comment/output makes clear whether Mneme actually ran.

    This fits the same issue because it is workflow hardening around distinguishing CI setup failures from actual Mneme governance outcomes.

     

    Related

    Tickets: #43

  • Anonymous

    Anonymous - 2026-05-13

    Originally posted by: TheoV823

    Additional CI reporting issue found from PR [#50].

    The workflow completed successfully, but the sticky comment showed every checked file as UNKNOWN:

    Summary: 0 pass, 0 warn, 0 fail
    UNKNOWN .mneme/project_memory.json
    UNKNOWN docs/adr/ADR-009-explicit-text-encoding-in-automation.md
    UNKNOWN site/...
    

    This is misleading because UNKNOWN is not counted in the summary totals, and the workflow currently only includes command output details for WARN or FAIL verdicts. If the CLI errors, crashes, or emits output without a Result: line, the PR comment hides the underlying error and presents a successful run with 0/0/0 totals.

    Suggested workflow hardening:

    • Track an explicit unknown counter.
    • Include UNKNOWN in the summary, e.g. 0 pass, 0 warn, 0 fail, 65 unknown.
    • Add details blocks for UNKNOWN, not only WARN/FAIL, so CLI stderr/stdout is visible.
    • Consider making any UNKNOWN result a workflow failure or at least a clearly marked governance-not-run/invalid-output state.
    • Preserve the raw CLI output when Result: cannot be parsed.

    This fits [#45] because it is another case where CI setup/reporting hides whether Mneme actually ran and produced a valid governance verdict.

     

    Related

    Tickets: #45
    Tickets: #50

  • Anonymous

    Anonymous - 2026-05-22

    Originally posted by: TheoV823

    Root cause confirmed as an undeclared PyYAML runtime dependency in mneme-project-memory/pyproject.toml, not an install-step hardening problem.

    mneme/adr_parser.py does a top-level import yaml. PyYAML was missing from [project.dependencies], so on a fresh CI runner pip install -e mneme-project-memory succeeded but python -m mneme.cli crashed at import time with ModuleNotFoundError: No module named 'yaml'. The CLI never reached check_prompt, never printed Result:, and the workflow defaulted every file to UNKNOWN. Local dev environments masked the gap because PyYAML was installed as a transitive of unrelated tools.

    Diagnosable because PR [#102] surfaced UNKNOWN verdicts with raw CLI output in the sticky PR comment. Without that visibility the failure looked like a clean 0 pass, 0 warn, 0 fail summary.

    Fixed in PR [#124] (squash 4e9fe1f). One-line declarative change; no code paths, governance, benchmark, retrieval, or CLI semantics touched. Verified in a clean venv before merge: install brings in PyYAML 6.0.3, mneme.cli --help exits 0, and mneme check --mode warn against a real in-scope file emits a proper Result: line.

    The next in-scope PR after [#124] will be the in-the-wild confirmation — UNKNOWN should drop to 0.

     

    Related

    Tickets: #102
    Tickets: #124


Log in to post a comment.

Auth0 Logo