Menu ▾ ▴

#150 fix(proof): proof.json was frozen — every write failed and every command returned 0 (DIVE-1888)

closed
nobody
None
2026-07-25
2026-07-25
Anonymous
No

Originally created by: lodar

Closes DIVE-1888.

The bug

No write to ${STATE_DIR}/proof.json had landed since the badge went live — two weeks of a config describing a publisher that no longer exists (user: root, hour: 9, the 09:00 root cron deleted under DIVE-1865). Five write sites, failing two different silent ways:

  • lastPublished stamp: guarded by [ -w "$(dirname "$f")" ] || [ -w "$f" ]. False here, so the write was skipped entirely — no error, no log line, nothing. A guard intended to be defensive instead erased the evidence.
  • The other four: unguarded, so the redirection error did escape — into the publisher's own log, one proof.json.tmp: Permission denied per publish across consecutive days — and then || true returned 0 anyway.

Root cause is a permission: /var/lib/5dive is drwxr-s--- root:claude (no group write) while the publisher runs as unix user claude.

The trap that made this high priority: 5dive proof on --user=claude runs as root, so it persists fine and reports success — anyone repointing the publisher without fixing perms first would believe they had.

The fix

  1. Permission (applied on the box, not in this diff): chmod g+w /var/lib/5dive/proof.json — the file, deliberately not the directory. /var/lib/5dive also holds agents.json, auth-sessions/, auth-profiles/ and constitution.yaml; group-write on the dir would let all 17 agent users rename or delete any of them. Granting it on the one file is sufficient given the code change below.
  2. _proof_pref_write replaces all five hand-rolled jq … > "$f.tmp" && mv … || true sites. Atomic tmp+rename when the dir is writable; truncate-in-place when only the file is — that fallback is what makes the least-privilege permission above work, and it generalizes to any self-hoster with a locked-down state dir. chown/chmod --reference carries owner+mode onto the replacement so a root-run write cannot strip the group-write bit the non-root publisher depends on (that would silently restore this exact bug).
  3. Fail loud. The lastPublished stamp is unguarded and now fatal — a publish that cannot record that it ran must not return 0, because staleness monitoring (DIVE-1889) reads exactly that field. proof tick no longer sends stderr to /dev/null, or every loud message would die in the cron driver. proof status stops printing last published: never for two different conditions, and --json gains stateWritable. proof on --user=<u> checks writability as <u> and prints the remediation.
  4. Publisher identity in the payload. zero-human.json and every append-only history.jsonl row carry publishedBy: {host, user}; proof.json gains lastPublishedBy. A cliVersion dates an artifact, it does not identify its author — reading it as identity is what sent the DIVE-1865 publisher hunt to a machine that did not exist. Additive-only per the zero-human.json API contract; omitted entirely when unknown, so existing readers are unaffected.

Verification

  • tests/proof_state_persist_unit.sh — 17 assertions: both write mechanisms, the loud non-zero failure when neither is available, mode preservation across a rewrite, the status warning + stateWritable, source-level guards so the [ -w ] skip and the || true swallow cannot come back, and the publishedBy stamp (present when known, omitted when not). The two unwritable cases skip under root, where perms do not apply.
  • Existing proof_cron_user_unit, proof_ledger_unit, proof_publish_gate_unit, proof_publish_unit all pass.
  • End-to-end on a fixture with production-identical perms (drwxr-s--- root:claude dir, root:claude file), driven as claude:
  • mode 0644 → PERSIST FAILED (rc=1) + CANNOT PERSIST STATE … + the exact fix command, and status says state: ⚠ … is NOT writable by claude.
  • after chmod g+w on the file → PERSIST OK, last published: 2026-07-25 (today) by claude@<host>, mode still -rw-rw-r--.
  • Live: proof status --json as claude against the real /var/lib/5dive/proof.json now reports stateWritable: true (it reported false before the chmod).

Notes for the reviewer

  • The old installed binary is still blocked even with the file chmod, because its tmp+rename needs the directory. The unblock requires this code change to reach the box.
  • I also repointed the live proof.json .user from root to claude (truncate-in-place, g+w verified to survive). hour: 9 is left alone: it describes the deleted cron, and the real 02:40 publisher is agent-invoked and not driven by this config. Going forward lastPublishedBy is the authoritative identity.
  • Version stamped 0.14.10 against main at 0.14.9. PR [#147] also wants 0.14.10 and is currently conflicting — whichever merges second rebases the bump.

🤖 Generated with Claude Code

Related

Tickets: #147
Tickets: #163

Discussion

  • Anonymous

    Anonymous - 2026-07-25

    Originally posted by: lodar

    Both review points addressed; rebased onto main (2305f13) and force-pushed.

    1. Version — you were right, mine was stale. Read main HEAD rather than trusting my note: git show origin/main:src/header.sh is 0.14.10, CHANGELOG top is 0.14.10/DIVE-1880/#147. Now 0.14.11, with my entry stacked above main's 0.14.10 rather than replacing it.

    2. The loudness did not land — confirmed, and it was worse than the comment. Verified both halves myself:

    • /var/log/5dive-proof.log is -rw-r--r-- root:root, 0 bytes, mtime Jul 24 09:00 (the dead cron's last silent no-op).
    • Nothing invokes proof tick: no /etc/cron.d/*, no /etc/crontab, no user crontab, no systemd timer. The only reference in the entire source is the cron line _proof_install_cron writes.

    The bit I had missed: the cron's >> $log redirect is opened by the shell as ${user}, before /usr/local/bin/5dive runs at all. So on a non-root publisher the command dies on the redirect, before publishing — it would not even have reached the code I made fatal. Freeing stderr moved the message into a file the runner cannot open; you are right that this is the same shape as the DIVE-1896 monitor.

    Three changes rather than just the comment, because DIVE-1889 is about to walk this path:

    • _proof_tick no longer || trues its result. The exit code now carries the signal even when the log is unreadable (exit 3, already-published, still maps to success — that is a real success for a scheduled run). The old "always return 0 so a miss never spams cron mail" trade is backwards: cron mail is a channel, silence is not.
    • _proof_install_cron proves its log destination while it still has root — creates it, chowns it to the cron user, and warns loudly if it cannot, instead of writing a cron line pointing at a file nobody checked. Your DIVE-1896 preflight rule, applied one layer down.
    • The comment now says plainly that proof tick has no live caller on this box, that its only caller was the cron removed under DIVE-1865, and that the real publisher invokes proof publish directly — so a re-wiring targets the live path, not this one.

    Tests: tests/proof_state_persist_unit.sh is now 23 assertions (+6): tick propagates a publish failure in its exit code, maps 3 to success, stays a no-op when disabled; the cron install creates its log target, the installed cron line redirects to that same proven path, and an unwritable destination warns. Four existing proof suites and the bundle check pass.

    Thanks for the cliVersion note — publishedBy {host, user} is in zero-human.json and every append-only history.jsonl row, so the artifact names its own author from the next publish onward.

     
  • Anonymous

    Anonymous - 2026-07-25

    Ticket changed by: lodar

    • status: open --> closed
     

Log in to post a comment.