#372 ci(runtime): auto-publish runtime on merge with a version gate (#2/#3)

[Anonymous]

Originally created by: Akarsh-Hegde

Replaces the manual "remember to tag a runtime release" flow — the footgun that left production stuck on a broken 1.66.2 runtime — with merge-triggered publishing, gated so it can never downgrade or silently no-op.

⚠️ MERGE ORDER — merge this LAST

There is no paths: filter, so the moment this lands on pre-main, the next push to pre-main runs the gate, sees pre-main's version (1.64.x) > staging-live (1.63.0), and auto-builds + publishes a staging runtime. If this merges before the others, it would publish a runtime that still contains the WORKLOG_SYSTEM bug, smoke-tested by the old weak smoke script that can't catch it.

Required order on pre-main:

1. #369 — hardened smoke test (walk-imports every agents submodule) — so a broken runtime can't pass smoke.
2. #365 — worklog WORKLOG_SYSTEM fix — so the published runtime isn't broken.
3. this PR — turns on auto-publish.

Expected side effect

Merging this to pre-main triggers an immediate staging runtime build + publish (a long macOS CI job), not a passive config change. That's desirable here (staging is behind at 1.63.0), but it's not silent — flagging so it's not a surprise.

[#2] — auto-publish on merge (build-mlx-runtime.yml)

• New gate job (scripts/runtime-publish-gate.sh), the single source of truth: derives the channel (pre-main → runtime-staging, main → runtime-latest) and decides should_publish by comparing services/pyproject.toml to the channel's live runtime-manifest.json.
• branch push → publish only when local version is strictly > live (no downgrade, no equal-version churn — a merge that lands a stale version just doesn't ship).
• tag push → publish on any difference (the rollback/re-pin escape hatch, unchanged).
• manifest fetch error → fail-closed.
• Self-tested: runtime-publish-gate.sh --self-test (20 assertions).
• build runs only when the gate says publish (or workflow_dispatch for a test build). Two explicit publish jobs; publish-production runs in the production-runtime GitHub Environment → required-reviewer approval, so a main merge can't reach customers unattended. (Environment already created: reviewers Akarsh-Hegde + adityaharishch, main-only.)
• Publish jobs are non-cancellable; build cancels stale in-flight runs. Release upload factored into scripts/publish-runtime-channel.sh (shared by both publish jobs).
• Why no paths: filter: GitHub applies paths: to tag pushes too, which would silently break the runtime-v* rollback hatch. The version gate self-limits to one publish per version-ahead state, so a path filter would be redundant.

[#3] — version-bump PR check (services-version-bump.yml)

• A PR touching services/** must bump the version above its target channel's live runtime (main → runtime-latest, pre-main → runtime-staging), via scripts/check-runtime-version-bump.sh. Catches the silent no-op before merge.
• Per-channel (not max(both channels)) on purpose: pre-main legitimately trails main under set-version lockstep, so max() would make every staging services PR unmergeable. Promotion safety still holds — a pre-main → main promotion is a PR with base = main, so it's checked against production-live and fails loudly on a stale version.

Verification (local)

• runtime-publish-gate.sh --self-test: 20/20 (strict-> for branches, differ for tags, fail-closed on error, channel mapping incl. app v* tags → no publish).
• CI-mode dry run against the live repo: pre-main → should_publish=true (1.64.0 > 1.63.0); main with a stale 1.64.0 → should_publish=false (would-be downgrade blocked).
• version-bump check: base pre-main passes (1.64.0 > 1.63.0); base main fails loudly (1.64.0 ≯ 1.66.2).
• Both workflows parse; all jobs present; publish-production carries environment: production-runtime.

Minor (non-blocking)

Build artifacts have retention-days: 7. If a production approval ever sits >7 days in the environment gate, the publish job would fail on artifact download. Unlikely; noting it.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CT9wnEUuQeoBeufqTh46JC

[Anonymous]

Originally posted by: coderabbitai[bot]

[!IMPORTANT]

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: b3471458-4a17-4c2b-8048-7acbb4e435a6

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:
- [ ] 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

- [ ] Create PR with unit tests - [ ] Commit unit tests in branch `ci/runtime-auto-publish`

---

_{Comment @coderabbitai help to get the list of available commands.}

[Anonymous]

Originally posted by: Akarsh-Hegde

Update — review finding addressed, CI green ✅

Pushed 9416e434 resolving the one actionable issue from the review.

Fixed (🟠 [#1] — dead self-test): added .github/workflows/runtime-gate-selftest.yml, a pull_request check that runs scripts/runtime-publish-gate.sh --self-test (the decision-table tests for version_gt / decide_publish / channel_for_ref — the downgrade-prevention and fail-closed logic). It was previously wired into no workflow.

• Placement is intentional: a PR check, not a step in the gate job — so it catches regressions before they reach pre-main/main, and a self-test failure can never block a live runtime publish (the gate→publish path stays uncoupled from the test).
• Validated in real CI: the new check ran on this PR and passed (4s) — run.

Left as notes (non-blocking, no code change):

• 🟡 [#2] — if services-version-bump.yml is ever marked a required status check, the paths: services/** filter will make PRs that don't touch services/** hang (GitHub's required-check + paths trap). Keep it non-required, or make it an always-run job that self-skips.
• 🟢 [#3] — fetch_live_version fail-closes permanently if a channel release exists but its runtime-manifest.json asset is missing/corrupt. Correct (never the wrong publish); just a known half-published-release recovery case.

Merge-readiness ✅

• Prerequisites merged on pre-main: #365 (WORKLOG_SYSTEM fix) and #369 (hardened smoke). The "merge this last" ordering is satisfied.
• State: MERGEABLE / CLEAN; checks green.
• On merge to pre-main, the gate will publish: services/pyproject.toml is 1.64.1 > live runtime-staging 1.63.0 → should_publish=true → build → smoke → publish runtime-staging at 1.64.1 (carrying the [#365] worklog fix).

Good to merge.

[Anonymous]

Ticket changed by: Akarsh-Hegde

• status: open --> closed