#508 ci: new prepare + tag-triggered release pipeline (staging only, for now) + faster CI

[Anonymous]

Originally created by: adityaharishch

A macOS release build takes 72 minutes today. This rewrites the pipeline to
land at ~23 min cold / ~16 min warm, and fixes a correctness bug that has
already published a wrong version once.

Four independent problems, each verified before being fixed:

1. The double compile (the big one)

cargo tree against the real lockfile confirms the daemon's graph is fully
contained in the tray's - but they were built by two separate cargo
invocations, and features unify per-invocation. 54-96 shared crates resolved
with different feature sets, changing their -C metadata hash and forcing
214 crates to compile twice, cascading upward from the bottom of the graph.

The divergence is bidirectional (8 crates resolve wider on the daemon side:
tokio/macros, hyper/http1, serde_json/std, getrandom/std...), which is
what proves no amount of reordering fixes it. The fix is one invocation:

npx tauri build --no-bundle --target <triple> -- -p meridian --bin meridian

Note this is not "split them into separate jobs" - that was the first
proposal and it is wrong. Separate target dirs guarantee the daemon's 320
crates build twice and lose the 106 the tray currently reuses.

2. The publish race (a correctness bug, not a speed one)

Run 29748827473 built both architectures and then died on The local branch pre-main is behind the remote one. semantic-release's isBranchUpToDate check
requires the branch HEAD, which is structurally incompatible with building a
pinned SHA; upstream declined to make it configurable (semantic-release#1849).

So semantic-release no longer runs during the build. release-prepare.yml
runs it at branch HEAD in ~60s, pushes the tag and a draft release;
release-build.yml triggers on the tag and only ever attaches assets.

Related: tauri-action's built-in latest.json upload is a lockless
read-modify-write against one shared asset, once per matrix job - two runners
finishing in the same window silently drop each other's platform key. Its own
source concedes it. scripts/compose-updater-manifest.py composes the manifest
once, after every runner is done, and refuses to emit anything it cannot
first prove is complete.

3. Per-arch instead of universal

Two macos matrix jobs (aarch64 / x86_64) rather than one universal build.
The win is not raw compile time - it is that the two notarization waits
overlap instead of queueing behind a lipo join. Also halves DMG size and
deletes the join job. Existing installs migrate transparently: the updater
resolves its platform key from the running CPU, not from the manifest.

4. Cache

Keys were fragmented three ways, release.yml had a bare rust-cache, and
restore falls back only to the default branch (so main could never read
pre-main's). 8.53 GB of a 10 GB quota, actively evicting.

---

CI speedups (third commit)

• windows-portability ran cargo check --all-targets and cargo test over
  the same packages. Compiling is strictly stronger than checking, so the check
  could only fail where the test build was about to - a full extra graph
  traversal on the slowest runner we have.
• Named cache lineages (ci-linux/ci-windows/ci-macos) instead of job-id
  keys, and keep ~/.cargo/registry/src rather than re-unpacking ~1000 crate
  tarballs per job.
• CARGO_INCREMENTAL=0 - incremental state is only ever written on CI, never
  read back, and it inflates the uploaded cache tarball.

---

What is NOT done, and please do not merge to main before it is

Nothing here has ever executed. Static validation only (actionlint,
py_compile, an end-to-end composer test with hand-made fragments). I would
put first-try success at maybe 30-40%.

Two things a reviewer should know:

1. The public download link will 404 on the first production release.
   Per-arch builds cannot share Meridian.dmg, so the assets are
   Meridian-aarch64.dmg / Meridian-x64.dmg. The download page needs two
   links before this reaches main.
2. There is no production fallback once merged - release.yml is deleted
   by this PR.

Installed apps cannot be bricked by a bad build: minisign signature
verification happens before an update is applied.

Suggested path: merge to pre-main, cut two clean [staging-release]
builds end to end, and only then consider main.

Test plan

• [ ] CI green on this PR
• [ ] workflow_dispatch release-prepare.yml with dry_run: true
• [ ] A real [staging-release] run, twice, verifying both DMGs, both platform
  keys in the merged latest.json, and an actual staging auto-update
• [ ] Confirm secrets.GH_TOKEN can push tags (see settings note below)

secrets.GH_TOKEN being a PAT is load-bearing, not a nicety: GitHub does
not re-trigger workflows for refs pushed with the default GITHUB_TOKEN.
A tag pushed with it would succeed and then nothing would build.

[Anonymous]

Originally posted by: coderabbitai[bot]

[!IMPORTANT]

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: d68d224c-b96d-45e6-99da-3920ed26cbc4

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:
- [ ] 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

- [ ] Create PR with unit tests - [ ] Commit unit tests in branch `ci/release-pipeline-clean`

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

- [X](https://twitter.com/intent/tweet?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A&url=https%3A//coderabbit.ai) - [Mastodon](https://mastodon.social/share?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A%20https%3A%2F%2Fcoderabbit.ai) - [Reddit](https://www.reddit.com/submit?title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&text=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code.%20Check%20it%20out%3A%20https%3A//coderabbit.ai) - [LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcoderabbit.ai&mini=true&title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&summary=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code)

_{Comment @coderabbitai help to get the list of available commands.}

[Anonymous]

Originally posted by: adityaharishch

Update — scoped to staging only, production left untouched

Per the ask ("do not change the main pipeline, first test only the staging pipeline if it's become faster"), this no longer touches production:

• release.yml (production, main) is byte-for-byte pre-main's current state — including the recent [#499] parallelization. Restored, not deleted.
• The new two-workflow pipeline is gated to staging: release-prepare.yml triggers on pre-main only (drops the [release] marker); release-build.yml's tag filter is v*-staging*, not v* — so a production tag (v1.73.0) fires only the old release.yml and the two pipelines can never collide.
• release-staging.yml stays deleted — the new pipeline replaces it.
• The channel-derivation logic already handles stable; it's just currently unreachable behind the tag filter.

Migration path once staging proves faster: widen the build tag filter to v* and re-add main to prepare — one change, both in the same commit.

Also: merged latest pre-main in, so this is now conflict-free. And #506 (PostHog) was closed — [#505] merged the identical fix directly.

How to test staging (nothing production touched)

1. Merge this to pre-main.
2. Push a [staging-release] commit to pre-main (or workflow_dispatch → release-prepare with dry_run: true first).
3. Watch release-prepare tag v*-staging.N in ~60s, then release-build fan out per-arch. Time it against a recent staging run.
4. Verify both DMGs, both platform keys in the merged latest.json, and a real staging auto-update.

Production keeps shipping through release.yml the entire time.

[Anonymous]

Originally posted by: Akarsh-Hegde

Review

Read the full diff plus the actual release-prepare.yml / release-build.yml / compose-updater-manifest.py / package-updater.sh / bootstrap.sh / both Cargo.tomls / tray/package.json and the deleted release.yml + release-staging.yml. The architecture is right — split prepare (branch-HEAD, ~60s) from tag-triggered build (immutable ref) is the correct fix for the isBranchUpToDate vs pinned-SHA conflict, draft-first + compose-once + completeness-assert genuinely removes the tauri-action manifest race, and the reasoning is unusually well documented. But there are three things I'd treat as blockers, and they line up with your own "nothing here has ever executed" caveat.

Blockers

1. The single cargo invocation almost certainly does not build the tray at all. release-build.yml — npx tauri build --no-bundle --target X -- -p meridian --bin meridian, run from working-directory: tray.
meridian (daemon, .) and meridian-tray (tray/src-tauri) are in one shared workspace (root Cargo.toml members) — I confirmed this. Tauri appends everything after -- to its cargo call and does not itself inject -p meridian-tray. So the effective command is cargo build … -p meridian --bin meridian, and -p meridian selects only the daemon package — the tray is excluded, and --bin meridian further restricts to the daemon bin. The subsequent tauri bundle step does zero compilation (your own comment says so), so it then has no meridian-tray binary to package. Under every interpretation this command builds the daemon or errors on --features mismatch; none of them produce the tray. Your "must print nothing" uniq -d check only catches double compiles, not a missing tray. Concrete test before anything else: from tray/src-tauri, run cargo build -p meridian --bin meridian and check whether target/*/release/meridian-tray exists. If it doesn't, every build job is broken — this blocks staging too, not just main. To build both in one invocation you likely want --bin meridian-tray --bin meridian (both bins, one workspace resolve) rather than -p meridian --bin meridian.

2. Stable tags will push but never build — [skip ci] on the release commit suppresses the tag trigger. .releaserc.json keeps @semantic-release/git with "message": "chore(release): ${version} [skip ci]…". semantic-release tags that commit; release-build.yml fires on push: tags: [v*]; GitHub evaluates skip directives against the tagged commit's message on a tag push and skips the workflow. Net on stable: tag appears, nothing builds. Staging is unaffected (.releaserc.staging.json has no @semantic-release/git, so no [skip ci] commit and the tag points at the real HEAD). Verify on a throwaway tag; if confirmed, move [skip ci] out of the release commit message or trigger the build off release created instead of the tag push. Not blocking the pre-main path, but a hard blocker before main.

3. bootstrap.sh — the primary curl | bash macOS installer — will 404 on stable. scripts/bootstrap.sh:30 downloads releases/latest/download/Meridian.dmg. Per-arch builds only ever produce Meridian-aarch64.dmg / Meridian-x64.dmg (package-updater.sh emits the plain Meridian.dmg name only in the universal-apple-darwin branch, which no longer runs). README.md:72 and SETUP.md:13 point at the same Meridian.dmg. This is a broken install path + broken docs, not just "the download page needs two links" — the installer script needs an arch-detect (uname -m → aarch64/x64) before this reaches main. bootstrap.ps1 is fine.

Major

4. Release builds never warm from cache — the branch save-if clauses are dead code. Both macOS (shared-key: macos-release-${target}) and Windows jobs gate save-if on refs/heads/main || refs/heads/pre-main || startsWith(github.ref,'refs/tags/v'). But release-build.yml only ever runs on a v* tag (or dispatch), so the two branch clauses are never true — only the tag clause fires, and a cache saved under refs/tags/vX is scoped to that ref and unreadable by the next release's refs/tags/vY. Nothing writes these shared-keys on a branch ref, and ci.yml's keys are deliberately disjoint (ci-macos, …), so release builds always cold-miss. That defeats the headline "~16 min warm" goal for the path that matters most. To actually warm, these need to be saved on a branch ref the next tag build can inherit (or reuse the ci-* lineage).

Minor

• add-job-id-key: false is not a documented Swatinem/rust-cache@v2 input (used throughout ci.yml + release-build.yml). Unknown with: keys are silently ignored; the cross-job sharing actually works because shared-key alone replaces the job-id component. Harmless but misleading to a future reader — drop it or confirm against the action's action.yml.
• Staging's fixed-name DMG convenience link goes away. mirror-updater-channel.sh runs in publish on ubuntu-latest, which built nothing, so its DMG glob matches nothing and only latest.json is mirrored to updater-staging. Auto-update still works (manifest URLs point at the versioned release); the releases/download/updater-staging/Meridian*.dmg link testers used is a silent regression from mirror-staging-release.sh.
• create-icons.sh no longer runs (old stable prepareCmd ran it; nothing does now). Fine if the committed icon.icns is current — given the icon-clobber history, confirm the committed icons are the intended spirograph.
• Prepare vs semantic-release draft race (theoretical). On staging, .releaserc.staging.json's @semantic-release/github has draftRelease: true, so semantic-release creates the draft; release-build's prepare job also gh release create --drafts and reuses it. Fine in the normal order; if prepare ever wins, semantic-release 422s. Low probability, worth a guard-comment.

Nit

• Windows "Stage the real daemon" step does cp target/release/meridian.exe …meridian.exe.stub || true — copies the real daemon to a .stub name and does nothing useful before tauri bundle. Harmless leftover.

Verified OK / correcting the record

• The runtime-manifest concern (dropping MERIDIAN_RUNTIME_MANIFEST_URL on staging) is a non-issue — I grepped the whole tree and it has no consumer (only the old release-staging.yml and the local build:staging script set it; no env!/option_env! reads it). It's vestigial after the MLX-runtime retirement, so dropping it changes nothing. MERIDIAN_CHANNEL (which is read via option_env! in version.rs) is set correctly.
• compose-updater-manifest.py is solid: refuses partial/empty/version-mismatched/duplicate fragments and requires all three platform keys, so no partial manifest can publish. With fail-fast: false, a failed matrix leg fails macos and publish is skipped (needs-failed) — nothing partial ships. Manifest-uploaded-last ordering is correct.
• Fragment naming is consistent end to end (updater-<triple>.json written / uploaded / consumed); Windows emits both windows-x86_64 and -nsis keys. import-apple-cert exists; daemon-sign → bundle → notarize-DMG order is preserved for both arches. tauri.staging.conf.json exists.

Bottom line: the design is sound and I'd take it over the old pipeline, but [#1] needs an actual local cargo build check before this runs anywhere, and [#2]/#3 must be fixed before it reaches main. Your suggested path (merge to pre-main, cut two [staging-release] builds end to end first) is the right call — just note [#1] would break those staging builds too, so verify it first.

[Anonymous]

Originally posted by: adityaharishch

CI restructured into a native-runner matrix (+ real Intel, post-merge)

The three hand-written Rust jobs are now one matrix that fans out per native runner, plus the fixes discussed:

when	jobs
every PR (fast)	changes gate → fmt (once) · Linux · Windows · macOS Apple Silicon · UI
push to pre-main/main	all of the above + real Intel tests on macos-13

What changed and why:

• Real Intel testing on a macos-13 runner (arm can't execute x86_64 binaries, so the old arm "Intel slice" was only a compile-check). But — measured live — a macos-13 matrix entry sat queued 25+ min without ever getting a runner (GitHub is retiring Intel runners). Putting that on every PR would make Intel the long pole and kill the speed win. So Intel runs post-merge only (push to pre-main/main), where a long queue is fine; PRs stay fast. Confirmed: the PR run above skips Intel.
• Apple Silicon now tests the daemon+core too (previously only the tray was tested on arm, despite arm being the primary shipped arch).
• dorny/paths-filter gate — a docs/UI-only PR skips all three compile jobs (safe: pre-main is unprotected, so a skipped job just doesn't run).
• fmt runs once off the matrix instead of 3×.
• The double-compile fix (Windows ran check then test — a wasted full build) is in.

Each matrix entry keeps its exact prior per-platform commands (the workspace is asymmetric — the tray is macOS-only), so Linux/Windows behaviour is byte-for-byte preserved; only Intel (check→test) and arm (added daemon) gained coverage.

Note: first run on the new cache keys (ci-<target>) is cold; caches save only on main, so it warms up one run after this merges.

[Anonymous]

Originally posted by: adityaharishch

CI final shape — macOS Apple Silicon only (supersedes the matrix comment above)

Simplified further: Meridian ships macOS arm, so that's the only Rust gate. Windows, Intel (macos-13), and Linux jobs are removed.

Jobs now:

• Rust (macOS Apple Silicon) — fmt + clippy + test, full workspace incl. the macOS-only tray
• Detect changes (paths-filter, ~15s) — skips the compile on docs/UI-only PRs
• UI — popover + UI JS tests
• Migrations append-only — immutable-migration guard (PRs)
• screenpipe MIT pin — license guard

Caching saves on pre-main (a PR restores its base branch's cache), so PRs go warm as soon as the first pre-main run seeds the key — no wait for a release to main. One cold run when this lands, ~4-6 min warm after.

Trade-off, stated plainly: dropping Windows + Intel means those platforms are no longer built/tested in CI. Intel still ships in the universal binary but is unverified here; a Windows-breaking change won't be caught. Accepted — macOS arm is the shipped target.

[Anonymous]

Ticket changed by: adityaharishch

• status: open --> closed