#165 fix(deploy): build the Frappe image once, not six times in parallel (#152)

[Anonymous]

Originally created by: grynn-in

What changed

The Frappe + Konsol image is now built once, not six times in parallel, under a tag scoped to the Compose project. Nothing pulls it from Docker Hub, and a backup never builds it.

[!IMPORTANT]
After merging, run ./deploy.sh once before the next scheduled backup. Until then, repo-frappe:latest doesn't exist on the host. A scheduled ./deploy.sh backup (for example the 02:00 cron the deployment guide suggests; the current host has none) would then exit 1 with Frappe image repo-frappe:latest not found. Run ./deploy.sh first; backup does not build. and take no backup. I checked with docker image inspect: the tag is absent on the live host today.

• docker-compose.yml
• The x-frappe-common anchor no longer has a build:. It has one shared image: ${COMPOSE_PROJECT_NAME}-frappe:latest and pull_policy: never. All six Frappe services inherit both.
• Only frappe_backend carries build: (same context ./docker/frappe and FRAPPE_BRANCH arg as before). never only rules out pulls; Compose still builds a service with build:, without first trying docker.io/library/<project>-frappe.
• The five image-only services (frappe_worker, frappe_scheduler, configurator, dbt_init, backup) fail with "no such image" if it is missing, instead of attempting Docker Hub.
• The tag is project-scoped: the project name is the checkout's directory name, normalised (lower-cased, characters other than a-z 0-9 - _ dropped, leading - and _ trimmed), so the live checkout in ./repo gets repo-frappe:latest.
  • A build from a checkout with a different directory name can't overwrite the live tag. Two checkouts both named repo still share it.
  • COMPOSE_PROJECT_NAME is deliberately not pinned. It also prefixes the data volumes (repo_mariadb_data, …), so pinning it would start the stack on empty volumes.
• frappe_backend is in the default profile, so a plain docker compose up -d builds the image when it is missing.
• A comment on the anchor warns against putting build: back there.
• deploy.sh
• Build step: COMPOSE_PARALLEL_LIMIT=1 docker compose --profile setup --profile backup build, with the comment rewritten to explain the one-image design, covering both [#58] (version skew) and [#152] (OOM).
• Why keep COMPOSE_PARALLEL_LIMIT=1: it does nothing while exactly one service builds. If a second build: ever comes back, it serialises the builds with the classic builder, which was verified on Compose 5.1.4 on a host without buildx. It may not apply when Compose builds through buildx/Bake, so the real safeguard is keeping a single build:. It applies to the build command only, so the up steps stay parallel.
• backup) sub-command never builds. It resolves the Frappe image name from config --images backup, filtered to the -frappe:latest tag because that command also lists dependency images. If the name can't be resolved (for example, config failed) or the image isn't present, it exits 1 with Frappe image … not found. Run ./deploy.sh first; backup does not build. An unattended full build next to a running ClickHouse on an 8 GiB host could OOM, and there would be no backup either way. Failing loudly makes the missed backup visible in the cron log.
• docs/admin-guide/deployment-guide.md
• Next to the cron line: backup never builds, and the error line to watch for.
• Upgrading section:
  • It recommends git pull then ./deploy.sh. git pull doesn't update the konsol app in docker/frappe/konsol; deploy.sh clones KONSOL_BRANCH from KONSOL_REPO on the first run, then fetches and hard-resets it on later runs.
  • It lists deploy.sh's six steps: infra up, stage konsol (clone, or fetch and hard-reset), build, configurator (bench migrate), up -d, dbt.
  • A warning box says the first upgrade to the shared image must be a full ./deploy.sh before the next backup.
  • It covers the project-scoped tag (normalisation, same-name caveat, don't pin COMPOSE_PROJECT_NAME), pull_policy: never on all six, the softened COMPOSE_PARALLEL_LIMIT claim, and cleanup of the old per-service images.
• README.md: the Quick Start is now ./deploy.sh. The manual order is spelled out with deploy.sh's own commands:
• up -d infra
• build frappe_backend
• --profile setup run --rm configurator
• up -d
• --profile setup run --rm dbt_init

It notes that the configurator can't build the image, so on a fresh host it must come after the build.

The [#58] fix is kept: every Frappe service runs the same tag, so a rebuild moves all of them together, and docker compose up -d recreates any container whose image changed.

Other consumers checked:

• docker-compose.cluster.yml is ClickHouse-only.
• docker/configurator/init.sh does not reference image names.
• The Makefile's stale --profile init has no services; its build now builds just frappe_backend.
• The CI workflows never build the stack.
• Nothing in the repo references the old <project>-<service> image names.

Verified statically

Only bash -n, docker compose … config and docker image inspect/ls were used against the real Docker.

• bash -n deploy.sh: OK.
• docker compose -f docker-compose.yml --profile setup --profile backup config, no COMPOSE_PROJECT_NAME set, run in a directory named wt-152:
  project name: wt-152 backup image=wt-152-frappe:latest pull_policy=never build=- configurator image=wt-152-frappe:latest pull_policy=never build=- dbt_init image=wt-152-frappe:latest pull_policy=never build=- frappe_backend image=wt-152-frappe:latest pull_policy=never build=context=docker/frappe frappe_scheduler image=wt-152-frappe:latest pull_policy=never build=- frappe_worker image=wt-152-frappe:latest pull_policy=never build=- services with build: ['frappe_backend'] distinct Frappe image names across 6 services: {'wt-152-frappe:latest'}
  With COMPOSE_PROJECT_NAME=repo it resolves to repo-frappe:latest. On main, the same command shows six services with build and no image.
• config --services resolves 9 services (default profile) and 12 (with setup + backup), the same sets as before.
• docker compose -f docker-compose.cluster.yml config -q: valid.
• Backup failure paths, tested on a scratch copy of deploy.sh with a fake docker on PATH that logs every call and never reaches the real daemon:
  --- image missing exit=1 [ERROR] Frappe image repo-frappe:latest not found. Run ./deploy.sh first; backup does not build. calls: compose --profile backup config --images backup; image inspect repo-frappe:latest build called: no --- config fails exit=1 [ERROR] Frappe image <unresolved> not found. Run ./deploy.sh first; backup does not build. calls: compose --profile backup config --images backup build called: no --- image present exit=0 calls: config --images backup; image inspect repo-frappe:latest; compose --profile backup run --rm backup build called: no

Not verified: the build itself, or any container behaviour. A build is what OOM-kills ClickHouse on the dev machine, so no build or deploy was run on purpose.

One slip to disclose from the first commit: I tried docker compose build --print, expecting it to only print the bake definition. The host has no buildx plugin, so Compose ignored --print and started a classic build under a throwaway project name. It hit the layer cache for steps 1–10 and failed at step 11 (COPY konsol), because the worktree has no staged app, so no RUN step executed. No image or container was created or changed. ClickHouse stayed healthy with RestartCount 0.

Needs a real deploy to confirm

• Run ./deploy.sh once after merging, before the next scheduled ./deploy.sh backup, if you run one (see the note at the top).
• ./deploy.sh Step 2 builds exactly one image, repo-frappe:latest: one bench get-app / vite build, not six. It exits 0 without COMPOSE_PARALLEL_LIMIT set in the caller's environment.
• pull_policy: never + build: builds on a real run. frappe_backend builds the image during deploy.sh's build step and on a plain docker compose up -d when the image is missing, and never reports a pull.
• No Docker Hub pull is attempted for the Frappe image at any step, by any service. There should be no pull access denied … frappe in the output.
• ClickHouse RestartCount does not rise during the deploy, and dmesg in the VM shows no OOM kill.
• docker compose up -d in Step 4 recreates frappe_backend, frappe_worker and frappe_scheduler on repo-frappe:latest. configurator and dbt_init run on it too (docker inspect … --format '{{.Config.Image}}').
• Backup on a host with no image: ./deploy.sh backup exits 1 with the "not found … backup does not build" line and does not build. With the image present, it runs the backup normally.
• Optional cleanup afterwards: remove the stale repo-frappe_backend, repo-frappe_worker and repo-frappe_scheduler images, and any other repo-<service> leftovers.

Closes [#152]

🤖 Generated with Claude Code

https://claude.ai/code/session_013WewQKFQgG7o2M3mUDPRR5

[Anonymous]

Ticket changed by: grynn-in

• status: open --> closed