In short

• A mistboard release took a median of 5m31s over thirty days, and 148 of 479 attempts failed before reaching production.
• The deploy compiled six chess engines from source on every push: two and a half minutes of every build, for code that changes a few times a year.
• A GitHub Actions workflow now builds them once and publishes them under a hash of their pins and patches. The deploy fetches and verifies them in 19 seconds.
• Push to live went from 5m36s to 2m19s, hosted CI from 215 seconds to 132, and releases killed by a second push from 11 a month to none.

mistboard ships from one branch. Whichever Claude Code session finishes first pushes main, and every push is a production release: a local test gate, hosted CI, a Railway deploy, smoke tests against the live site. On September 22 that happened 35 times.

Problem: a six-minute release, one attempt in three failing

Every release prints one timing line per stage, and every session’s transcript keeps them. Parsing the transcripts for the 30 days to September 22 gave 479 release attempts, 16 a day.

One mark per attempt. Circles are successful releases, crosses are failed attempts, the line is the weekly median.

Stage	Median
Local gate	64 s
Hosted CI	about 3.5 min
Wait for production to serve the commit, after CI	11 s, p90 52 s, worst 35 min
Smokes	25 s
Whole release	5 m 31 s, p90 7 m 48 s

148 of the 479 attempts failed before reaching production, and the reasons are as telling as the times:

Why the attempt died	Attempts
The local gate: a formatting error or a test, found 60 to 90 s in	41
The push rejected, because main moved during the gate	30
Hosted CI red	17
A second push cancelled the CI run, so the release had no verdict	11
The drain for live games failed or had no token	10
A production smoke	4
Output cut short before it said	29

The 11-second wait after CI is the tell. It means the deploy was finishing at about the same moment as CI on a quiet hour, and the 52-second p90 and 35-minute worst case mean it finished after CI on a busy afternoon. Two Railway build logs showed why:

fairy-stockfish-xiangqi    fetch + make            12 s
fairy-stockfish-duck       apply patch + make      13 s
fairy-stockfish-atomic     apply patch + make      11 s
stockfish                  make, downloads nets    25 s
pikafish-jieqi             make                    13 s
pikafish                   make                    40 s
npm run build                                      28 s

The server runs six engine binaries, each pinned to a commit in a .ref file, and the build step compiled all six on every deploy: two and a half minutes of a 5m36s deploy, for code that changes a few times a year. The layer never cached because the build step runs after the source copy, and every push changes the source.

Solution: build the engines once, fetch them by hash

A GitHub Actions workflow compiles the six binaries when a pin or patch changes, verifies each one the way the deploy used to (Fairy-Stockfish answers uci with the variant list; each patched build matches the game kernel’s legal-move count at three positions), and publishes them as release engines-<hash>. The hash covers the pins, the patches and a recipe version:

recipe_hash() {
  {
    echo "recipe-version=$RECIPE_VERSION"
    echo "arch=$ARCH"
    for input in $RECIPE_INPUTS; do
      case "$input" in
        *.ref) echo "$input=$(head -1 "$ROOT/$input" | tr -d '[:space:]')" ;;
        *) echo "$input=$(sha256sum "$ROOT/$input" | cut -c1-64)" ;;
      esac
    done
  } | sha256sum | cut -c1-12
}

The deploy computes the same hash, downloads that release, checks SHA256SUMS, and runs the same verify step. Nothing resolves to “latest”: a bumped pin nobody has published fails the deploy naming the tag it wanted. The binaries are statically linked, so the runner’s libc and the image’s need not match. One test keeps the workflow’s trigger paths, the Railway watch patterns and the deploy step in step with the recipe’s inputs.

Four smaller fixes came out of the same profile:

• A release overtaken by another push follows it. When a newer commit on main contains the pushed one and has its own CI run, the release waits for that run and smokes that deploy, instead of failing and re-running everything.
• Biome runs on staged files at commit time. The pre-commit hook ran only the typechecker; September 22 alone had three format-only commits, each a push, a CI run and a deploy.
• The push gate runs only the web tests the change reaches, via vitest --changed: 6 tests in 4 seconds for a leaf edit, instead of 3,358. Hosted CI still runs everything.
• CI jobs are sized by measurement. One parity test walking 2.8 million positions was 34 seconds of the game suite and moved to its own job. The web, server and Postgres suites are sharded so every job lands between 64 and 98 seconds, and the release passes as soon as the required jobs are green rather than when the run’s reporting jobs finish.

Results

	September	Now
Deploy, push to live	4 m 18 s quiet, 5 m 36 s typical	2 m 19 s
Engine step in the build	about 150 s of compiles	19 s fetch and verify
Hosted CI run	175 to 215 s	132 s
Local gate, web-only change	64 s	under 20 s
Releases killed by a second push	11 a month	0

A web-only release now spends about 20 seconds in the local gate, two and a half minutes in CI with the deploy finishing underneath it, and 25 seconds in smokes. The engine build ran once, 153 seconds on the runner, and the first deploy on the new path downloaded 218 MB and verified all six binaries in 19 seconds.

What is left

GitHub’s runner queue, 0 to 40 seconds per job on the runs I watched, and Railway’s fixed costs: image setup, the web build, pushing and booting the image. Neither is mine to shorten.

The local gate is the gate that runs on the loaded machine. With eight sessions on one laptop and a load average over 100, two tests failed there and nowhere else: one gave a fake engine 200 ms to start, and one spawned a git process per changelog link, 137 of them. Both had passed for weeks on an idle machine. Their budgets are now sized for the busy one.

Thirty of the 148 failures were pushes rejected because main moved while the gate ran. A 20-second gate narrows that window and does not close it; a queue would, and that is the next thing to measure rather than build.

The transcripts already held every number in this post, the same files the peers tooling reads for who is working on what. Every release now writes its own record instead, one JSON line with each stage’s duration and the reason it failed, and npm run release:profile prints the medians, the failure table and the chart above. I’ll run it again at the end of October and post the chart with the drop in it, or without. The recipe, the workflow and the release script are in brianhliou/mistboard: scripts/engine-assets.sh, .github/workflows/build-engines.yml, scripts/release-prod.mjs.