← TAB Forward — Chapter 5: The Silent Seven Nights
Where Chapter 5 left off
Yesterday's chapter ended with a list: a required CI check, error tracking, a test for the purge, a December baseline. It also ended with an admission that the only automated validation RunMirror had was npm run build. The CLAUDE.md file that guides every coding session said so in bold: no test suite, no linter, no formatter.
Today that sentence is gone. This chapter is the diff.
I am going to be more specific than usual, because Fundari, the assessment tool from the last two chapters, scores repositories against the DORA capabilities, and a score has to cite a file. So this is written the way its reports are: capability, evidence, what it costs to run, and a section at the end called what wasn't done, stated rather than hidden.
The release in one picture
Three commits shipped in about four hours: v4.37.0 (the pyramid), v4.37.1 (a Semgrep finding and the platform setup), v4.37.2 (three harness bugs the first real runs exposed). 41 files, about 1,400 lines of test code, 155 distinct checks.
Capability by capability
DORA's capabilities are the things the research says actually predict delivery performance. Here is each one this release touched, with the code that implements it. Where a capability was not touched, it says so.
Test automation
Evidence: tests/unit/ (12 files, 67 tests, Vitest), tests/e2e/journeys/ (35 Playwright tests), tests/e2e/security/ (53 Playwright tests), vitest.config.mjs, tests/e2e/playwright.config.js.
The unit layer covers the pure engines in lib/: session signing and verification, marathon detection, the two-level PR system, course projection and the Riegel exponent, the heat model, split extraction from laps, standard splits and raw streams, system and personal ritual detection, the identity profile, the retention constants, the admin allowlist. Every one of those modules is deterministic with zero network calls (that is a Strava compliance rule, not a testing preference), which is exactly why they test in about a second.
The journey layer is written as a runner would experience the product: land, see the official Strava button, follow the OAuth hand-off and check it asks for read-only scopes; browse the five Guides and open a race page; upload a GPX to the anonymous Mile Planner and get a plan back; read privacy, terms, help, support, the blog; and an opt-in authenticated journey that mints a signed cookie for my own athlete only. Smoke-tagged tests run twice, once on desktop Chrome and once on a Pixel 7 profile, because the phone layout is where things break first.
What the unit tests immediately taught me: the engines convert metres to miles with 0.000621371, not 1/1609.344, so an activity of exactly 3.0 miles classifies as 2.99999 miles and lands in the wrong bucket. That is not a bug anyone will hit with real GPS data, but it is the kind of boundary a test pins and a reader learns from.
Test data management
Evidence: tests/fixtures/activities.js, tests/fixtures/gpx.js, tests/fixtures/strava-csv.js, tests/unit/fixtures.test.js.
This one mattered more here than it would on most apps. RunMirror holds other people's Strava data, and Strava's API agreement forbids re-exposing it. So no fixture can ever be a real athlete's export. Instead there is a seeded generator: a fictional runner in a fictional town, three years of weekday runs, a yearly marathon that gets faster, a Turkey Trot every Thanksgiving, a birthday run every March 3, a dedication written in a description. Same seed, same history, every run. Per-mile splits with an optional fade, Strava-shaped laps, raw time/distance/heart-rate/altitude streams, course-mile profiles, a GPX writer that walks north along a meridian at a chosen pace, and a 103-column Strava CSV in the importer's positional layout.
The fixtures have their own tests. The one I am most pleased with asserts that ordinary weekly runs never land on the same month-and-day three years running, so the ritual detector only finds the rituals that were planted. Getting that right took three attempts, because 2024 is a leap year and shifts every post-February date by one.
Continuous integration
Evidence: .github/workflows/ci.yml, four parallel jobs: lint, unit, build, security.
Chapter 5's CI job built the app. Today's runs ESLint (errors block; the 35 pre-existing warnings from the React Compiler rules are tracked, not hidden), the unit suite, the build, and the static security job. Every push to main and every pull request. Third-party actions pinned to commit SHAs. Total wall time about two minutes.
The first run of the new security job failed. Good. Semgrep's dependabot-missing-cooldown rule pointed out that the Dependabot config from yesterday's PR would propose a package version the day it was published, which is how supply-chain attacks get merged by well-meaning bots. A seven-day cooldown went in as v4.37.1. A finding became a PR within the hour, which is the Chapter 5 rule working on the Chapter 5 code.
Shifting left on security
Evidence: the security job (npm audit at high and above, gitleaks over the commit range, Semgrep with the OWASP Top 10, JavaScript and secrets rule packs, 284 rules); tests/e2e/security/ (the contract); .github/workflows/nightly.yml (OWASP ZAP baseline, weekly); .zap/rules.tsv; next.config.mjs (headers).
I wanted SonarQube and Burp Suite Pro. Neither is free for a private repository, and the repo is staying private for reasons explained in Chapter 5's commit history. Semgrep's community edition and ZAP cover the same ground in CI at zero cost, and a generic scanner cannot know the app-specific things anyway. So the security contract is hand-written against the OWASP categories:
| Category | What the contract asserts |
|---|---|
| A01 Broken access control | Every athlete route (20 of them) answers 401 without a session. A cookie with an id but a forged signature is rejected. Every cron and admin endpoint refuses a missing or wrong bearer. The operator page is a 404, not a 401, so its existence is not revealed. The two endpoints removed for Strava §5.16 stay removed. |
| A07 Authentication | A forged webhook event (wrong subscription id) is refused. The verification handshake needs the right token. The OAuth hand-off goes to strava.com with read-only scopes and the registered callback. |
| A03 Injection / A04 Insecure design | The one anonymous file-accepting route (the Mile Planner upload) answers 400, never 500, to a non-multipart body, a missing file, a missing course, HTML instead of GPX, a track with no timestamps, and a SQL-looking course id, without leaking a stack trace. |
| A05 Security misconfiguration | Every response carries nosniff, frame denial, a referrer policy, a permissions policy, and no powered-by header. Logout clears every session cookie. Dot-files and config files are not served. |
Running the contract against production before the release found two real things. The upload route returned 500 to an empty body because request.formData() throws into the catch-all. And production sent exactly one hardening header, HSTS, which Vercel adds on its own; the app set none. Both fixed in v4.37.0, both now locked by a test. No CSP yet, because the app uses inline styles everywhere and loads Vercel's analytics scripts; that needs a report-only phase first and is listed below.
Continuous delivery and deployment automation
Evidence: .github/workflows/deploy-verify.yml, triggered by Vercel's deployment_status event; Vercel Rolling Releases configured on the project.
Deploys were already automatic: push to main, Vercel builds, the custom domain flips. What was missing was anything checking the result. Now every Vercel preview gets the smoke journeys and the security contract run against its URL, triggered by the deployment event Vercel posts to GitHub. Production is different, and finding out how cost an hour: with rolling releases on, Vercel posts no deployment event for the canary. The first production deploy had one; the two rolling releases after it had none, and the verification workflow simply never ran. Then a second lesson, an hour after the first: every production alias, the branch alias included, is governed by the traffic split, so polling an alias reaches the canary one request in ten. My first attempt happened to get lucky and I believed it. Then a third lesson, from a GitHub runner that polled the alias seventy-two times during a live canary and never once reached it: the split is per client, not per request, so "retry until you land on the new build" is not a strategy either. What finally worked is the thing a visitor's browser does. Vercel leaves a commit status on GitHub whose link ends in the canary's deployment id, and it keeps a visitor on one version with a sticky cookie that names a deployment and a draw. So the workflow reads the status, builds that cookie for the canary with a draw of 0.01, waits until a tiny new endpoint, /api/version, answers with the pushed commit's SHA under it, and runs smoke, security and the full regression pinned that way. No API token needed; Vercel's API refused to mint one from the command line, which is probably the right call on its part.
Why the deployment's own URL matters, rather than the custom domain, is the next capability.
Streamlining change approval
Evidence: project rolling-release config { stages: [10% with approval, 100%] }; the approval API call in the session log; docs/TESTING.md section "Canary".
DORA's research is clear that heavyweight external change approval slows delivery without improving stability, and that the thing that works is peer review plus automation. I am one person, so "peer review" is a hard problem. What I can do is replace me reading the diff again with the new build serving 10% of real traffic while 89 checks run against it and the error log stays quiet.
That is what Rolling Releases do, and it turned out the project already had the entitlement. Every production deploy now builds, takes zero traffic, gets verified on its own URL, then serves 10% of visitors (sticky per visitor, so nobody flips between versions mid-session), and waits. I approve it to 100%, or abort and the previous build takes everything back. Skew Protection was switched on automatically with it, so a visitor's client and server stay on the same build.
v4.37.1 was the first release to go through it. Built at 01:45 UTC, verified on its own URL (87 of 89 checks passed; the two failures were harness bugs, below), sat at 10% for fourteen minutes, approved, complete. The "couple of days" hold that the plan describes is for risky changes; a Dependabot cooldown does not need to bake.
The honest cost: every push to main now stops at 10% until someone approves it. Forget, and 90% of athletes stay on the old build. That is written into the project notes in capital letters.
Monitoring and observability
Evidence: Vercel Web Analytics enabled on the project (vercel project web-analytics runmirror); Speed Insights already mounted; runtime errors filterable by deployment; the engagement page.
This was the embarrassing one. The <Analytics /> component has been in the root layout since May. The project-level switch was never turned on. The query API said so in four words: Web Analytics not found. Five months of a script loading on every page and reporting to nothing. It is on now; data starts today.
The canary needs it. Watching a 10% release means comparing runtime errors and Core Web Vitals between two deployment ids, and that is exactly the filter Vercel offers. Custom events are a paid tier, so there is still no funnel from the Connect button to the first reveal; that stays on the list.
Working in small batches and trunk-based development
Evidence: three commits to main in four hours, each with a version in the commit message and a finding named; no long-lived branch.
v4.37.0 was large (the whole pyramid), and I would split it next time. v4.37.1 and v4.37.2 were each under an hour from finding to production. The second one is the most useful kind of commit: bugs in the test harness itself, found by the harness running for real. Four so far.
- The bypass header I was sending to get past preview protection also asked Vercel to set a cookie, which Vercel does by answering the first request with a 307. Every assertion with redirects disabled saw the 307 instead of the real response.
- With rolling releases on, the custom domain serves mostly the previous build while a canary sits at 10%. Verifying "production" by hitting the custom domain was verifying the wrong thing. The workflow now hits the canary's own URL.
- The OAuth
redirect_uricomes fromNEXT_PUBLIC_BASE_URL, so on a preview URL it namesrunmirror.app. That is correct (Strava only accepts the registered callback); the test was wrong to expect the preview origin. - The npm scripts for the security and regression buckets still named Playwright projects from an earlier draft of the config, which had since moved to tags. Locally I had been typing the tag form by hand, so I never noticed. The very first production verification failed on that, and I misread it as the missing bypass secret until the log said otherwise.
And one more from the platform rather than the code: after five full-suite runs from my laptop in fifteen minutes, Vercel's firewall decided I was a bot and answered every request with a challenge page. Reasonable of it. The bypass header exempts the GitHub runner from that; the config now also caps remote runs at two workers.
Code maintainability
Evidence: eslint.config.mjs; CLAUDE.md commands section; docs/TESTING.md.
Next 16 removed next lint, so ESLint 9 runs directly with the Next core-web-vitals rules. The first run found 21 errors, all pre-existing: nine setState calls inside effects, six raw anchor tags where a Link belongs, four components defined during render. None are regressions, so they run as warnings and the config says, per rule, how many there are and that the rule gets promoted back to an error when its class is cleaned up. The point is that CI blocks on new errors from today, without a cleanup project standing in the way.
Version control
Evidence: every action pinned to a SHA; the bypass secret stored in GitHub Actions, never in the repo; gitleaks on every push.
One wrinkle: gitleaks cannot use Dependabot's read-only token for its pull-request commit lookup, so the step is skipped for dependabot[bot] and the other four jobs still run on those PRs.
What each check costs
| Bucket | Count | Runs | Time |
|---|---|---|---|
| Unit (Vitest) | 67 | every push, locally first | ~1 s |
| Lint + build | 2 jobs | every push | ~2 min |
| Static security (audit, gitleaks, Semgrep) | 284 rules | every push | ~2 min |
| Security contract (Playwright) | 53 | every preview, every canary | ~10 s |
| Journeys, smoke subset | 18 desktop + 18 phone | every preview, every canary | ~1 min |
| Journeys, full | 35 | every production release, weekly with all 100 race pages | ~4 min |
| OWASP ZAP baseline | passive scan + spider | weekly | ~10 min |
What wasn't done, stated rather than hidden
- No Content Security Policy. Inline styles everywhere and third-party analytics scripts. Needs a report-only rollout; ZAP will keep flagging it at WARN until then.
- No required status check. CI is advisory on a private repo without GitHub Pro, which is deferred until roughly February. Rolling releases partly compensate: a bad build reaches 10%, not 100%.
- No test for the purge job itself. The retention constants are pinned, the batching parameters are asserted against the statement timeout, but the nightly purge still has no integration test against a database. That is the one Chapter 5 promised and this chapter did not deliver.
- No error tracking. Runtime errors are visible per deployment in Vercel for about an hour on this plan. Nothing pages anyone. The silent-seven-nights class of failure is narrower now (the route returns a real 500) but not closed.
- No funnel. Pageviews from today; no custom events (paid tier).
- Database change management: untouched. Migrations are still numbered SQL files applied by hand.
- SonarQube, CodeQL, Burp Pro: not used. Paid on a private repo or no free tier. Semgrep and ZAP instead, and the hand-written contract for what scanners cannot know.
- The authenticated journey is opt-in and local. It needs the production cookie secret and my own athlete id, so it never runs in CI.
The sentence that changed
CLAUDE.md used to say: There is no test suite, no linter, and no formatter configured. Don't invent test commands.
It now lists six commands and points at a document that says which bucket runs when. The next assessment will read that file, find the workflows, count the tests, and compute a trend against the October 12 baseline that said thirty-five commits, zero capability-score movement. I expect the trend to move this time. The report will say whether it did.
RunMirror is a Strava-connected reflection tool for runners. Fundari is the assessment tool that scored it. The test plan, with the canary procedure and the analytics inventory, is in the repo as docs/TESTING.md.