When to start writing the runbook
Hint: not after launch.
The mistake
The runbook always gets written under duress. Three weeks after launch, when oncall is paging someone for the third time at 2am, and the only person who actually understands the failure mode is the engineer who built the feature and would very much like to be asleep.
By the time the pain is bad enough to justify writing things down, the team is too busy firefighting to write them down well. What gets produced is a rushed, incomplete document, written from memory, missing exactly the detail that would have made the next incident faster to resolve.
The fix
We start the runbook in week one of the build, not after launch. It’s a single markdown file, in the repo, next to the code it describes. There’s no special tool and no separate wiki to keep in sync. Every “oh, by the way” the team learns while building goes in, the moment it’s learned, not the moment someone remembers to write it down. Every undocumented config var, every workaround, every “don’t touch this until X happens first” goes in with a date and a name attached.
A runbook entry during the build looks like this:
## Webhook retries silently drop after 3 attempts
Discovered: during load test, 2026-02-11
The provider SDK caps retries internally and swallows the 4th
failure instead of raising. We wrap it with our own retry count
and alert on the 3rd failure, not the (nonexistent) 4th.
That’s thirty seconds to write while the discovery is fresh. It’s twenty minutes to reconstruct from memory three weeks later, assuming the person who discovered it hasn’t left the project.
What it looks like by launch
By launch, the runbook is maybe 60% complete, not because the team stopped adding to it, but because most of what matters got captured during the actual building, when it was cheap to write. The remaining 40% fills in over the first month of real traffic, as production surfaces the failure modes a build phase never could.
By month two, the on-call rotation can use it without paging the senior engineer who wrote the original code. That’s the actual test of whether a runbook earned its keep: can someone who wasn’t there for the build resolve a known issue without pulling in someone who was.
When it graduates
Once a runbook stabilizes, usually a few months post-launch, it’s a candidate to move into whatever permanent operations documentation the team already uses. But it shouldn’t start there. Starting in a wiki means it’s one more tab to remember to open mid-incident. Starting in the repo means it’s one grep away from the code it’s describing, and that’s exactly where it needs to be while it’s still being written.
The actual tradeoff
Writing it during the build feels like overhead you don’t have time for. Not writing it during the build is a tax you pay later, with interest, usually at 2am, usually to whoever is least prepared to pay it.