The bottleneck moved to verification
Something shifted in how I work over the last few months, and it took me a while to name it. The change isn’t really the speed, though the speed is real; it’s that writing the code stopped being the part I worry about. The worry moved downstream, to whether what got written is actually right and whether I have any reliable way to tell. On a good day that reads as progress. On a bad one it is the same problem, relocated to a place I’d been paying less attention to.
Put plainly: the bottleneck has moved from producing code to verifying it. When a model can turn a clear intent into a working-looking implementation in a minute, the minute was never the expensive part. The expensive part is the hour you then do, or fail to do, establishing that the implementation does what you meant and nothing you didn’t. That hour used to be folded invisibly into the writing, because writing something yourself is itself a slow act of verification, you reason about each line as you type it. Take the typing away and that hidden checking goes with it. You have to put it back deliberately, and most of the time we don’t.
It helps to be honest about how little of the week writing code ever was. On any real project it’s maybe a fifth of the job. The rest is working out what to build, designing it, reviewing, debugging, integrating, documenting, and deciding whether the thing is actually done. Every AI coding tool of the last few years optimised that same fifth: they made the writing fast. So the constraint was always going to move to the other four-fifths, and the part of it rising fastest is the checking, because checking is the work that writing used to do for you quietly and for free.
There’s a cleaner way to say it that I’ve started using. AI-assisted work is where the tool speeds you up but you still drive every step. AI-driven work is where you define the task and judge the result, and the execution in between is handed off. As you move from the first to the second, the effort collects at the two ends: saying precisely what good looks like going in, and telling whether you got it coming out. The first end, specification, gets plenty of attention. It’s satisfying, and it feels like the clever part. The second end, verification, gets far less, and it’s the one that scales worst as you produce more. That second end is the bottleneck, and it is mostly unmanned.
The two ends are coupled, which is the real reason front-loading the first one pays. A sharp definition is what verification checks against, and half of verifying is just comparison; you can’t compare against a vague ask, so a precise spec turns “does this feel right” into “does this match”, a far cheaper question to answer. You can also specify the thing to be checkable in the first place: the game’s engine is deterministic because I asked for it to be, which is the only reason a test can pin an exact score. What front-loading can’t do is stand in for the check. A careful spec quietly tempts you to trust it, and that is exactly how the check gets skipped. And some kinds of wrong no spec pre-empts: a mechanic I’d written into the game’s spec was built exactly as described and simply wasn’t any good, which only playing it ever told me. Define what you can, and still verify what you must.
Two things make it harder than the old “just review the diff” answer accounts for.
The first is volume. Code arrives faster than anyone reviews it carefully, and “looks reasonable” quietly becomes the bar. A diff that would have taken you an afternoon to write takes ten minutes to skim, and the skim feels like review because it has the shape of review. The second is the competence gap. I shipped a game recently in a language I don’t really write, and reading the implementation was mostly closed to me: I could not meaningfully audit hundreds of tests of TypeScript I could not have produced. Reading the diff, my usual way of trusting a change, simply wasn’t available. Between those two forces, the traditional mechanism for trusting code, a human reads it and understands it, is either skipped under load or impossible by competence. Something else has to carry the trust.
The thing that has worked for me is to move the check to the layer I can actually judge from, rather than the layer the code happens to live at.
For anything with observable behaviour, that means verifying the behaviour, not the source. On the game, trust came from a pure engine unit-tested to determinism, from a headless browser that drove the real thing at desktop and phone widths and failed on a single console error, and from me playing it every day. None of that required reading React. It required the behaviour to be observable and cheap to check, which is a property you engineer in on purpose.1 For the output of an LLM, the same instinct shows up as independent grading. When I built evals for an agent, the case that justified the whole approach was one where the agent wrote genuinely good prose around a fabricated answer and an LLM judge scored it 4.25 out of 5, while a dumb deterministic check caught in one line that the structured output was hollow. The judge read the writing and was charmed. The check read the shape and asked the rude question. Verification worked because two layers looked at the same output from angles that fail differently.
A smaller principle sits underneath both, and I’ve come to weight it heavily: a tool that reports its own blind spots is worth more than one that reports only success. The single most useful line in any of the game’s overnight build logs was one where the agent admitted it hadn’t managed to exercise a particular interaction under test because the timing was fiddly to drive headlessly. That admission told me exactly where the verification was thin and where to go and look myself. A green tick that hides its own coverage gaps is how you end up trusting the untested path. Supervision needs the map of what wasn’t checked more than it needs the reassurance that everything passed.
It’s also why, whenever I’ve automated a review or a build to run unattended, the one step I never hand off is the verification at the end. The loops I’ve built will happily plan, execute, and re-check their own work on their own, but a person still makes the final call before anything merges. Automate the execution in the middle; keep the human on the end that decides whether it’s right. The touchpoint you protect is the verifying one, because it is the one the machine is worst at and the one that carries the consequences.
The failure mode to watch for is the one where production races ahead and verification stays flat. That gap, between how much you’re now generating and how much you can actually stand behind, is where the silent failures live. I’ve had a pipeline go green from top to bottom while writing nothing but empty files, because a collector returned a 500 inside a 200-shaped response and every downstream step dutifully processed the emptiness.2 Nothing errored. Everything “passed”. The only reason I caught it is that I eventually looked at the actual output rather than the status. As more of the work is produced by something that always reports success, that habit, looking at the thing itself and not the green light, stops being diligence and becomes the job.
The obvious objection is that none of this is new, that it’s just testing with a fresh coat of urgency. Partly fair. Tests, observability, staged rollouts, code review, these are old tools and I’m not claiming otherwise. What’s changed is the ratio. When you wrote the code, verification was maybe a third of the work and writing was the rest. When the code writes itself, verification is most of what’s left that a person is uniquely good at, and it is the part that doesn’t get cheaper as the models improve. A better model produces plausible-wrong output faster, not less often. So the skill worth compounding is not prompting, which everyone will have, but the judgement to design a system so its correctness is observable and then to actually go and observe it.
I don’t think this is fully solved, on my own work or anywhere I’ve looked. I still lean on the model to fix bugs in code I can’t read, which is a dependency I’m clear-eyed and slightly uneasy about. The method holds only where behaviour is cheap to observe, and there are whole classes of system where it isn’t, where I’d be trusting far more than I could check and would be right to slow down. I don’t have a general answer for those yet. What I have is a change of emphasis: I now spend the effort I saved on writing, and then some, on being able to tell whether what I got is right.
The teams that pull ahead over the next couple of years won’t be the ones that generate code the fastest. Everyone will have that. They’ll be the ones who can still say, with something better than a shrug, that it works.
1 Determinism is the quiet precondition. You can only diff what reproduces, so the engine takes the date as an input and seeds any randomness from it rather than reaching for the clock, which is what lets a test assert an exact score and lets me compare two runs and trust the difference is the code, not the dice.
2 The full story is on the FPL Pulse writeup: a LambdaFunctionSucceeded from Step Functions’ point of view, because the invocation completed and returned a JSON body, with the application-level failure sitting inside that body where the state machine never looks.