Claude wrote it in 40 seconds. Production killed it in 4.

Google Trends this week made a gap hard to ignore: Claude AI is leading tool interest, "AI software engineer" is up roughly 250%, and "system design" still owns the sustained search volume — the query that doesn't spike, it just never goes away. Three numbers, one story. People are vibing the demos. Interviewers and on-call rotations are still asking what happens when the system fails.

I write rate limiters and partition keys for a living, so I have a specific, unglamorous answer to "what happens when the system fails," and it is almost never the thing the AI-generated pull request was tested against.

A demo is a system with the hard constraints removed

Ask an AI coding tool for an endpoint and you get something that passes its own tests in about the time it takes to read them. It is genuinely good at this. What it is good at, specifically, is a narrow and flattering slice of production:

• One user. No two requests racing for the same row, no retry storms fighting each other for the same lock.
• One region. Latency is a constant in a demo. In production it is a distribution, and the tail of that distribution is where the incidents live.
• Green tests. The suite passes. Nothing has run for six months against traffic that doesn't look like the traffic you wrote the tests against.
• No scrapers. Nobody is patiently draining your corpus one request under the rate limit, indistinguishable from a legitimate integration for the first thirty seconds.

None of those are bugs in the tool. They're the definition of a demo. A demo is a system with the hard constraints removed, and removing them is exactly what makes it fast to write and fast to watch. The problem is only that "fast to watch" and "survives Monday morning" are unrelated properties, and a green checkmark doesn't tell you which one you're looking at.

Four questions the tests didn't ask

These are the four that keep showing up, across completely unrelated systems, as the actual shape of "it worked in the demo and then it didn't."

Stale reads. You wrote it. The replica disagreed. Every read-your-writes guarantee is a design decision — sync, async, or semi-sync replication are three different products with three different failure modes — and "point reads at a replica" is the default an AI tool reaches for because it is the one that makes the demo simplest, not the one that matches what you promised the client.

Hot partitions. One key ate the whole shard while seven others sat idle. A partition key is a load-balancing decision wearing a schema decision's clothes, and it is invisible in a demo because a demo doesn't have enough data or concurrency for one key to matter more than another. It matters a great deal at the row count where you actually get paged.

Flat rate limits. Best customer blocked; patient scraper stays politely under the cap and keeps going. A limiter that only counts requests treats a retry-storming integration and a determined scraper as the same problem on the first violation, when the only thing that actually tells them apart is what each one does after being told no. I wrote at length about the specific curve — Fibonacci escalation, not exponential — that makes that distinction resolve itself over a 24-hour window without a human reading logs on a Saturday, in Fibonacci lockouts. It is the single most "looks fine in the demo, wrong in production" decision I've shipped.

Failover without fencing. Two leaders after an election. Split brain. A consensus algorithm can hand you a new leader; only a fencing token stops the old one from still believing it's in charge and still accepting writes. This one is particularly unkind to AI-generated code, because the happy-path failover — new leader elected, requests resume — is the part that's easy to generate and easy to demo. The old leader's writes are the part nobody watches for, because in a demo there is no old leader still running.

None of these four show up in a screenshot of a green test suite. All four show up in an incident channel.

Two different optimizations

Vibe coding optimizes for green: ship the feature, pass the test, show the demo, move on. System design optimizes for failure: assume load, assume lag, assume bad keys, assume clients that don't announce themselves as attacks. These aren't opposed so much as orthogonal — you can have code that is fast to write and survives load, but you don't get the second property for free by getting better at the first.

AI multiplies speed. It writes, very quickly, the code a competent engineer would eventually have typed anyway. What it doesn't do is decide the things that only get decided by someone who has been paged for the failure mode before: what unit the rate-limit budget attaches to, which replication mode this particular read path actually needs, whether this key distribution will still be uniform at ten times the row count, whether the failover story has a fencing token or just an election.

Those decisions are judgment, not code, and judgment is exactly the thing a tool trained to produce plausible, working-looking output has no mechanism for acquiring. AI multiplies judgment. It doesn't replace it.

What I'd actually tell a team shipping with these tools

Use them — I do, daily, for the boilerplate and the first draft and the refactor I'd otherwise put off. But change what you review for. Stop reading AI-generated pull requests for correctness on the happy path; the model is usually right about that part, and it's not where the risk is. Read them for the four questions above instead: what does this do under contention, under partial failure, under a client that doesn't behave, at ten times the scale of whatever the test fixture has. If the PR doesn't make you ask those questions, that's not evidence the system is fine — it's evidence nobody has asked them yet.

The scarce skill was never typing code faster. It's knowing what breaks when ten thousand clients show up at once, and reaching for the fix before the on-call phone does. That skill gets more valuable, not less, the faster the code in front of it gets written.