Verify, do not trust: a working rule for AI-assisted development

The failure mode nobody wants to admit

A team adopts AI-assisted development. Output volume goes up. Confidence goes up. Then a feature ships that is plausible at every layer except the one that mattered, and somebody senior spends a week working out why.

I have watched this pattern on more than one engagement now. The code compiles. The tests pass, because the tests were written by the same session that wrote the code. The pull request reads cleanly. And the integration point is wrong, because the model never actually understood the boundary it was crossing.

This is the failure mode that quietly burns credibility. Not a dramatic outage, just a slow accumulation of work that looked done and wasn't.

Two rules

I have ended up with two rules I apply to every AI-assisted session, my own and other people's.

The first is that every output is a draft. Not a candidate for the codebase. A draft. The same status you would give a junior's first pass at an unfamiliar problem. Useful, often correct in shape, not yet trusted.

The second is that every session has a brief and a verifier, both defined before the session starts. The brief is what you are trying to produce. The verifier is the person, or the test, or the procedure, that will tell you whether the output is right.

If you cannot describe the verifier before you start, do not start.

That is the load-bearing line. Everything else in this post is a consequence of it.

What a verifier actually is

A verifier is not a vibe check. It is something specific enough that you can point at it and say "this told me the output was correct" or "this told me it wasn't".

In practice, a verifier is one of a small number of things.

A person who knows the domain. Not a generalist reviewer, somebody who can read the output and tell you immediately whether it matches reality. On a recent education-sector engagement, the verifier for anything touching the e-portfolio platform was the one teammate who had actually worked with that platform's quirks. Without her in the loop, an AI-generated integration plan was just fiction with good grammar.

A test that exercises the real behaviour. Not a test the model wrote alongside the code (those are tautologies), but a test written from the brief, ideally before the implementation. If the test is meaningful, the model can write the implementation and you still have signal.

A side-by-side against an existing system. When you are replacing or wrapping legacy behaviour, the verifier can be the legacy system itself. Run the same input through both. Compare. The legacy system is, by definition, ground truth for what users currently experience.

A code review with time to do its job. Not a rubber stamp, not a five-minute skim. A review where the reviewer has actually held the relevant context in their head and can challenge the design. This is the most expensive verifier and the easiest one to fake.

If your session has none of these, you do not have a verifier. You have a hope.

A concrete moment

On the engagement I mentioned, I had Claude Code running parallel research agents across three unfamiliar third-party platforms. A legacy VLE on one side, an e-portfolio platform on another, an internal LMS in the middle. Discovery was driven heavily through transcripts, design screenshots, and synthesised summaries.

This worked extraordinarily well for the first eighty per cent. By the end of week one we had a runnable Vue.js front end and a matching C# back end in front of the team. The proof of concept became the MVP with no rebuild.

It also produced output that was confidently, plausibly wrong in specific places.

One example: an integration pattern that assumed a clean handoff between two of the third-party platforms, where in reality the boundary was fuzzy and partly handled inside a third system. The AI could not reliably tell where one platform ended and the next began without being shown. The output read as authoritative. It cited the right vocabulary. It was wrong.

The verifier in that case was me, sitting with somebody from the partner organisation who had lived with the actual integration for years. Five minutes of conversation killed an architecture that, on paper, looked finished. That five minutes was the entire value of the verification step. Without it, the team would have built against the wrong model of the world and discovered the problem in production.

The lesson I took from that, and have since repeated to myself: when AI is operating across a boundary it has not seen first-hand, treat every output as suspect at the boundary specifically. The interior of each system is usually fine. The seams are where it invents.

Where else AI was concretely wrong

I want to be honest about this, because the AI-assisted development conversation tends to oscillate between cheerleading and dismissal, and neither is useful.

On that same codebase, AI generated more component variations than the design actually required. Three flavours of card, two flavours of table row, a dialog that overlapped with an existing dialog. The model defaulted to producing options because it had been asked to produce. A human designer would have asked "do we need this?" before drawing it.

It also proposed integration patterns that did not survive contact with the third-party platforms. Patterns that work for a generic OAuth flow do not work for a SAML SSO into a legacy VLE with its own session quirks. The model knew the textbook answer. The codebase did not live in the textbook.

These were not catastrophic failures. They were time taxes. Each one cost an hour or a day to identify and correct, and the cumulative cost only stays bounded if you have verifiers in place to catch them early.

Teaching this to a team without slowing them down

The team I was working with was predominantly junior. Mid-engagement they adopted Claude Code internally, which meant a portion of my time went into mentoring on AI-assisted practice alongside the architecture work.

The temptation, when teaching juniors about AI verification, is to give them a checklist. Don't trust outputs. Always review. Write tests first. The checklist is fine and they will mostly ignore it, because checklists feel theoretical until you have been bitten.

What worked better was showing them where AI had been wrong on their own codebase. Not a hypothetical, not a blog post (sorry), the actual diff where a generated integration pattern had to be torn out and replaced. Once a junior has seen one of those, the lesson sticks. They start asking, unprompted, "what's the verifier for this?" because they have an internal picture of what happens when there isn't one.

The other thing that helped was making the brief and the verifier visible at the start of any AI-assisted task. Not a heavy ceremony. A sentence or two in the ticket, or said out loud before the session: "I am trying to produce X. I will know it is right because Y." If you cannot finish that sentence, you are not ready to start.

For juniors specifically, this turns out to be a generally useful habit. It is the same habit that makes someone a good engineer regardless of AI involvement. AI just makes the cost of skipping it more visible, faster.

What this isn't

This is not an argument for distrust. I am, in practice, an enthusiastic user of AI-assisted development. The engagement I have been describing went from greenfield to runnable MVP in a week. That happened because of AI, not in spite of it.

It is also not an argument for heavy process. I am not asking anybody to write a verification plan document. The whole point of the brief-and-verifier rule is that it fits in a sentence.

What this is, is a recognition that AI tools produce outputs at a speed that outruns human attention by default. The only way to keep them useful, rather than impressive-looking and quietly destructive, is to decide in advance how you will know whether each output is correct. Drafts plus verifiers ship. Outputs without verifiers accumulate, and at some point somebody has to pay the bill.

Verify, do not trust. Define the verifier before the session, not after. If you cannot describe it, you are not ready to start, and the most useful thing you can do is stop and work out what would tell you the output is right.

A small invitation

If you are running a team that has recently adopted AI-assisted development, or are about to, and want to talk through what verification looks like in your specific stack, I am happy to have that conversation. No agenda beyond it being useful to you. You can reach me through the contact page on this site.