Every developer uses AI now. Ask “do you use AI to write code” and you get a yes almost every time — which tells you nothing you can act on. What you want to know is what happens between the AI generating something and that something reaching your live system. That gap is where a good developer looks nothing like a careless one — and it is something you can check without reading a line of code.
You don’t need to read code to check this. I’ve spent a lot of my 17-plus years in IT judging whether someone else’s technical decision holds up — long before AI made that call both more necessary and easier to fake. What came out of those years is a shortlist, and it’s yours: five questions, plus the patience to hear a specific answer instead of a comfortable one. Ask them before you sign anything. Ask them again once work starts.
Why “they use AI” stopped being the right question
A few years ago, using AI to write code was worth flagging on its own. That moment has passed. The tools write real, working code quickly, and most developers now use them daily. Refusing to work with anyone who touches AI narrows your options and leaves the actual risk exactly where it was.
The risk was never the AI. It’s what a developer does, or doesn’t do, after the AI produces something. Ungoverned AI use produces code that runs today and breaks in three months, for reasons nobody can explain — because nobody tracked what changed. Governed AI use looks identical from the outside. That is not a failure of your judgment; it is the nature of the thing. The two stay identical until something goes wrong. Then the difference is the whole story.
The five questions
Ask these in order. You are not being tested on technical knowledge here — you are listening. A developer who is doing this well answers all five plainly, without jargon and without defensiveness.
1. What do you personally check before you accept what the AI wrote?
Why it matters: an AI model generates code confidently whether or not it’s correct. Confidence is not a signal you can trust from something that has never once meant “I’m not sure.” Somebody has to be the actual check — a real, describable process you can hear described out loud. Not a vague feeling of “it looks right.”
- A good answer sounds like: something specific — small, reviewable chunks of work, a manual read-through of the actual change before it merges, a defined set of checks that have to pass first.
- A bad answer sounds like: “I read it over,” with nothing further when you ask what that means. Or “the AI is pretty reliable now” — as if reliability were a reason to stop checking.
2. How do you know it is tested, not only that it runs?
Why it matters: code that runs without errors, and code that does the right thing, are two different facts. AI-generated code is especially prone to the first without the second — it’s fluent, not necessarily correct. If you want the longer version, what a passing test actually proves covers it: a green checkmark only tells you the assertions someone wrote ran clean.
- A good answer sounds like: naming the specific behaviors that get checked — the payment going through, the form saving correctly — and being upfront about what isn’t covered yet.
- A bad answer sounds like: “Don’t worry, it’s tested.” That sentence is reassurance, not evidence. A developer with a real answer won’t need to fall back on it.
3. Who checks the work besides you?
Why it matters: if the person who wrote the prompt, generated the code, and ran the tests also decides it’s good enough — that is not independent verification. It’s one person satisfied with their own output, which may be true, and still isn’t proof. A self-approved result looks identical to an independently checked one, which is exactly why you have to ask rather than look.
- A good answer sounds like: a named second reviewer, a defined review step, or a separate check that has to sign off before anything ships — something outside the same head that wrote the prompt.
- A bad answer sounds like: “I’m pretty careful, so it’s fine” — an answer about the person’s character, not about the process.
4. Tell me about the last time the AI got something wrong.
Why it matters: anyone doing real work with AI-assisted development has a story. It happens. A developer who claims it never has is either not paying attention, or not being straight with you. You are testing candor here as much as competence — and candor is something you can already judge in any field.
- A good answer sounds like: a specific, slightly uncomfortable story — what broke, how it was caught, what changed in the process afterward so it doesn’t happen the same way twice.
- A bad answer sounds like: a flat denial, or a generic answer with no detail, delivered a little too smoothly.
5. If the AI wrote most of this, can you explain what it did without saying “the AI handled it”?
Why it matters: this is about ownership, and the ownership in question is yours. If your developer can’t explain the logic in plain terms, the next one won’t be able to either — and there will be a next one, by choice or because the first left. Code nobody can explain is a liability wearing the shape of an asset.
- A good answer sounds like: a plain-language walkthrough of what the system does and why, delivered without hand-waving, even if you don’t follow every technical detail.
- A bad answer sounds like: “it’s complicated,” or a retreat into jargon the moment you ask a follow-up question.
A bad answer is not a lie. It is reassurance standing in for a process that was never built.
Why you can judge this without reading code
None of these five questions require you to know anything about code. They require the judgment you already use when a supplier explains a delay or a candidate explains a gap: the difference between a specific answer and a comfortable one. A developer who has genuinely governed their AI use will answer plainly — sometimes with an example that’s a little embarrassing to tell. One who hasn’t will reach for reassurance instead, and you will hear it.
This matters more, not less, as AI writes a larger share of the code you pay for. Speed was never the problem. Speed with nobody checking is — and that’s as true of AI-written code as it has always been of human-written code. For the fuller picture of what “tested” should mean, read what a passing test proves — and what it doesn’t. For what a properly structured engagement looks like, see how Anito structures ownership from day one.
If you want a second opinion on work you’ve already been handed — code, a quote, a claim that “it’s tested” — the diagnosis is free. You get a straight answer in writing, whether or not you ever hire Anito for anything after that.
Common questions
Should I refuse to hire a developer who uses AI to write code?
Refusing costs you good people and leaves the actual risk exactly where it was, because the risk was never the tool. There is one restriction worth writing down, and it has nothing to do with their keyboard: ask where your code and your customer data go when they use it. Whether anything you send gets retained, or used to train the next version of the model, depends on which account tier they are on — and every major vendor documents that on its own site. So the question to ask is which account they use, and then check the answer against the vendor’s page rather than against their reassurance.
My developer says they review everything the AI writes. How do I check that without reading code?
Pick one change that went live recently and ask them to walk you through it: what the AI produced, what they changed about it, who else looked at it, and what the checks said before it shipped. Then ask the harder one — what was the last thing the AI wrote that you threw away? Someone who genuinely reviews has rejections and remembers them, and the answer arrives with detail because it cost them an afternoon. Ask for the walkthrough in writing rather than in a meeting. A claim behaves differently once somebody has to type it.
Can I find out whether AI wrote the code I already paid for?
Not reliably. Detection tools for code produce a confident-looking number with no way for you to check it, and one careful human edit changes the answer — so anyone quoting you a percentage is guessing. Here is the part that puts you back in control: it stops mattering. What you need to know is whether the thing can be maintained, and that you can check without knowing who typed it. Ask whoever holds it now to explain any one part of it in plain language, ask to see the history of what was changed and when, and ask what happened the last time someone made a small change — whether anything unrelated broke. Those three answers tell you what a detection score never could.
What should our contract say about AI-written code?
Do not try to ban the tool. You would be policing something you cannot see, and paying someone to do the policing. Write down the outputs instead, because those you can hold up and check: a written record of what changed and why, a named human who signs off on each change before it reaches your live system, and the code plus its full history delivered into accounts in your company’s name. The handover clause is the one that quietly decides what leaving costs you later, so read it before you read the pricing, and get the account names spelled out rather than described. A clause you cannot verify without trusting the person it constrains is decoration.