How to Vet a Developer When You Cannot Read Code

Hiring an engineer when you cannot read code is an unusual purchase. You are buying something you have no way to inspect, from someone whose claims you have no way to verify, at a price that is among the largest line items in the business.

Most advice for this situation is unhelpful. It tells you to "hire for culture fit" or "trust your gut", which is a polite way of saying you should give up on evaluating the actual work.

You can do considerably better than that. Not by learning to read code, but by evaluating the thing that actually predicts outcomes — judgement — which happens to be visible in conversation if you know what to listen for.

The proxies everyone uses are mostly noise

Before the things that work, the things that do not:

• Years of experience. Ten years can mean ten years of compounding judgement or one year repeated ten times. The number alone does not distinguish them.
• Big-name employers. A recognisable logo tells you the candidate passed that company's hiring bar at some point. It says nothing about what they did there, and large companies contain an enormous range of ability.
• GitHub activity. Plenty of excellent engineers have an empty profile because their good work is in private repositories. Plenty of mediocre ones have a busy one.
• Confidence. This is the dangerous one. In an interview with a non-technical buyer, confidence is nearly free to produce and reliably mistaken for competence. The strongest engineers are usually the ones qualifying their answers, because they know where the risk is.
• Brainteasers. They test whether someone has seen the puzzle before.

The common failure is that all of these are cheap for a weak candidate to fake and easy for a strong one to lack.

Ask about a decision, not a technology

The single most useful question you can ask has no technical vocabulary in it:

"Tell me about a technical decision you made that you would make differently now."

Listen for three things.

Did they weigh something against something else? Every real engineering decision is a tradeoff: faster to build against cheaper to run, simpler now against flexible later. A candidate who describes a past decision as obviously correct either was not making a real decision or was not thinking about it. The phrase you want to hear is some version of "we chose X, which cost us Y, and we accepted that because Z."

Can they say what they got wrong? Senior engineers have a long list of things they would do differently, and they discuss them without embarrassment. A candidate who cannot produce a single genuine mistake in a decade of work is telling you either that they were never trusted with a consequential decision, or that they do not examine their own work.

Does the story have specifics? "We improved performance" is a claim. "Pages took about four seconds and users were dropping off, we found most of it was one slow query, and it came down to under a second" is a memory. People who did the work remember the numbers and the dead ends. People who watched it happen remember the summary.

You do not need to understand the technology in the answer to evaluate the shape of the reasoning.

Make them explain something to you

Ask the candidate to explain a technical concept from their work — one you do not understand — in a way that you do.

This is not a soft skills test, although it is often described as one. Explaining something simply requires having a real model of it. Someone who understands a system deeply can choose which details to drop. Someone who has only memorised the vocabulary cannot, because they do not know which parts carry the weight.

Two failure modes are informative:

• They hide behind jargon. Watch for an explanation that never touches anything you recognise. Sometimes this is nerves. Often it is a lack of understanding wearing the vocabulary as a costume.
• They condescend. How someone handles a knowledge gap in a low-stakes conversation is a preview of how they will handle it when you are the one asking why something is late.

The best answers usually reach for an analogy and then immediately say where the analogy breaks down. That instinct — being precise about the limits of your own simplification — is what expertise sounds like.

Check references with better questions

Most reference checks are worthless because the questions are worthless. "Was she a good engineer?" invites a polite yes.

Three questions that actually produce information:

1. "Would you hire them again?" Then wait through the pause. The pause is the answer.
2. "What did they need the most help with?" This grants permission to say something negative, which makes an honest answer easy to give. Everyone needs help with something; a reference who cannot name anything is not being candid.
3. "What kind of work would you not give them?" Strong references answer this one happily, because every engineer has a shape. The answer tells you whether your work is that shape.

Ask for a reference who managed them and one who worked alongside them. Peers know things managers do not.

Run a small paid trial, scoped to an outcome

The most reliable signal is a small piece of real work. The mistake non-technical buyers make is scoping it as code, which they then cannot evaluate.

Scope it as an outcome instead:

• Not "build an API endpoint" but "when I submit this form, the enquiry should arrive in my inbox with these fields."
• Not "improve our performance" but "this page takes six seconds to load on my phone; make it faster and tell me what was causing it."
• Not "review our codebase" but "here is a bug our users keep reporting — find out why it happens."

Each of those has a result you can check yourself, without reading a line of anything.

Three rules:

Pay for it. A day or two at their rate. Unpaid work filters out strong candidates first, because they have other options and no shortage of demand. Paying also changes the relationship into a professional engagement, which is what you are actually sampling.

Keep it small. If a trial takes more than a couple of days, you are getting free consulting rather than a signal, and good people will decline.

Leave something ambiguous on purpose. Do not fully specify it. The most valuable thing you will learn is what they do with the gap: whether they ask you a clarifying question, make a reasonable assumption and flag it, or quietly build the wrong thing. That single behaviour predicts more about the working relationship than anything else in the process.

Red flags worth taking seriously

• They propose a solution before asking about your business. Someone who knows what to build before understanding who uses it is selling you what they already know how to make.
• Estimates with no uncertainty attached. "Two weeks" is a guess presented as a fact. "Probably two weeks, three if the payment integration is as undocumented as it looks" is an estimate from someone who has been wrong before and learned from it.
• Every past failure was someone else's fault. Bad teams exist. A candidate who has only ever encountered them is describing the one constant across all of those teams.
• They cannot say what they do not know. Nobody knows everything. "I have not worked with that, here is how I would approach learning it" is a much stronger answer than a confident guess.

Where this fits with the cost of hiring

Every step above takes your time, and that is the actual expense. In our breakdown of what a senior engineering hire really costs, the largest line items are not the ones people budget for — the placement fee, the seventy-odd days the role sits open, and roughly twenty-five hours of interviewer time per hire.

That reframes what a vetting process is for. You are not trying to find the best engineer in the market. You are trying to reduce the chance of an expensive mistake, at a cost in time proportionate to the decision.

Where a pre-vetted network fits

This is the case for using a network like Toptal: the technical filtering has already happened before you meet anyone. Toptal's well-known claim is that it admits roughly the top 3% of applicants — that figure is theirs and self-reported rather than independently audited, so treat it as positioning. What I can say directly is that the screening is real, because I went through it and I am in the network.

The practical consequence for a non-technical buyer is a division of labour that plays to your strengths. Someone else does the part you are badly equipped for — verifying that a person can actually build software. You do the part you are well equipped for, and which no screening process can do for you: deciding whether this person understands your business, asks good questions about your users, and is someone you want to hear from every week.

Where it is still the wrong tool

• For a permanent, core role, hire. A contractor does not accumulate institutional knowledge, and over a multi-year horizon the hourly premium stops making sense.
• Vetting is not fit. No screening tells you whether someone will work well in your domain or with your team. Run the conversation anyway.
• If you can check the work cheaply, you may not need the filter. For small, low-stakes, easily verified tasks, you are paying for screening you could do yourself.

The short version

1. Evaluate judgement, not vocabulary. Ask about a decision they would make differently now, and listen for a tradeoff, a genuine mistake, and specifics.
2. Make them explain something to you. Clear explanation requires a real model; jargon often conceals the absence of one.
3. Ask references what the person needed help with, and what work you would not give them.
4. Run a small paid trial scoped to an outcome you can verify, and leave one thing deliberately ambiguous.
5. Decide what you are optimising — the cheapest hire, the fastest start, or the lowest chance of an expensive mistake. They are not the same process.