An unstructured interview loop explains about 4% of later job performance. A structured one explains about 18%.
Both figures come from Sackett and colleagues, 2022. An interview loop leaks that signal in 4 places.
The question is designed last. Nobody writes anything down. The loop runs too long. The bar lives in people’s heads.
On this page
Your interview loop rejects engineers at close to random. From the inside it looks fine.
I have run 100+ technical interviews.
The number that should end the argument
Unstructured interviews explain about 4% of later job performance.
Structured ones explain about 18% (Sackett and colleagues, 2022).
4 times more of the outcome is explained when the interview has a structure.
The 4% means this. Interview 2 candidates, pick one, and you mostly measured 3 things.
The interviewer’s mood, the candidate’s nerves, and whether the 2 clicked.
What does a normal interview loop actually measure?
A normal interview loop measures four things, none of them the job. Comfort under observation, and shared vocabulary.
Whoever interviewed most recently, and a view formed in the first 15 minutes.
None of it compares between candidates, and none of it predicts the actual work.
Interview fluency. Comfort being watched while thinking. A real skill, and not the one you are hiring for.
Shared vocabulary. “Eventual consistency” outscores the same idea in plain words.
Recency. Last week’s candidate is compared to this morning’s. The older memory has already flattened.
The first 15 minutes. In a study of 691 real interviews, most interviewers decided between minute 5 and 15.
The rest of the slot then tends to look for support.
Where does an interview loop leak signal?
An interview loop leaks signal in 4 places. Almost every broken loop leaks in all 4.
The question gets designed after the job ad instead of before it. Nobody writes anything down until the debrief.
The interview loop runs 6 rounds when 4 would do. And the bar lives in people’s heads.
Leak 1: the question came after the job ad. Write the 3 to 5 things this person must do in year one first. Then design the question. The scorecard takes an afternoon.
Leak 2: nothing written before the debrief. The room is decided by whoever speaks first.
Scores submitted in writing, alone, fix it at zero cost.
Leak 3: the loop is too long. Rounds 4 to 6 re-measure what rounds 1 to 3 told you. The candidate is 3 weeks in, with 2 other offers.
Leak 4: the bar lives in people’s heads. Ask 3 interviewers what “strong hire” means. You get 3 bars. Calibration on one recorded candidate is the fix.
Why do you never find out about a false negative?
A rejected candidate does not send a report in 18 months explaining what you missed.
The cost of rejecting good people stays invisible. A bad hire is loud and remembered.
Teams therefore add rounds, raise the bar vaguely, and reject more good people.
You see false negatives only when the criteria are written down.
A rejection then traces to a line. You can ask whether that line was fair.
What does fixing the interview loop look like on Monday?
Fixing an interview loop takes one afternoon and one uncomfortable meeting.
Write the 3 to 5 things your next hire does in year one. Add the evidence that would convince you of each.
Delete any round that produces no evidence against that list.
Require written scores before every debrief. No exceptions for senior people.
Run 1 calibration session where the whole panel scores the same candidate. Step 4 is the one teams skip.
Expect panel disagreement to go up at first. That is the process working.
Common questions
How many interview rounds should an engineering loop have?
An engineering loop needs 4 rounds for almost every role. A screen. A coding or practical round. A design or depth round. A hiring manager conversation. Rounds 5 and 6 re-measure what the first 3 showed. Meanwhile the candidate collects offers elsewhere. If 4 rounds cannot decide it, the criteria are unclear.
Does structure work for senior and staff hires too?
Yes. The scorecard keeps its shape and the lines change. At staff the evidence is influence without authority. Choosing what not to build. Handling ambiguity. Those are harder to write down than coding ability. Without a written line, a senior loop scores impressions.
What is the smallest change worth making first?
The smallest change worth making is written scores submitted before the debrief. It costs nothing and needs no new tooling. It stops the loudest person in the room setting the outcome. The effect shows in the first debrief. Disagreement is on paper before anyone in the room speaks.
How do you fix an engineering team overwhelmed with interview loops?
Cut rounds first. Four rounds cover almost every role, so rounds 5 and 6 go. Give each interviewer one signal from the scorecard. Then no two people test the same thing. Collect written scores before the debrief, so the meeting only compares notes. If 4 rounds still cannot decide, the criteria are unclear.
Why are panel interviews used?
Panels exist to average out one interviewer’s bias. Panels only do that when each person owns a different signal. Four people asking the same questions produces 1 opinion, 4 times. Panels also push nervous candidates below their level. Two interviewers per round, with separate assignments, gathers more evidence.
The short version
Unstructured interviews explain 4% of performance. Structure takes it to about 18%.
Write the scorecard before the question. Cut the interview loop to 4 rounds. Collect written scores before the debrief.
Calibrate the panel on a real candidate.
None of this needs budget. It needs someone to say out loud that the interview loop is not working.
Want it done with you rather than by you?
I run hiring audits and interviewer calibration as fixed engagements.
Book a free 30-minute call. I will say if your interview loop is already fine.
PDF, 2 pages
The 4 loop leaks: check and fix sheet
The 4 places a loop loses its signal, how to tell if yours does, the fix for each, and the Monday plan in 4 steps.
Leave a work email: the file opens right here, and one hiring stage, taken apart, follows every week. One click to leave.
Read next
The Technical Assessment That Tests the Job
A take-home costs the candidate 6 hours and you 45 minutes of marking. A 60-minute pair costs 1 hour each side and tells you more. Here is the comparison.
When a Take-Home Assignment Works
A take-home costs a candidate a weekend and costs you the ones with children. Here is when it is worth asking.
Want this fixed in your loop?
SmartHiring trains engineering interview panels on their own open roles. Elman Huseynov, a Tech Hiring Expert in Berlin, runs every engagement. One workshop writes the scorecards for your roles, then calibration runs until the panel agrees. It starts with a free 30-minute call.
No pitch if the loop is already fine. See an example audit report.