05 Hiring Leadership
Interviewing
Without Guessing
Better hiring starts with better signals.
The evaluation system
Signal
Observable, specific evidence
Evidence
Concrete examples from the interview
Calibration
Panel aligned on what good looks like
Decision
Grounded in evidence, not impression
Guessing is what happens when the interview didn't produce enough evidence. Evidence is a design problem — not a talent problem.
Why this matters
Most hiring guesswork is a process problem, not a people problem.
After most interviews, hiring managers make decisions based on a mixture of impression, instinct, and whatever stayed in memory from the conversation. Sometimes this works. More often it produces inconsistent outcomes — strong candidates declined, weak candidates progressed, and no one can explain precisely why.
Intuition is not worthless in hiring. But it is unreliable as the primary decision mechanism — particularly when the stakes are high, the role is specialist, and the panel is not calibrated. Intuition fills the gaps that structure should have closed.
Structured interviewing is not about removing human judgment from the process. It's about giving human judgment better material to work with: specific evidence, defined signals, and a common evaluation language across the panel.
Founder reality
Assess how much guessing exists in your current hiring process:
After a typical interview, can every panel member articulate three specific pieces of evidence for their recommendation?
When the panel disagrees, do they disagree about evidence — or about impression?
Is there a defined set of signals the panel is evaluating in each interview stage?
Do different interviewers evaluate the same candidate differently — and do we know why?
Have we ever hired someone who seemed strong in the interview and underperformed in the role?
If the answer to the last question is yes, and the team can't explain specifically why the prediction failed, there's a structure gap in the evaluation system.
The framework
Four elements of an interview system that produces confident decisions
Each element reduces the gap between interview impression and actual performance.
01
Signal definition — agree on what good looks like before the interview
For each interview stage, define the specific capabilities being evaluated and what a strong, acceptable, and weak signal for each looks like. This is done before the first candidate is seen — not during or after. Without signal definition, calibration is impossible.
02
Question design — prepare questions that produce the defined signals
Each question maps to a specific signal. Behavioural questions produce past behaviour. Problem-solving questions produce reasoning quality. Role-specific questions produce domain depth. Questions designed without a signal map produce conversation — which is pleasant but not evaluative.
03
Evidence capture — document what the candidate actually said, not impressions
Interviewers who write notes on impression ('seemed sharp', 'good energy') produce unreliable panel debrief input. Interviewers who capture what the candidate specifically said — 'described rebuilding the consensus module from scratch after a production failure' — produce evidence. The note format shapes the debrief quality.
04
Calibration — align the panel on evidence before reaching a decision
A debrief that starts with 'what did everyone think?' produces an anchoring cascade — the first person's impression shapes everyone else's. A calibrated debrief starts with independent signal ratings, then discusses disagreements. This produces better decisions and reveals calibration gaps in the panel.
Common mistakes
01
Confusing articulate candidates with capable ones
Strong communicators perform well in unstructured interviews regardless of their technical depth. Structure corrects for this — because the evidence requirements are the same for everyone.
02
Debrief without independent signal ratings first
When the most senior person in the room speaks first in a debrief, the discussion often ends there. Require everyone to submit independent signal ratings before the debrief begins.
03
No post-hire calibration
The best way to improve interview accuracy is to track hiring predictions against actual performance. Without this loop, the interview system never improves.
04
Structure as bureaucracy rather than discipline
Structured interviewing is sometimes resisted because it feels robotic. The structure is in the preparation — the signal map, the questions, the evidence format. The conversation itself can remain natural.
Example scenario
A cybersecurity startup, Series B. Hiring three senior engineers in 6 months. Interview process: two technical rounds, one culture fit. Decisions made by hiring manager based on panel discussion. Two of three hires underperformed in the first 6 months.
The diagnosis
Panel debrief notes reviewed: all three hires had been described as 'strong' with supporting language like 'clearly gets it', 'great energy', 'would work well with the team'. No specific technical evidence in any debrief note.
The process redesign
Signals defined per interview stage before the next search opened: 4 capabilities, each with explicit strong/acceptable/weak indicators.
Question bank built: 3 behavioural questions per capability, mapped explicitly.
Note format changed: 'candidate said X when asked Y' — not impressions.
Debrief format: independent ratings submitted before discussion begins.
The outcome
Next three hires: all three rated strong performers at 6-month review. Debrief quality improved significantly — disagreements became discussions of evidence rather than impressions. Average debrief time reduced by 12 minutes.
Takeaway
Guessing in hiring is expensive. Structure is the alternative.
Define signals, design questions, capture evidence, calibrate the panel. These are four decisions made before the interview — that determine the quality of the decision made after it.
Related thinking
Continue reading
The Manager Interview Problem