How to Run Structured Interviews (With a Scorecard Template)

1. Why Unstructured Interviews Fail

The case against winging it is one of the most consistent findings in personnel psychology. Across decades of meta-analytic research on selection methods, structured interviews roughly double the predictive validity of unstructured ones — they sit near the top of the list of predictors of job performance, while free-form conversations hover closer to the bottom. The reason is simple: an unstructured interview measures whatever each interviewer happens to ask about, filtered through whatever mood and biases they brought into the room.

• ✓
  Different questions make candidates incomparable. If candidate A got a softball about their favorite project and candidate B got grilled on a failure, you are not comparing candidates — you are comparing questions. Asking everyone the same core questions is the only way two answers can be weighed against each other, which is the entire point of interviewing more than one person.
• ✓
  Unstructured judgments are dominated by first impressions. Without a scorecard forcing attention onto evidence, interviewers form a verdict in the opening minutes and spend the rest of the hour confirming it. Structure interrupts that loop: when you must produce a score and a supporting quote for each competency, a good handshake stops counting as data.
• ✓
  Structure is your legal defensibility. If a hiring decision is ever challenged, “every candidate answered the same job-related questions and was scored on the same documented scale” is a defensible process. “Each interviewer chatted about whatever came up” is not. Consistency protects candidates from bias and protects you from the consequences of it.

2. Design the Question Set from the Competencies

Structured interview questions are derived, not brainstormed. Start with the four to six competencies that actually separate strong performers in the role — the same criteria you used to screen resumes, if you built a proper screening rubric at intake. For each competency, write one or two questions in advance, so the full loop covers everything once and nothing three times.

• ✓
  Pair a behavioral question with a situational one per competency. Behavioral questions (“Tell me about a time you had to deliver bad news to a stakeholder”) sample real past behavior; situational questions (“A client demands a feature you know is a mistake — what do you do?”) test judgment when the candidate lacks directly relevant experience. Together they cover both what someone has done and how they think.
• ✓
  Anchor behavioral prompts in specific past events, not opinions. “How do you handle conflict?” invites a rehearsed philosophy; “Tell me about the last real disagreement you had with a colleague — what happened?” demands a specific story with checkable details. Past behavior is the best available predictor of future behavior, but only if the question forces the candidate to describe actual events.
• ✓
  Script the follow-up probes, too. The signal usually lives two questions deep: “What was your specific role?”, “What would you do differently?”, “What did the other person say?”. Writing three standard probes under each question keeps interviewers digging in the same direction for every candidate instead of improvising — and probing is where vague, borrowed, or inflated stories fall apart.

3. Build the Scorecard

The interview scorecard is where structure becomes enforceable. One row per competency, a weight reflecting how much it matters for this role, a 1–4 scale with a written anchor for each level, and a space for verbatim evidence. Here is a template for a generic individual-contributor role — copy it and rewrite the rows for yours:

• ✓
  Use a 4-point scale with written anchors. An even number of points removes the safe middle that 5-point scales invite, and four levels are few enough to define concretely. The anchors do the real work: if two interviewers cannot agree on what a 3 looks like, the numbers are decoration.
• ✓
  Make evidence notes mandatory, not optional. A score without a supporting quote or example is an impression wearing a number. Requiring one piece of verbatim evidence per rating slows interviewers down in exactly the right way — and it is what makes the debrief a discussion of facts instead of feelings.
• ✓
  Weight competencies before you meet any candidates. Deciding after the interviews that “communication matters most” is usually a disguised way of promoting your favorite candidate. Fix the weights at intake, when they reflect the role rather than a person, and let the weighted total be a strong input to the decision — not a substitute for judgment.

Start the Loop with a Ranked Shortlist

A structured loop only pays off if the right candidates reach it. If interviewers keep meeting people who should have been filtered at the screen, no scorecard can rescue the hours already spent.

Daisy Recruiter applies the same discipline upstream: it screens and ranks every applicant against the competencies you defined — the very ones on your scorecard — with the evidence attached, then schedules the interview loop automatically. Your panel's structured hours get spent on candidates who have already cleared the bar on paper.

4. Run the Interview Loop with Discipline

A great question set and scorecard can still be undone by a sloppy loop. The classic failure mode: four interviewers independently ask about the same flagship project, nobody covers half the competencies, everyone compares impressions in the hallway, and the loudest voice writes the outcome. The loop needs the same structure as the questions.

• ✓
  Assign competencies to interviewers so nothing repeats. With five competencies and a four-person panel, give each interviewer one or two to own, with their scripted questions attached. The candidate stops answering “walk me through your background” four times, every competency gets scored by someone, and each interviewer goes deep instead of wide.
• ✓
  Score independently, in writing, before any discussion. Feedback is due within 24 hours and before the interviewer sees anyone else's ratings — the moment scores are shared early, they converge, and you have paid for four interviews to get one opinion. Independent scores are the raw material of a real decision; contaminated ones are theater.
• ✓
  Debrief on evidence and timebox the decision. Run the debrief competency by competency, opening with the biggest score disagreements and asking each side for the evidence behind their number — “what did the candidate say that made this a 2?”. Keep it to 30 minutes with a decision-maker in the room; a debrief that drifts past an hour has left the evidence and entered persuasion.

5. Calibrate and Improve the System

A structured interview is a measurement instrument, and instruments drift. Interviewers develop private interpretations of the anchors, questions leak onto the internet and start drawing rehearsed answers, and new panelists inherit the scorecard without the shared understanding behind it. Treat calibration as routine maintenance, the same way you review your core recruiting metrics.

• ✓
  Watch inter-rater agreement. When two interviewers score the same competency for the same candidate, their ratings should usually land within a point of each other. If a pair diverges by two or more points regularly, the anchors are ambiguous or someone is using a private scale — either way, that is a calibration session, not a shrug.
• ✓
  Check for drift and retire stale questions. Review score distributions each quarter: an interviewer whose average creeps toward 4 has stopped discriminating, and a question where every candidate suddenly delivers a polished, structurally identical answer has been memorized by the market. Rotate in fresh questions per competency and retire the burned ones.
• ✓
  Train panelists before they carry a scorecard. Have new interviewers shadow two loops, score alongside a calibrated interviewer, and compare notes against real answers before they vote. An untrained interviewer with a scorecard is still better than no scorecard — but the predictive power of structured interviewing comes from the combination of good questions, anchored scores, and people who apply them the same way.