What Is a Structured Interview? How to Reduce Rating Gaps Between Interviewers

Two interviewers meet the same candidate. One says hire; the other says hold. If you work in recruiting, you’ve probably been there. The calibration meeting drags on, and in the end the most senior or loudest person in the room decides. Ask for the reasons behind the rejection, and no one can point to solid evidence.

It’s hard to blame this on differences in individual interviewers’ skill. In most cases, it happens because the interview has no structure. This article looks at why ratings diverge and how to make structured and competency-based interviews actually work in practice.

‍

Why Ratings Differ Between Interviewers Isn’t a People Problem

In an unstructured interview, every interviewer asks different questions. Even for the same position, one asks about experience while another asks about values. Different questions draw different answers, and everyone interprets those answers by their own yardstick.

And it doesn’t stop there. When it’s time to score after the interview, many interviewers rely on memory. See four or five candidates in a day, and their answers start to blur together, with one or two memorable moments driving the whole evaluation. Ratings diverge not because interviewers fall short, but because there’s no mechanism for asking the same things against the same standard and recording them the same way.

Researchers reached the same conclusion long ago. Unstructured interviews are weak predictors of on-the-job performance, and interviewers’ subjectivity and bias heavily shape the outcome. By contrast, adding structure noticeably raises an interview’s predictive validity. (Schmidt & Hunter, 1998)

‍

What Is a Structured Interview?

A structured interview asks every candidate the same questions about the same competencies and evaluates the answers against predefined criteria. Instead of questions interviewers come up with on the spot, it uses questions designed in advance, and answers are scored on a scale defined ahead of time.

Unstructured vs. Structured Interviews

The difference from an unstructured interview is simple. In an unstructured interview, interviewers rely on their own instincts and experience, ask each candidate different questions, and let the conversation flow naturally.

A structured interview sets two things in advance: (1) the question set and (2) the rating scale and scoring criteria. Every candidate gets the same questions in the same order, and each question is tied to a specific competency. That way, even when interviewers change, you can still compare candidates on the same competencies.

Structured interviews matter because of predictive validity and comparability. When you evaluate Candidate A and Candidate B against the same standard, you can say who is stronger and back it up with evidence. Without a standard, what should be a comparison easily turns into a clash of impressions.

‍

Competency-Based Interviews, BEI, and Culture-Fit Interviews: How Do They Differ?

Look into structured interviews and you’ll run into terms like competency-based interviews, BEI, and culture-fit interviews. They’re easy to mix up, but they aren’t competing ideas. The difference is simply what you assess and how you ask.

A competency-based interview evaluates the job-specific and common competencies a position requires. Rather than looking for a vaguely “good” person, it defines the competencies the role needs and focuses only on those.

BEI, or behavioral event interviewing, is a questioning method that verifies competencies by asking about things a candidate actually did in the past. The key is drawing out concrete experiences instead of asking hypotheticals. If you want to assess collaboration, for example, you don’t ask whether someone is a good collaborator; you ask what they did when opinions were split.

A culture-fit interview assesses how well a candidate matches the organization’s ways of working and values. Because it’s a different dimension from competency, it’s often designed separately.

Put simply, a competency-based interview defines what you assess, BEI is how you ask, and culture fit is a separate evaluation dimension. A structured interview is the framework that brings all three together under one standard.

‍

How to Design a Structured Interview in 4 Steps

A structured interview isn’t a massive implementation project. Getting just four steps right can greatly improve interview consistency.

1. Define the job competencies. Narrow down the four or five core job-specific and common competencies someone needs to succeed in the role. Any more, and you won’t be able to cover them in the time you have.

2. Design the questions. For each competency, prepare questions in advance that will bring it to the surface. For experienced hires, BEI questions that ask about real judgment calls on past projects, and the reasoning behind them, work well. You either predict future behavior from past behavior, or present a specific situation to see how the candidate judges and thinks.

3. Set the evaluation criteria and rating scale.  Lay out in a scoring rubric which criteria answers will be scored on, how many points are possible, and what kind of answer earns each score. Skip this step, and interviewers will score the same answer differently. We’ll cover how to build a scorecard, with a template, in a separate article.

‍4. Record the evidence. Evaluations only stop depending on memory when you capture what the candidate actually said as evidence. Summarize it right after the interview, based on what was said, and the later calibration discussion can rest on evidence rather than impressions.

‍

Why It’s Hard in the Interview Room, Even When You Know the Theory

So far, none of this looks especially hard. Yet few organizations actually run structured interviews consistently.

The reason is that interviews happen live. Interviewers have to listen to the candidate, respond, think of the next question, and evaluate, all at the same time. Even with a prepared question list, the conversation drifts into improvised questions and misses the points worth probing. Note-taking keeps slipping, and in the end the evaluation falls back on memory.

It isn’t always trained experts in the room, either. Business leaders have their own day jobs. Even large enterprises struggle to get every interviewer running structured interviews properly. So the carefully designed question list falls out of use, and interviews slide back to the interviewer’s gut feel.

‍

How to Keep Interview Questions Structured in the Moment

What you need is something that holds the structure in place while the interviewer focuses on the conversation. That’s where Telta Interview Pro helps interviewers.

Once the interview starts, the conversation is transcribed in real time. What the interviewer and candidate say is captured separately by speaker, so no one has to take notes. During the interview, it surfaces answer summaries, competency signals, and context-aware follow-up questions alongside the conversation, helping interviewers catch the points worth digging into. Within an hour of the interview ending, a competency report is ready, with scores for job-specific and common competencies, the candidate statements behind each score, and strengths and areas for development.

Let’s clear up a common misconception. Interview Pro is not a tool where AI questions candidates directly or makes the hiring decision. People run the interview; Telta Interview Pro handles the record-keeping and analysis alongside them. Rather than replacing the interviewer’s judgment, it helps interviewers judge against the same standard.

A single bad hire costs a substantial share of that person’s annual salary, often tens of millions of won. Bringing ratings that used to vary by interviewer onto a single standard is the very first step in reducing that risk.

‍

Interview Consistency Starts With Structure, Not Tools

Hiring more interviewers or training them longer won’t make evaluations consistent on its own. The first step is building the structure: what to look for, how to ask, and how to record. The next is helping interviewers hold to that structure all the way through the interview.

You can try Interview Pro free for up to five interviews, with no need to contact Telta first. Try it in a real interview and see for yourself what the competency report looks like. If you’re curious how it differs from STT meeting note-takers like Tiro and Clova Note, we also recommend this next article.

“Is a Meeting Transcription Tool Enough to Record and Analyze Interviews?” Read the article

Related Reading

• Interviewing in the AI Era: Why Your Scorecard Matters More Than Your Questions
• K-Beauty Case: AI Interview Analysis for Consistent Interview Ratings

‍