AI Hiring Case Study #1: Turning Interviewer Gut Feel into Evidence

When interviewers walk out of the meeting room after an interview, they often share the same nagging doubt. They talked with the candidate for more than an hour, yet they still can’t answer one simple question with confidence: “Will this person actually be good at the job?”

The Limits of Interview Evaluation Built on Gut Feel

Hiring is, at its core, a string of high-context judgment calls. In a short conversation, interviewers have to piece together a candidate’s tone, reasoning, quick thinking, and countless other qualitative cues to predict how well that person will fit the team. It takes an enormous amount of energy from the interviewer.

AI in hiring is a hot topic, yet this part of the process has been slow to change, and that’s hardly surprising. “How could a machine pick up on the subtle intent and context behind what a person says?” is a perfectly reasonable concern for anyone responsible for hiring.

Our team at Telta had the same concern. We don’t believe AI can replace an interviewer’s intuition. But we wondered whether it could serve as a reliable assistant that helps turn intuition into confidence. We wanted to capture the keywords that slip by and the context that fades, and use that data to make each judgment better informed.

That question kicked off an eight-week AI hiring experiment at Telta. We didn’t roll out a big new system. What follows is a small, candid record of what we chose to do instead.

‍

Two Strategies Behind Our AI Hiring Experiment

Our goal was clear: capture as much objective evidence as possible from the conversation without lowering the quality of the qualitative dialogue between interviewer and candidate. To get there, we set two strategies.

Interviewers Ask the Questions; AI Records and Sorts the Answers

For recruiters and hiring managers, few things feel more burdensome than a new evaluation tool. The moment you force a fixed list of questions or a complicated scorecard on interviewers, they end up staring at a laptop screen instead of looking the candidate in the eye.

So we kept our existing free-flowing interview style and designed the process so that AI reads the context of the conversation and extracts the data. We drew a firm line between roles: people focus on listening and asking deeper questions, and the system handles note-taking and analysis.

Evaluation Criteria Grounded in Global Job Descriptions

If all we got was a speech-to-text (STT) transcript, the experiment would have been pointless. So we used AI to analyze tens of thousands of job postings from leading companies around the world and first defined a standard skill set that every business development (BD) professional needs.

That skill set then acts as the filter AI uses to analyze interview answers. When a candidate describes a past project, for example, the AI checks the answer against the global job description data and maps it: “This answer reflects strategic thinking and data analysis.”

As a result, once the interview is over, interviewers don’t have to replay more than an hour of conversation. They can review only the answers the AI has sorted by competency and do a final check on whether their judgment holds up.

‍

Telta’s AI Hiring Case: What We Experienced Firsthand

With our strategy in place, we applied this approach right away to interviews for a business development (BD) role. We simply slotted AI in before and after the interviews we already ran. Here is how the process actually played out.

Before the Interview: Setting Criteria with Telta’s BD Skill Set

Before each interview, we reviewed the core BD skill set Telta had defined in advance. Standards for what “good at the job” means, which had only ever lived vaguely in our heads, now came into focus as objective indicators and precise language. It helped us revisit which competencies our team truly needed, and we went into each interview with clearer, more structured questions to test them.

During the Interview: Laptops Closed, Full Attention on the Candidate

💡 Interviews were recorded only after candidates gave prior written consent.

Once the interview began, we pressed record and started the conversation. Normally the room would be filled with the clatter of laptop keys as we typed out answers, but not this time. We left the note-taking to AI and gave the candidate our full attention, looking them in the eye.

The skill-based interview guide we had prepared proved especially useful. Throughout the interview, we could glance at it to see which competencies had been fully tested and which areas still needed more questions, and steer the conversation accordingly. That let us put our energy into what human interviewers do best, reading context and asking deeper questions, and we clearly remember having far richer conversations than usual.

After the Interview: Confidence Backed by a Data Report

After each 90-minute interview, we reviewed the report the AI had generated. From the long conversation, it had pulled out the answers that mattered for testing competencies and sorted them by skill. Even after interviewing several candidates, we could quickly recall any one candidate’s key answers. Instead of a vague sense that someone was “good at market analysis,” we could reread the supporting statements the AI had flagged and build a shared conviction as a panel.

The data kept paying off after the hiring decision, too. From a manager’s perspective, it showed us in advance how to onboard the new hire and where they would need support once they joined the team, giving us specific points to manage around.

‍

Two Things We Gained from AI Hiring

After building our own AI hiring case in-house, our conclusion is clear.

First, the basis for our decisions became transparent. We could check an interviewer’s intuition, which used to feel hard to pin down, against concrete, visible evidence before making a hiring decision.

Second, the interviews themselves got better. Freed from the burden of taking notes, interviewers could focus entirely on the conversation with the candidate, and that was a major gain in its own right.

All of this, however, rests on one crucial condition: how precise the criteria are that AI uses to analyze the conversation. No matter how smart the AI, if its criteria are vague, you can’t trust what the interview report tells you.

So before starting the experiment, Telta first had to answer a basic question: What does a BD professional do, and what do they need to be good at? In Part 2, we’ll walk through exactly how we analyzed tens of thousands of global data points to break BD competencies down into 24 defined skills.

*This article is based on Telta’s own internal hiring experience.

Related Reading

• AI Hiring Case Study #2: 24 Criteria That Define a Strong BD Hire
• AI Hiring Case Study #3: How a 90-Minute Interview Becomes a Report

‍