They Aced the Interview. Will They Do the Job Well?

Have you ever hired someone who seemed like a perfect fit in the interview, only to find the work didn’t match your expectations? The same gap can open up between people who can explain AI well and people who can actually use it to get work done.

Picture a marketing candidate who walks you through a range of AI tools without missing a beat and shares how they cut content production time. After they join, the product description they write includes a benefit that appears nowhere in the product materials. When you ask where it came from, they say the AI wrote that sentence. Their interview answers alone couldn’t tell you how they check their output or take responsibility for it.

TestGorilla’s 2026 report on hiring for AI fluency looks at exactly this problem. In a February survey of 1,928 respondents in the US and UK, 59% said their organization had hired someone in the past year who seemed AI-fluent in the interview but couldn’t apply those skills on the job.

That doesn’t mean 59% of hires failed. The survey was run by a company that sells hiring assessments, and the figure reflects what respondents reported. Still, it gives employers good reason to separate the impression a candidate makes in an interview from how they actually perform.

The Questions a Polished Answer Leaves Open

Candidates can use AI to polish their personal statements and portfolios or to rehearse answers to likely questions. Getting that kind of help is no reason to doubt someone’s experience. The problem starts when a well-organized explanation makes us believe we’ve also confirmed what the person can do.

A Western Governors University (WGU) survey released on September 30 points to the same difficulty. Of 3,128 US workers directly involved in hiring, 60% said AI has made it harder to assess candidates’ real abilities. That is a perception of how hard evaluation has become, not evidence that candidates are less capable.

Interviewers can push for more detail. If a candidate says, “I used AI to improve our content production,” ask what material they fed in, what they changed in the first output, and who checked the final version against which standard. It also helps to separate the results of the whole project from the work the candidate personally owned.

Questions like these turn an impression of fluency into information about actual behavior. But explaining past work well is still different from doing the work in front of you. If you can see the candidate make the judgments that matter most in the role, you can weigh what they told you in the interview against more concrete evidence.

Don’t Drop the Interview; Decide What It Should Confirm

None of this means interviews are useless. A 2022 meta-analysis by Sackett and colleagues found structured interviews to be among the most valid predictors of job performance. There is no basis for dismissing interviews that ask every candidate questions tied to the same job criteria and score the answers against them.

What matters is the purpose of the conversation. If you’re hiring someone to run a new campaign, you need to see how well they interpret product information and communicate it to customers. Beyond whether the candidate sounded confident, check what evidence they relied on and what they made sure not to miss. When interviewers agree on these criteria up front, a verdict like “I had a good feeling about them” can be reviewed against concrete evidence.

Use a Small Task That Mirrors the Real Work

For the marketing hire in our example, you could hand candidates a product brief and a few customer reviews and ask them to write a short launch announcement. If the role uses AI day to day, allow it in the task too, and give every candidate the same rules for using it. Tell them the scope of the task and how long it should take in advance.

A finished draft alone won’t show you what the candidate checked. Ask them to explain which materials they chose and which AI suggestions they cut or changed. You can then add a new condition, such as “This benefit hasn’t been verified yet,” and have them revise the announcement. You’ll see whether they spot unsupported claims and adjust their judgment to new constraints.

Set the scoring criteria to match. Polished prose matters less than whether the copy matches the product materials, whether anything could mislead customers, and whether the candidate can explain each change. If a candidate takes an unexpected approach but the result and reasoning hold up, you should be able to give them credit for it.

No single task can predict everything about how someone will perform after joining. The point is to weigh the experience you heard about in the interview, the performance you saw in the task, and the way the candidate reviews their own results together. Hiring processes get closer to the real role when the hiring manager explains the work they’ll actually assign and helps set the scoring criteria.

Don’t Mistake First-Day Awkwardness for a Bad Hire

By the same token, a new hire who hesitates on day one hasn’t necessarily oversold themselves in the interview. New employees have to learn the company’s materials, tools, and way of making decisions. If instructions are vague or they can’t get to the information they need, even capable people will struggle to deliver right away.

Companies also need to create the conditions for new hires to use the abilities they were hired for. Tell them what their first assignments are, what results you expect, and whom to ask for help, then review their results against the same criteria after a set period. When problems come up, look at both the individual’s abilities and the work environment the company provided. That makes your next call more accurate.

To close the gap between confidence in the interview and expectations after hiring, try changing the question in your hiring debrief. Alongside “Do we think this person will do well?” ask, “Which parts of the job did we actually see them do?” The more specific the answer, the clearer your reasons for trusting a new colleague with the work, and the clearer the criteria you’ll use to check in together after they start.

‍

Related Reading

• How Can You Tell If a Candidate Will Do the Job Well Before You Hire?
• Do AI Detectors Work on Résumés? How AI Screening Finds Real Competency

References

• TestGorilla · The State of Hiring for AI Fluency 2026
• TestGorilla · Survey Methodology and Findings
• WGU · Workforce Decoded Release · Sept. 30, 2026
• Sackett et al. · Meta-Analysis of the Predictive Validity of Selection Methods · 2022

‍