Hiring & Interviews

The End of the Exam #2 — What Were Test Scores Really Deciding?

In hiring, promotion, and training, test scores have stood in for “Can this person do the job?” Here’s how AI is shaking the two assumptions behind them.
Hundreds of candidates taking a written exam at rows of desks in a vast exhibition hall, with the title “What Were Test Scores Really Deciding?”
Seunghyun Eum
2026-11-18
Seunghyun Eum
|
2026-11-18
Contents

This year, the top scorer on Korea’s certified tax accountant exam got answers from generative AI with a proctor right in front of them. If we see the case from Part 1 only as cheating, we miss a bigger question: what have organizations been deciding with test scores all along?

For years, organizations have used test scores to answer one question: “Can this person do the job?” That answer rested on two assumptions: candidates work alone, and people who know the right answers also do the work well. Generative AI is shaking both at once. In Part 2, we look at the judgments test scores have been making on organizations’ behalf, and why those judgments are getting harder to trust.

For Hiring, Promotion, and Training, Test Scores Answered “Can This Person Do the Job?”

Organizations make three big calls about people: whom to hire, whom to trust with a bigger role, and whether training actually worked. Tests have been a key input for all three. In hiring, that meant aptitude and personality tests, written and coding tests, and language scores. In promotion, it meant promotion exams plus language and certification requirements. In training, it meant completion quizzes and internal certification exams.

Tests held that spot for a reason. They let organizations assess many people at once, use the same questions and scoring criteria for everyone, and compare the results as numbers. Tests never showed the whole person. They lasted because few other tools delivered all three at once. A test score stood in for something you can’t see directly: whether someone can actually do the work.

In hiring, aptitude and personality tests, written tests and coding tests answered “Can this person do the job?” In promotion, promotion exams and language and certification requirements answered “Can we trust them with a bigger role?” In training, completion quizzes and internal certification exams answered “Did the training actually make a difference?” Tests were a rare tool that assessed many people at once, applied the same standard, and produced comparable results.
Tests were a rare tool that delivered scale, a single standard and comparable results all at once.

That Answer Rested on Two Assumptions

Spelled out, the two assumptions are simple. First, a score reflects what the candidate did on their own. Second, people who know more right answers also do the job better. If the first breaks down, you can’t tell whose ability the score reflects. If the second is weak, even a high score may not tell you whether someone will do the job well.

Two HR staff at a meeting table point a pen at a printed candidate score sheet as they compare results.
HR staff reviewing candidate scores. Judging people by scores takes two assumptions.

Assumption 1, “Candidates Work Alone,” Is Breaking Down Beyond the Exam Room

As Part 1 showed, AI cheating went undetected on the tax accountant exam even with a proctor right in front of the candidate. When supervision is light or the test is taken remotely, it’s even harder to check. The same problem extends to assessments outside the exam room.

Hiring: finished work makes it hard to see the candidate’s own judgment. Using AI to prepare applications and complete hiring assignments has become common, and remote aptitude and personality tests come with weaker supervision. Looking only at the finished work and the score, it’s hard to tell how much of the candidate’s own judgment went into it. Open-ended questions and in-person interviews help, but their value is limited when the questions simply check for a set answer.

Promotion: AI polishes the self-assessment. AI can turn the same notes on someone’s results into a far more logically organized self-assessment. It’s getting harder to gauge a candidate’s judgment or how they work with others from how well the writing reads.

An employee edits a polished document on a monitor, with a notebook of handwritten notes beside them.
Rough notes, polished self-assessment. AI does the polishing.

Training: passing the quiz isn’t the same as understanding the material. Post-training completion quizzes can be answered by asking AI. When a learner passes with AI’s help, the result alone makes it hard to tell whether they understood the material.

In all three cases, you still get a deliverable and a score. What has become hard to tell from the result alone is how much of the person’s own judgment is in it.

Assumption 2, “Knowing the Answers Means Doing the Job Well,” Was Weakening Before AI

Hiring research has long compared how well each selection method predicts on-the-job performance, known as predictive validity. For years, the benchmark was a 1998 summary by Schmidt and Hunter. In that analysis, general cognitive ability tests ranked just behind work sample tests, which ask candidates to perform tasks like the ones the job requires. The findings were widely cited to support using cognitive ability tests in hiring.

In 2022, Sackett and colleagues, a team of US researchers, reanalyzed the studies behind those numbers. They concluded that the earlier analyses had corrected the validity estimates too aggressively, making them look larger than they were. Fixing the correction changed the rankings.[1] Among six methods, general cognitive ability tests fell from tied for second to fourth. First place went to structured interviews, which ask every candidate the same questions and score them against the same criteria. Second went to job knowledge tests, which ask about what the role actually requires. Unstructured interviews and years of experience ranked fifth and sixth in both analyses.[1][2]

Slope chart comparing how well six selection methods predict job performance, as ranked in a 1998 summary and a 2022 reanalysis. General cognitive ability tests fell from tied for second to fourth and work sample tests from first to third, while structured interviews rose from tied for second to first and job knowledge tests from fourth to second. Unstructured interviews and years of experience ranked fifth and sixth in both.
In the 2022 reanalysis, structured interviews ranked first in predictive validity and cognitive ability tests fell to fourth. Source: Sackett et al. (2022), Table 3

The leading methods in the reanalysis, structured interviews and job knowledge tests, are designed around the job and apply the same standard to everyone. How closely a score tracked performance varied widely by method, and broad tests of whether someone knows the right answers did not come out on top.

In AI-Assisted Work, Knowing the Answers and Getting the Work Done Ranked Differently

AI in the workplace weakens the second assumption further. Anyone can ask AI for an answer. Differences between people then start to depend less on how much they know and more on how they hand work to AI, check what comes back, and make a call.

That gap showed up clearly when four reporters from the digital tech desk at Maeil Business Newspaper, a Korean business daily, took Telta’s AI Literacy Assessment. The reporter who placed first in the understanding section, with 15 of 16 questions right, came third of four in the application section at Lv 2.5. The reporter who got 10 right in the understanding section scored Lv 3.0 in the application section, landed in the top 13% of all participants, and finished first overall.[3]

Reporter A got 15 of 16 understanding questions right and ranked first in that section, but scored Lv 2.5 in the application section and placed third of four. Reporter B got 10 right but scored Lv 3.0 in the application section, landed in the top 13% of all participants, and finished first overall.
Rankings on understanding questions and on scenario-based application did not line up. Source: Telta, AI Literacy Assessment results for four Maeil Business Newspaper reporters

The application section was a simulated assignment: each reporter played an IT employee at a company and used AI to solve an internal problem. The overall winner earned high marks for meta-prompting, having AI first write the prompt that would produce the report draft. The reporter who topped the understanding section was advised to break the work into smaller steps by specifying data formats and length.[4] None of that shows up in a count of right answers.

Using AI more didn’t mean using it better, either. When Telta assessed 130 employees at Korean companies, 76.9% said they used AI almost every day, yet there was no statistically significant correlation between how often they used it and how well they used it. The gap in actual proficiency between individuals reached more than 70 percentage points, and more than half of daily users fell short of the benchmark in risk awareness.[5]

Of 130 employees at Korean companies, 76.9% used AI almost every day, but there was no statistically significant correlation between frequency of use and proficiency. Proficiency varied by more than 70 percentage points from person to person, and more than half of daily users fell short of the benchmark in risk awareness.
76.9% of respondents used AI almost every day, but frequency of use did not explain proficiency. Source: Telta AI Literacy Report 2026

In October 2025, Gartner predicted that generative AI use would weaken critical thinking and that by 2026, 50% of organizations worldwide would require “AI-free” skills assessments. It also predicted that by 2027, 75% of hiring processes would include certifications or tests of workplace AI proficiency.[6] In other words, Gartner expects growing demand to check both the ability to think without AI and the ability to work with it, directly.

With Both Assumptions Shaken, Relying on Test Scores Alone Is Coming to an End

With both assumptions shaken at once, it has become hard to answer “Can this person do the job?” from a test score alone. It’s harder to confirm that the score reflects the person’s own ability, and even when it does, whether they’ll do the work well has to be checked separately.

Dropping tests with nothing to replace them would make things worse. It would be harder to compare many people on the same standard, and the risk of falling back on weaker evidence, such as an interviewer’s impression or years of experience, would grow. In the same reanalysis, unstructured interviews and years of experience sat at the bottom of the rankings.[1] Aptitude and personality tests, qualifications, and certifications are still useful for confirming a baseline of core skills.

Reviewers at a long executive meeting table go over HR binders and evaluation forms.
A promotion review meeting. When impressions fill the gap left by test scores, the evidence gets weaker.

The open question is how to see how people make judgment calls in real work situations, at scale and against the same standard. Assessments like that have existed for a long time: structured interviews, work sample tests, and assessment centers. In Part 3, we look at what has made them hard to scale, how the same AI that shook the exam changes things on the assessor’s side, and the methods we use at Telta in hiring, promotion, and training.

Sources

  1. Sackett, Zhang, Berry, Lievens · Revisiting meta-analytic estimates of validity in personnel selection · Journal of Applied Psychology, 2022 — Table 3, comparing Schmidt & Hunter (1998) estimates with the reanalyzed validities.
  2. HumRRO · Is cognitive ability the best predictor of job performance? — summary of the reanalysis.
  3. Telta · Four Maeil Business Newspaper digital tech reporters took the AI Literacy Assessment, and the results defied expectations (in Korean) · April 27, 2026.
  4. Maeil Business Newspaper · Kang Young-woon · “AI doesn’t rank you by seniority” (in Korean) · April 20, 2026 — the reporters’ assessment tasks and feedback (republished on Daum).
  5. Telta · AI Literacy Report 2026 (in Korean) · April 9, 2026 — assessment of 130 employees at Korean companies.
  6. Gartner · Gartner Unveils Top Predictions for IT Organizations and Users in 2026 and Beyond · October 21, 2025.
Seunghyun Eum
Seunghyun Eum
Client Executive, Telta
Business Administration, Sungkyunkwan University Former, Mercer (Senior Consultant)
Delivered HR projects for large Korean enterprises at a global HR consulting firm. At Telta, Seunghyun oversees client relationships.
Talk to Our Team