Logistics Case: Assessing AI Literacy in a Global Firm’s Key Talent

Company B, a large retail and logistics company, was an early AI adopter. It rolled out the tools, ran training on how to use them, and in an internal survey most employees said they used AI every day. On paper, the AI transformation looked on track. Look closer, though, and unresolved problems were piling up.

• Adoption was done, but the results were hard to see. Asked “How has AI changed the way we work?”, no one had a clear answer.
• Adoption rates were the only evidence on hand. Log-in counts and active users showed who was turning AI on, but not whether they did anything useful with it afterward.
• Results varied widely on the job. One employee pasted an AI-generated demand forecast draft straight into a report without checking it; another entered a client’s details into AI as is.

The hardest part was the next decision. Leadership wanted to consider a major, company-wide investment in upskilling, but had no evidence to show where to invest or how much. Plenty of people said they used AI daily, yet no one could say with confidence whether the company’s AI capability was actually growing.

In a volatile business like retail and logistics, that question is even more urgent. Large numbers of frontline staff make calls on demand forecasting, inventory allocation, and claims handling every day. When AI skills fall short, the mistakes don’t stay with one person; they spread across the organization. An off-target AI demand forecast taken at face value leads to excess inventory or misdeliveries. Client details or contract prices entered into an external AI tool become a data and regulatory risk that cuts across sites. The more volume and locations a company has, and the thinner its margins, the faster these small errors compound. That’s why, once adoption is done, the real challenge is depth of use.

‍

AI Literacy Is About What People Can Do, Not How Often They Use It

Company B isn’t alone. In a 2025 EY survey of 15,000 employees and 1,500 companies across 29 countries, 88% of employees said they used AI at work, but only 5% said they used it in advanced ways that change how work gets done. Just 12% said they had received sufficient training. (EY, 2025) McKinsey’s research from the same year likewise found that employees use AI more than leaders assume, yet only a minority of organizations can point to an impact on business results. (McKinsey, 2025) There’s a gap between adoption and results, and what fills it is the ability to use AI well.

So what does “using AI well” actually mean? AI proficiency isn’t one vague lump. It breaks down into several distinct skills, and in the same work situation, the outcome depends on which of those skills a person brings. Two people may look equally comfortable with AI on the surface, but break it down skill by skill and the differences show up one layer down.

Employees with weaker skills aren’t slacking off. They’re just working one layer shallower, and that’s where most of the risks above come from. The catch is that these skills don’t grow just because someone uses AI every day. To answer “Is our organization good at using AI?”, you have to look at which skills people have, not how often they use it.

‍

What a Knowledge-Based AI Test Can’t Show You

That’s why Company B didn’t choose a test of what people know about AI. A knowledge test that checks whether someone has memorized terms and concepts can’t tell you whether they can put that knowledge to work.

Telta’s AI Literacy Assessment takes a different approach. It presents scenarios that mirror real work and has participants use AI directly to work through them in open-ended responses. Instead of picking from set answer choices, participants actually write prompts, separate fact from error in AI’s plausible-sounding answers, and decide which information should never be entered in the first place. It observes practiced behavior, not memorized knowledge.

AI then scores these open-ended responses against consistent criteria. Scores don’t swing with an evaluator’s mood or personal judgment, and no one has to read and grade every answer by hand. The result: an abstract notion of “AI competency” becomes measurable behaviors such as structured prompting, output verification, identifying sensitive data, and setting boundaries between human and AI tasks.

‍

Why Start With 50 Key Employees Instead of the Whole Company

What stood out was the order the company chose. The usual move would have been, “Everyone’s using it, so let’s run another round of company-wide AI training.” Instead, they took a step back. Before committing to a big investment, they decided to run a pilot assessment with about 50 key employees.

There was a reason to start with key talent. These are the people who set the standard for how the organization works, and who will later spread effective practices to their colleagues. Get an accurate read on this group, and you have a baseline for rolling out company-wide. At the same time, 50 people is a big enough sample to reveal meaningful patterns without disrupting the whole organization at once. It was the most sensible choice: low burden, clear evidence.

This sequence is especially persuasive for fast decision-making and approval chains. Rather than betting the company-wide budget on an untested hypothesis, a small assessment first confirms with data whether a gap exists and where it is, and then the company acts.

‍

The Right AI Competency Assessment Changes Your Next Investment

The value of a pilot assessment isn’t a ranking. It’s a set of coordinates that shows where the next decision should go. Once you have those coordinates, the nature of the investment changes. Instead of one-size-fits-all training for everyone, you can put resources where they’re needed, in the amount they’re needed: verification skills for one team, task-planning skills for another.

Running blanket training without knowing where the weak spots are tends to bore the people who are already strong and stay too shallow for the people who need it most. With coordinates in hand, you decide the next step of your AI transformation based on data, not gut feel. In fact, Company B has used the results of its key-talent pilot to plan a broader rollout. Because AI scores the open-ended responses against consistent criteria, the standard holds even as the assessment expands to more people.

Plenty of organizations now turn on AI every day. But as the EY data shows, the distance between turning it on and using it well is greater than most assume. Whether an organization’s AI capability is really growing can only be answered with a map of skills, not usage frequency. Like Company B, which used a small assessment to check where it stood before making a big investment, an AI literacy assessment can start small and still be an excellent tool for setting the organization’s direction.

Related Reading

• Even KBS Reporters Took the Test: Companies Now Verify AI Skills Directly
• Workplace AI Literacy: What Exactly Should You Assess?

‍