AI Accelerates. Humans Still Decide.
Walk into almost any organization right now and you’ll hear the same anxious refrain: “We’re behind on AI.” On a recent episode of The Kirkpatrick Podcast, Chief AI Officer Myra Roldan offered a clarifying response. If every company, in every industry, is convinced it’s behind, then “behind” has lost its meaning. What organizations are actually feeling is not a competitive gap. It is the absence of a way to tell whether their AI investments are working at all.
That absence is exactly what evaluation exists to fill. At Kirkpatrick Partners, we believe evaluation is the unlock for the future of organizational health and performance — and nowhere is that more true than with AI. The technology is a genuine accelerator. But acceleration without direction is just speed, and speed without judgment is how organizations drive confidently in the wrong direction.
The Misconceptions That Cost The Most
Roldan walked through a hierarchy of AI misunderstandings, ordered by how much money they quietly burn. The cheapest is the belief that you’re falling behind. More expensive is the assumption that because employees have used AI to draft an email at home, they are ready to use it for serious business work. The most expensive of all is the conviction that everyone is now a programmer — a belief that has organizations handing out enterprise coding licenses with no guardrails and watching their technical debt balloon.
Vanessa knows the pull of that last one firsthand. She tells a story about building internal tools in no-code platforms, feeling capable right up until something breaks and she realizes she doesn’t know what she’s looking at, where the security holes are, or whether it’s safe to deploy. AI makes the on-ramp exhilarating. It does not, on its own, supply the judgment to know when to stop and call an expert.
Adoption Is Not Usage
The most important distinction in the conversation was also the simplest. Roldan described onboarding a client who proudly reported a 90% AI adoption rate — until their own IT team admitted that most of that “adoption” was people letting AI summarize their meeting notes. Licenses had been distributed. Logins had occurred. And almost nothing of strategic value was happening.
This is the gap the Kirkpatrick Model was built to expose. Counting how many employees have access to a tool is a Level 1 question — it measures Reaction and participation. It says nothing about whether people gained real capability (Level 2: Learning), changed how they work (Level 3: Behavior), or moved a business outcome (Level 4: Results). When an organization mistakes a Level 1 number for proof of impact, it isn’t measuring AI. It’s flattering itself.
The Kirkpatrick Four Levels® give leaders a way to ask better questions. Not “How many people are using it?” but “What critical behaviors are we trying to change, and is the Performance Environment — the access, the managerial support, the governance — actually set up to support them?”
Start With Results, Not Tools
When Vanessa asked what leaders should do, Roldan’s answer landed squarely in Kirkpatrick territory: start with your business goals. Vanessa named it in real time — “Level four.” Before selecting a tool or rolling out a license, leaders should be able to answer why they are integrating AI, what problem it solves, and what outcome they expect. “We want to increase efficiency” is not an answer; it’s a place to start digging.
This is Return on Expectations in practice. ROE begins not with the technology but with the end in mind: the results that matter to stakeholders, defined and agreed upon before the work begins. From there, leaders work backward to the leading indicators and behaviors that will signal progress along the way — the early, observable evidence that an AI initiative is on track long before the final numbers arrive.
A Negative Return Is Not A Failure
One of the most reassuring moments in the episode was Vanessa’s reminder that, like every software rollout in history, AI will often show a negative or muted return at the start. People are still learning. Trust is still forming. Governance is still catching up. The mistake is not the early dip; the mistake is treating the dip as a verdict and abandoning the initiative before the evaluation can teach you anything.
This is the discipline of monitoring and adjusting — the engine of a culture of evaluation. A weak early result is data. It tells you people never received governance documents, or that security fears went unaddressed, or that no one explained they weren’t training their own replacement. Evaluation turns a disappointing dashboard into a to-do list.
Why Humans Still Hold The Value
For all of AI’s acceleration, Roldan and Vanessa kept returning to what it cannot do. AI is good at assembling information. It cannot replicate human nuance, read organizational politics, or understand the context behind a quote in an impact study. It can draft the beginning of a strategy, but it can’t know the people, the history, or the unspoken dynamics that make a strategy actually work.
That is why Roldan urges leaders not to rush to cut staff but to re-skill and upskill them. “Your people are your business intelligence,” she says — especially those who have been around long enough to know where the bodies are buried. The human work of building relationships, interpreting data through the right lens, and exercising ethical judgment is not a cost to be automated away. In an AI-accelerated world, it is the differentiator.
Bring The Rigor That Makes Findings Credible
AI will keep getting faster. The organizations that win won’t be the ones that adopt it first — they’ll be the ones that can prove what it’s actually doing, in language their stakeholders trust. That requires applying AI strategically across all four Kirkpatrick Levels while keeping human judgment and ethical guardrails at the center.
That is precisely what the Kirkpatrick AI-Powered Evaluation Specialist Certification was built to teach. Developed in partnership with UnDesto AI, this first-of-its-kind program helps L&D professionals, HR leaders, and evaluation consultants use AI as an accelerator without sacrificing the rigor that makes findings credible. Three live online sessions run July 22–24, with a founding cohort rate of $1,050. Learn more and reserve a seat at https://www.kirkpatrickpartners.com/event/kirkpatrick-ai-specialist-1/.
