AI activity is everywhere, but the business value is much harder to see. What is behind this disconnect between activity and value?
Economics has an answer for understanding this problem, and that’s the topic of today’s essay.
Let’s think about the link between the individual and the organisation. Panos is a Risk Director at an investment bank, who I interviewed during my research for AI-Q. During our discussion he explained how he’d been using AI tools to automate part of his reporting process. When I asked him to estimate how much time he’d spent building the tools and how much time he’d saved, there was a net benefit of eight hours a month.
What Panos did shows great personal initiative and in the book I explore how the way he thinks about AI influences his approach. From a leadership perspective, that is exactly the sort of productivity gain that leaders hope AI will unlock for their organisation.
But imagine what happens when 50 people across the same organisation make similar improvements independently.
What you now have is 50 useful automations. You might also have potentially duplicated effort, 50 different ways of solving the same problem and no shared view of what is working, without a mechanism in place to scale the best ideas. In a regulated industry such as banking, you also have 50 different decisions being made about data, controls, and the risks that are associated with those decisions.
And even the eight hours Panos saved do not automatically become eight hours of business value. Without the right data in place there is no way to know if that capacity is actually redeployed, whether it improved quality or outcomes, or whether one person’s task simply became quicker.
Economists have a name for it. The fallacy of composition is the assumption that what is true for one part of a system must also be true for the whole.
In complex systems such as organisations, collective outcomes are not simply the sum of individual actions. Once people, processes and technologies interact, new dynamics emerge - new dynamics that we likely have not accounted for. We tend to extrapolate in a straight line from the present to the future.
It is easy to fall into precisely this trap with AI. We assume adoption is a fairly linear journey. A journey where you give employees AI tool licences, you run training sessions, encourage them to experiment, and collect use cases. And as more and more use cases spread across functions and departments and more people become comfortable using AI, the organisation steadily moves towards AI maturity.
However, as many leaders already know, the reality is a lot bumpier than that.
What they’re seeing is that individual productivity gains do not automatically add up to organisational ROI.
The answer is not to stop Panos using AI. Quite the opposite! The challenge is to redesign the organisation so that the value he creates by using AI can compound rather than fragment into the ether.
Too often, organisations have not accounted for these dynamics. Instead, they have taken what I think of as a “drop and hope” approach to AI: drop the technology into the organisation and hope the benefits emerge.
What does it take to go beyond drop and hope?
That is one of the questions I have spent the last few years researching for my forthcoming book, AI-Q: The New Leadership Imperative for the Human + AI Era. It draws on a survey of nearly 200 senior leaders, 150 academic studies and reports, more than 50 interviews with leaders across functions and sectors, and my own work with organisations.
Across that research, five practices kept appearing in the ways leaders and organisations were approaching the challenge of turning AI activity into results.
1. Start with the business problem, not the AI
This sounds obvious. The reality is that AI makes it remarkably easy to do the opposite.
Performance reviews offer a useful illustration. Many companies including JPMorgan and BCG have introduced in-house AI tools that can help employees draft performance reviews. The benefit is immediate: AI makes this rather laborious and lengthy task quicker. However it does not stack up to the transformative impact that AI promises!
One of the things I explore in the book is whether this is applying AI to the wrong problem to start with. Let me explain. I think you’d be hard pressed to find one person who would say traditional performance reviews work well. They are struggling to do everything we ask of this process. We expect that this process will support employee development, signal promotion readiness, inform pay decisions and satisfy regulatory requirements.
AI can give us a faster version of that existing process. That does not necessarily mean we have improved the process. Or to quote the Spanish Prime Minister Pedro Sánchez recently: “Speed is useless if we are heading in the wrong direction”.
Therefore the AI-first question is “how do we use AI to make this process faster?” but the more strategic and problem-first question is “How should we measure, assess, evaluate and reward employee performance in a workplace where humans and machines increasingly work together?” This leads us to rethink the process itself rather than simply automate parts of it.
And this applies well beyond performance reviews. AI makes it easier than ever to optimise a task. That makes it even more important to stop first and ask whether it is the right task, within the right process, solving the right problem.
Otherwise, we risk becoming very efficient at doing things that may no longer make sense.
2. Draw the red lines
Governance is often treated as a brake on AI adoption. Yet what I found fascinating when speaking to high AI-Q leaders was that they see governance and boundaries differently. They realise that when people do not know what they are allowed to do with AI, well, that uncertainty creates friction. Some employees avoid using it altogether. Others experiment anyway, sometimes using tools and data in ways the organisation has no visibility over.
What those leaders do is to focus on designing and communicating the red lines, within which their employees can use AI to solve the problems that matter most to them. In an organisational context, clear red lines provide permission.
Where are the red lines? There are no hard and fast rules and they vary depending on the leadership perspective, the organisation’s context and the broader industry environment.
I’ll share an example from my own work with the UK Government Office for Science.
In the public sector, reproducibility is a non-negotiable. It has to be possible for someone else to follow the same methodology that our project team followed and to reproduce the findings. That requirement immediately changed which AI tools we considered to be appropriate for the project. Some tools that might have been useful for deep research had to be ruled out for parts of the work where their probabilistic nature made reliable reproduction of outputs difficult.
That constraint did not prevent our adoption of AI. If anything, it made the boundaries clearer, which is an important distinction. Governance is not about producing the longest possible list of restrictions. It’s about helping people to understand where they have room to experiment, which risks they can take and which responsibilities cannot be delegated to a machine.
Good red lines reduce ambiguity. They tell people where they have freedom to move.
3. Experiment - but intentionally
Experimentation is getting a bad reputation in AI discussions. “We’re running lots of experiments, but they don’t lead anywhere” is what I hear. It’s understandable because organisations have accumulated pilots without necessarily accumulating much learning in that process.
How are high AI-Q leaders approaching experimentation? In three distinct ways.
The first is that they recognise its importance in building capability across the organisation. Research on AI fluency found that people who were more AI-fluent were twice as likely to say they had learned through experimentation. That makes sense to me. AI is difficult to understand in the abstract. People often need to use it in the context of their own work before they can see the art of the possible. This bottom-up experimentation is one of the ways organisations build capability.
The second is that they differentiate between experimentation and intentional experimentation. An intentional experiment starts with a clear problem or hypothesis. It has some criteria for success or failure. It captures what has been learned. And, if it works, there is some route from an experiment to a repeatable process. That last part is particularly important. An organisation can run dozens of successful experiments and still struggle to make meaningful progress if the results remain isolated within the teams that developed them.
The third is that they see experimentation as an important forcing mechanism for something organisations have talked about fixing for years, but remains the holy grail: their data.
Many AI experiments get stuck not because the idea is weak, but because teams cannot get access to usable data quickly enough to test it. Despite the constant claim that “data is king”, organisational data is surprisingly often fragmented, inaccessible or badly structured.
Even a failed AI experiment can create value if it reveals the infrastructure that needs fixing before the organisation can go further.
When you look at those three approaches as a whole the test of experimentation is therefore not just how many pilots an organisation has launched, but what it has learned and what has changed as a result. In order to assess that, you need to rethink your internal measures.
4. Measure impact, not activity
Which experiments matter? What is working? What should scale? What should be stopped? Which teams are ahead and which are lagging? All of those questions can be hard to answer with traditional business metrics. This is where it becomes important to remember that activity is not outcome.
Mike Jones, CTO at loveholidays, gave me a useful example when I interviewed him for the book. At loveholidays, AI implementations are assessed across three dimensions: adoption, impact and cost. An AI system that has high impact at manageable cost may be worth replicating and scaling. A tool that has high usage, low impact and high cost probably is not. That distinction matters because it separates something that is popular from something that is actually creating value. It also challenges the assumption that the goal should always be to get more people using AI more frequently. Usage only tells you if people are using it but you have no way of knowing if they are using it well.
What I am hearing from leaders in technology and consulting companies is that they are rolling out dashboards to employees which show them their token usage and cost - in an effort to help them differentiate between usage and impact. This appears to be a response to the trend of tokenmaxxing - encouraging employees to use AI led to skyrocketing token costs for many companies - and as IBM says “after experiencing its peak in the spring of 2026, tokenmaxxing’s popularity as an approach to AI adoption has largely begun to wane”.
The reason is clear: in a mad rush to encourage adoption, usage was equated with productivity, when the on-the-ground reality turned out to be that usage also brought things like AI-slop, shadow AI and the verification tax, the growing burden of checking and correcting AI outputs. However a data point on its own does not necessarily bring about behavioural change, if employees don’t know what the “using it well” part looks like from the leadership perspective. Token dashboards risk becoming the equivalent of calorie labels on food. They shift the onus of responsibility to the consumer (or in this parallel, the user of AI), but without the clarity on what and how to change.
5. Build bridges so learning travels
The final practice is essential for scale, because it delivers diffusion.
Most organisations currently have a multi-speed workforce when it comes to AI. Some employees are experimenting every day. Others barely use the tools at all. Another group avoids it because to use it would be “like turkeys voting for Christmas”.
The organisations making more progress focus less on what each individual knows on their own, and more on whether they have designed pathways for knowledge to spread. This brings us back to Panos, the risk director. His eight-hour saving becomes much more valuable if somebody else can learn from it, reuse it, improve it or apply his underlying approach to reporting somewhere else.
One way to make that happen is by finding your AI whisperers.
These are often not the people with the most formal authority or the deepest technical expertise. They are the colleagues who work out practical ways to use AI in their own context and then help others understand how to do the same. As one leader put it AI learning is often “caught, not taught.” The essence is simple: People learn from people.
Much of that learning happens informally: watching a colleague use a tool, asking how they approached a problem, finding out what went wrong the first time or being shown a workflow that makes your own job easier. AI whisperers can therefore act as translators and multipliers: showing colleagues what worked, sharing what did not, and passing those lessons back into the wider organisation.
The important thing is that useful knowledge does not remain trapped in isolated pockets of excellence, but is allowed to travel.
The result? Alignment and progress.
From AI activity to AI-Q
These five practices are not the whole story.
Along with other findings from the research, they formed part of a wider pattern that differentiated the leaders getting measurable results from AI. I call that wider leadership capability AI-Q, or AI Quotient.
AI-Q looks beyond AI literacy and familiarity with tools to the Mindsets, Abilities and Practices leaders need to create value in a world where human and machine intelligence work together.
One finding from my survey was particularly striking:
Leaders with a high AI-Q were three times as likely to report a quantified business outcome from AI.
This is an association, not proof that any one practice causes better returns. However it does reinforce the importance of looking beyond the technology itself to how leaders and organisations are adapting around it. And I think that is where our conversations about AI readiness need to go next. We have spent much of the last few years asking how to get more employees using AI. Whether they have the right tools, sufficient training and enough opportunities to experiment.
All of those things matter, but they are not enough. They are necessary conditions, but not sufficient.
Think again about Panos and the eight hours he saved. The question for his organisation is not simply whether he used AI successfully. It is what happens next.
Does somebody else benefit from what he has learned?
Does the organisation know what worked and what did not?
Is the capacity he freed up being put to better use?
Can the organisation point to a measurable outcome that improved?
Those are much more revealing questions about AI readiness than the number of licences deployed or pilots launched.
Ultimately, individual capability matters - but it does not automatically create an organisation capable of turning AI investment into sustained results.
That depends on whether leaders have redesigned their organisation so that those individual gains are allowed to compound rather than fragment.
I explore the wider research and the Mindsets, Abilities and Practices of high AI-Q leaders in AI-Q: The New Leadership Imperative for the Human + AI Era, published in January 2027.
If these are questions you’re working through too, subscribe to The Athenian Intelligence for my research and writing on AI leadership and the economy.
Recently on The Athenian Intelligence:
