A sunny rooftop near Farringdon, pizza and rosé. I walk in, and while I am saying hello to Matt, one of the authors, the first random thing happens. I spot George Walkley, someone I had not expected to see but who had recently endorsed my book and I’ve had the pleasure to get a sneak preview of his.
Then he introduces me to Joe, who used to organise events. I think to myself “how random” that I need to organise my book launch event and here am I talking to someone that did this for a living.
We take our seats for Matt’s interview about the book. In the spirit of the evening, I jot down the random words and ideas that stand out to me as he speaks. When I read them back the next morning, I realise they are not three random things after all. They have something in common. And the idea of randomness, it turns out, can teach us a great deal about how we approach AI adoption.
That is what I explore today.
The three random things:
Lottery
Apophenia
Luck Surface Area
1.Lottery
“Random selection leads to less biased outcomes than human judgement”, said Matt.
This is the kind of statement you don’t want to believe is true.
But think of it in terms of recruitment. You interview 10 different candidates. You quickly rule out the handful that are not a good fit. And you might have 2 or 3 that you think are particularly strong. How do you choose between them? Many companies add further interview rounds. It is not uncommon nowadays to go through 7 or 8 interview rounds for mid to senior positions.
Another option would be to use a lottery and choose at random. Dr Tomas Chamorro-Premuzic has previously argued that this would be worth experimenting with, though I have not come across any companies where this is publicly disclosed as part of their approach.
However, when researching this I came across other fields where this has been adopted, such as in grant applications for example. Volkswagen Foundation has experimented with grant lotteries, usually as a “lottery among the qualified,” where applications are screened to a threshold and then a winner is drawn at random from the shortlist.
The reasoning is that our human need to create consensus can actually hinder choosing really “out of the box” solutions. Having started this approach in 2017 VW has been able to assess the outcomes of this and they found that randomized selection avoids conflicts of interest and unconscious bias and has led to more diversity in grantees.
Humans construct narratives to justify picking one near-identical candidate over another, and that’s exactly where biases creep in. With a lottery or randomisation you admit “we can’t tell these apart, so we won’t pretend to.”
Enter AI.
The problem is not that AI is biased, at the end of the day us humans are too! The problem is that AI's bias appears as objectivity. It comes with a number and it is a black box. A committee or a leadership team can be argued with; someone can say "hang on, are we just picking the one who looks like us?" A score feels like a finding. This can also remove the friction that sometimes catches bias in the process. A bias that was previously visible and contestable becomes hidden and very hard to challenge.
How many of you have been in a decision making situation where someone says “the data says” or “the AI says” with an aura of authority and finality - as if that settles the question. In one meeting I heard a chief strategy officer say that they had asked ChatGPT of all things if their strategy was the right one. The catch for leaders is that accountability still rests with you. Yet can directors and executives truly understand and be held responsible for outcomes influenced by complex AI tools they cannot fully see inside?
2.Apophenia
This is our brain’s natural tendency to identify patterns and to spot connections even between seemingly unrelated things. This has its roots in psychology, though the word is from the Ancient Greek ἀποφαίνειν, “to appear”.
Our brains have evolved to be optimised pattern identifiers. We see patterns where there are none. We spot trends in what is simply randomness. I found this intriguing because one of the arguments around the use and benefits of AI is that they can be very good at identifying patterns. For example, the argument goes that it can be very hard for the human brain to review large amounts of customer data and spot a new trend in buyer behaviour, and AI can plug or complement that gap in human capability.
This means we have built a comfortable assumption into every AI business case: that AI finds patterns, and finding patterns means finding “insight”.
It does not. AI is built to find a pattern whether or not one is really there. Give it a blip in last year’s numbers and it will treat that blip as a rule and project it forward. If you ask it something it has no answer to and it will not pause or hedge. It will give you a confident, fluent, completely wrong answer in the same tone it uses when it is right.
That is the part that worries forward-thinking leadership teams. Not that the machine is sometimes wrong. We are also sometimes wrong. It is that we both try and look for patterns even where there are none and therefore the notion that investment in AI can help you address that human “shortcoming” for better business decisions, is flawed.
It is that the machine is wrong with confidence, without a warning sign. When you work next to a junior analyst you learn over time when they are unsure. They they hesitate, they flag the weak assumption. AI has none of that and it gives you the noise bundled up as signal. So “more data plus more AI equals better decisions” becomes “more confident noise, faster” instead and we cannot see the difference because our brain’s normal critiquing mechanisms are bypassed by the confidence of the AI output.
An even bigger risk comes when we read intelligence into the output. Because it sounds considered, we assume it has considered. We assume there is inherent understanding or judgement, even when that intent is simply not there.
It is perhaps unsurprising then when I was interviewing business leaders for my book “AI-Q” they mentioned that they are increasingly placing huge emphasis on critical thinking skills and the ability to have depth and to ask the right questions.
3.Luck Surface Area
This idea, from Jason Roberts, is that even when something looks like pure luck and feels beyond your control, two things are still in your hands: the work you Do, and the degree to which you Tell others about it. Luck is the product of the two.
L = D x T.
If you Do a great deal of work but never talk about it then serendipity can't find you; if you talk endlessly but produce nothing and there's nothing for luck to attach to.
The promise of AI for business leaders is that it can let you produce more and reach further at almost no cost. More content, more proposals, more outbound, more of everything, faster. In theory that should be pure upside. More chances for something good to land. But the problem with AI tools accessible to everyone is that it is not just happening to one business leader or one company, it is happening to everyone. The volume of everything goes up and it becomes harder to get noticed.
Mark Read, former CEO of WPP a marketing agency put it like this:
“The ability to produce stuff is obviously going to increase, but the ability to cut through is going to be harder.”
This means that the advantage you thought you were buying with AI, the cheaper production, the wider reach, will evaporate over time as AI tools become available to all.
Where will your competitive advantage come from then?
A couple of weeks ago I interviewed Francis Hintermann, Global Managing Director of Accenture Research, at the KIMRA conference, with research and insights professionals across banking and law firms. When asked this question he said it will come from us. Each one of us and the individuality and the unique combination of thinking and human skills we bring to the mix.
Therefore what is scarce will change. It is no longer production, and it is no longer reach, because those are now available to everyone. The scarce thing will be the ability to be the signal people actually choose to listen to in a sea of sameness. That is what does not multiply at the touch of a button, and where advantage relocates to.
What links a lottery, an ancient Greek word, and a formula for luck?
The three random things we just explored are not that random after all. Each one exposes a place where a company’s AI strategy may be more fragile than initial appearances suggest.
You buy AI to plug the gaps in human judgement. To take the uncertainty out of the decisions that keep you up at night. It is meant to reduce the uncertainty surrounding human-based decisions in business.
Except….not so fast:
Stuck choosing between two near-identical candidates, or two equally good investments for the budget? Use AI to strip out the bias and make the call. Except it encodes the bias instead, and wraps it into a data point that no one feels qualified to question.
Worried we are seeing patterns that are not really there? Use AI to surface the real ones. Except it sees phantom patterns too, and reports them with more confidence than any human would dare.
Hungry to manufacture more luck, more reach for your products and brand? Use AI to produce more and get in front of more people. Except everyone else does exactly the same, the noise rises for all of us, and being heard gets harder, not easier.
In every case we reach for AI to do the same thing: remove the uncertainty. And in every case the uncertainty does not actually leave. It gets hidden, automated, and relabelled as objectivity, which is far more dangerous than leaving it in plain sight.
Perhaps the antidote is not minimising uncertainty but to accept it - and even to use it.
I leave you with the thought that uncertainty is good for business, an idea we revisit in next week’s essay.
Intrigued by randomness? Then you may want to explore Matt and Nick’s book:
“In Random, Matt Ballantine and Nick Drage explore how randomness and uncertainty benefit our lives and suggest strategies for making the best use of randomness and managing those situations where uncertainty is unavoidable.”
Next week’s essay: Knightian uncertainty. Frank Knight, the economist and author of “Risk, Uncertainty, and Profit” and on the value of uncertainty for business.