In 2015, I wrote about why organisations struggle to make experimentation part of everyday work. People improve through practice, yet businesses often give teams neither the budget nor the permission to practise.

More than a decade later, we are still having that conversation.

The tools have improved enormously. We can build in an afternoon things that would once have taken weeks. But getting permission to test an idea, finding customers to test it with, and acting on an inconvenient result can still take longer than building the prototype.

That should bother us.

In a webinar I did with Steven Cohn in 2016, we highlighted two common problems: overengineering a prototype and trying to use one prototype to validate several assumptions. Ten years on, AI has made it much easier to produce an elaborate prototype before anyone has established whether the idea is worth pursuing.

We have become much faster at building things we are reluctant to reconsider.

Part of the problem is how we fund the work. A team spends weeks preparing its experiment. They build the prototype, agree the success criteria, recruit participants and schedule the presentation to leadership. By the time anyone sees a result, there is quite a lot riding on it.

A disappointing finding means another round of approvals, assuming there is any budget left. We have made the first experiment so expensive that we can barely afford to learn from it.

Imagine a team exploring an AI assistant for customer service. They could spend months building something that handles a conversation from beginning to end. If customers reject it, the result will be difficult to interpret. Perhaps they disliked the answers. Perhaps they wanted a person. Perhaps the assistant was solving a problem they didn’t have.

A series of smaller tests gives the team somewhere to go with each finding.

They might first examine whether the assistant can find the information an experienced agent needs. Then test whether agents can use that information during a call. The second test may show that technically correct answers are too long to be useful. That gives the team a specific change to make and something concrete to test again.

The later experiments are better informed because the earlier ones happened. Spending longer planning the first one cannot give you information that only becomes available when someone tries it.

This is familiar territory. We have been discussing these principles for years. The difficult part has been making them survive contact with annual budgets, approval processes and the expectation that a competent team should already know the answer.

AI gives us another opportunity to address that. As prototypes become cheaper, businesses can afford to explore more possibilities before committing. They will need to change how they organise the work to take advantage of it.

There is still discipline involved. Before running a test, the team should know what it is investigating, what evidence would change its mind and what decision the result will inform. Repeating a weak test twenty times will not rescue it. Neither will changing the definition of success whenever the findings are inconvenient.

Leaders can make a practical difference by funding a sequence of tests, with decisions between them. Give the team room to respond to an unexpected finding without rebuilding the entire business case. Ask what changed in their thinking and what they intend to investigate next.

After a decade of talking about experimentation, I would like more teams to be able to return from a test with disappointing news and still have the time, money and support to do something useful with it.