In a previous article on CustomerThink, I argued that most Agile backlogs are graveyards for customer value. The Decision Gap is the void between what an organization knows about its customers and what it chooses to build kills more good ideas than poor execution ever does. Teams move fast, ship efficiently, and deliver work that no one asked for. We are being Agile, but running in the wrong direction.
That article ended with a diagnosis. This one is about the treatment.
Over the past year, I have been operationalizing the Agile Customer Experience framework at Online Plastics Group (OPG), a Dutch multi-market e-commerce group with twelve webshops across nine European markets. What started as an idea that customer value must be a structural input into the decision system, not a downstream check became a working operating model. The Decision Gap narrowed. The backlog stopped being a graveyard.
The unlock was not a methodology. It was a system. And at the heart of that system sits something most CX leaders have not yet figured out how to use well: conversational AI as an interface to organizational learning.
The Real Cause of the Decision Gap
When I first wrote about the Decision Gap, I described it as a translation problem. CX research speaks one language; the Product Owner speaks another. NPS scores and journey maps do not slot neatly into RICE scoring. Customer insight loses to the Jira ticket marked URGENT.
That diagnosis is correct, but it is incomplete. The deeper cause is infrastructural.
CX insights do not disappear because Product Owners ignore them. They disappear because there is no system that captures, classifies, and surfaces them at the moment a decision is made. The marketing team learns something from an A/B test. The customer service team learns something from a complaint pattern. The SEO team learns something from a content experiment. These learnings sit in separate documents, separate databases, separate heads — and by the time the Product Owner is prioritizing the next sprint, none of them are within reach.
In firefighting terms: you can train every officer in risk assessment, but if the dispatch system cannot relay the right information to the right crew at the right moment, the training does not matter. Knowledge without infrastructure is just personal expertise. And personal expertise does not scale.
This is what we set out to fix at OPG. The full framework behind this shift including the Agile Customer Experience Loop and the role of the Decide phase is documented in my framework.
Three Roles, One Interface
We rebuilt experimentation as an operating model with three explicit roles, each addressing a specific failure mode in the old setup. The structure draws directly from the Experience Optimisation Cycle: experimentation, product development, and customer insights have to be connected, not parallel.
Notion became the knowledge bank — but with a twist. Most organizations document A/B tests. We expanded the system to hold every experiment type: sequential tests, SEO experiments, content trials, pricing tests, AI initiatives. The reasoning is simple. If only marketing experiments live in the central system, then only marketing learnings are retrievable, and the Decision Gap stays open for every other department.
Airtable remained the planning layer for A/B test workflows: status tracking, ownership, projected business impact. We did not replace it, because the failure was not in planning. The failure was in retrieval.
Claude AI became the interface. Not a reporting tool. Not an autopilot. A conversational layer between the teams and their accumulated learnings. We developed custom Claude skills that classify experiments as they are documented, analyze results when they come in, project business impact with a one-year revenue decay model, and synchronize Airtable and Notion automatically.
The infrastructural change matters more than any individual feature. Before this system, a Product Owner asking “have we ever tested anything like this?” faced a research project. After this system, they ask the question and get an answer in a paragraph, with the underlying experiments linked.
Retrieval changed from minutes of work to seconds of conversation. And when retrieval becomes effortless, CX insights stop being a “nice to have” in backlog discussions. They become the default reference point.
Where Most AI Implementations Go Wrong
I want to address something directly, because I have seen this fail more often than it has succeeded. The instinct in most organizations is to bolt AI onto existing reporting workflows. The result is faster reports, prettier dashboards, and the same Decision Gap.
AI in service of the Decision Gap requires a different architecture. Three principles guided our design.
First: AI as interface, never as oracle. Every output from Claude was validated by a human in the source systems before it influenced a decision. This was not a technical limitation. It was a design choice. Trust in AI is built through consistent validation, not through delegation. A team that delegates judgment to an AI loses the muscle to recognize when the AI is wrong.
Second: documentation as a byproduct, not a tax. The largest barrier to experimentation culture is not running tests. It is documenting them afterward. Marketers stop documenting because it takes too long; the next experiment overwrites the unfinished writeup of the last one. Our Claude skills inverted this: documentation became a conversational byproduct of running the experiment, not an additional task. Quality of documentation went up because friction went down.
Third: one repository for all experiment types. This is where most organizations fail. They build a beautiful A/B test repository and leave pricing experiments in spreadsheets, SEO tests in slide decks, and customer service learnings in email threads. The Decision Gap stays open because the system that closes it covers only one team’s work.
What Actually Changed
The operational numbers tell part of the story. Experimentation output went from 22 experiments in 2024 to 56 in 2025, with seven continuously running tests across markets. The program generated 31 roadmap-impacting insights, contributing nearly €900K in projected business value. Alongside that, we measured an estimated €100K in preventive loss — revenue preserved by stopping experiments that would have caused damage if shipped.
But the operational numbers are not the proof point I care about most. The proof point is this: roughly 21 percent of all experiment ideas came from non-marketing teams. Customer service contributed. Pricing contributed. SEO contributed. Development contributed.
That is what closing the Decision Gap actually looks like. It is not a marketing team running more tests. It is the entire organization treating customer value as a validated input into its decisions, retrievable in seconds when a decision is being made. The pattern resembles what I have previously described as decentralized CRO: experimentation as a shared capability instead of a specialist function.
During a major replatforming initiative, this turned out to be more valuable than I had expected. Every migration decision is a potential cost center, and the temptation is to rebuild what you had. With a centralized learning system, we could pull up prior experiments and avoid recreating things we had already validated as suboptimal. The Decision Gap, once closed, prevents not just bad new decisions but also the repetition of bad old ones.
Five Principles for Closing the Decision Gap
Looking back, the transition rested on five principles. I offer them here as a starting checklist for any CX leader trying to do something similar.
One: build the retrieval system before the volume. More experiments without a retrieval layer just produces more noise. Start with the question of how a Product Owner will find prior learnings in five seconds, not how to run more tests.
Two: extend beyond A/B testing from day one. Every team experiments. If your central system only captures one team’s experiments, you have built another silo, not a solution.
Three: treat AI as an interface, not as judgment. The job of the AI is to make organizational knowledge conversational. The job of the human is to validate, decide, and own the outcome. Confusing these roles produces fragility.
Four: measure preventive loss, not just upside. Counting only winners hides half the value of an experimentation system. A stopped loser is preserved revenue. Make it visible, or your CFO will undervalue the entire program.
Five: start with the decision, not with the tooling. Identify which decisions in your organization are currently made without validated customer input. That map is your real backlog. The tools are implementation choices that follow.
The Compass for the Engine
In my previous article, I wrote that Scrum is a powerful engine, but it needs a compass. Agile Customer Experience is that compass. What I have learned over the past year at OPG is that the compass alone is not enough — you also need the infrastructure to read it in real time, in the language each team speaks.
Conversational AI, used carefully, is that infrastructure. Not because it replaces human judgment, but because it makes organizational learning accessible at the moment of decision. That is the difference between a backlog as a graveyard and a backlog as a learning system.
Assumptions are still deadly. But they are no longer the only thing in the room when a decision gets made.