Testing and Learning for Customer Experience Excellence


Share on LinkedIn

On any given day I find myself playing the You know you’re old when… game. Here’s how it goes for me on broader social themes, You know you’re old when you have to explain to someone what a pay phone was or when you reflect back on the Sears catalog.

From a customer experience perspective, that game takes the form of You know you’re old when something you used to call ‘test and learn’ is now the foundation for Agile Design or when High levels of human service delivery are looked at as a retro approach to customer experience strategy.

Of course, I know I am old whenever I use the term customer experience because I was working in customer service improvement before Gilmore and Pine gained traction for that term thanks to their breakthrough book The Experience Economy. Now, I’ll stop talking about agedness and pivot to elements of customer experience improvement which have transcended place and time.

Some leaders at companies like Capital One have been using test and learn approaches for product development, service process improvements, and even human service delivery enhancements dating back to the late 1980s.

In the early days of my career, companies that I worked with, like Starbucks, were actively deploying test and learn approaches for all aspects of the customer experience.

Many elements in test and learns fail in their initial presentation but frequently evolve into breakthrough successes through continued iterative design.

For example, in 1994, a cold, lightly-carbonated, bottled, coffee beverage named Mazagran had a less than successful consumer response.

Howard Schultz, then CEO of Starbucks, put it this way in his book, Pour Your Heart In It:

When we test-marketed it in southern California…it polarized people. Some loved it, others hated it. A lot of customers were willing to give it a try because of the Starbucks brand name, but Mazagran didn’t get the repeat business we had hoped for…So we kept pushing until, in 1995, we found a better approach. Frappuccino had been a surprise hit that summer, drawing in tens of thousands of customers who were not normally coffee drinkers, filling our stores in afternoons and in hot months when the coffee business is usually slow.

Howard put carbonated coffee on hold for a time and instead focused on the bottled element of Mazagran – thus putting the Frappuccino in a bottle in partnership with Pepsi. The rest is history since bottled Frappuccino was a home run for the brand and the consumer.

Thanks to lessons learned from Mazagran, Starbucks entered the grab and go market making bottled Frappuccino available across a wide footprint of retail outlets (its own stores, supermarkets, convenience stores, etc.).

Fast forward to 2014, Howard and the product innovation team leveraged carbonated beverages (albeit not bottled) via its Fizzio Handcrafted Sodas. Then fast forward again to 2018 (through ongoing efforts traceable to 1994) the brand had iterated yet again to deliver Cold Foam Dark Cocoa nitro (again not bottled but a cold caffeinated brew that gets its effervescence from nitrogen as opposed to carbon dioxide).

Granted the methodologies of agile design are far more formal than primitive test and learn strategies we used in the late ’80s/early ’90s, but all these approaches share similarities including:

Listen to your consumer





Listen for more opportunities to improve

How are you doing with your exploration and evolution of customer solutions? How effectively are you deploying agile design or test and learn approaches to help you develop product or service delivery success?

My team and I have been doing this for a while and we’re here to help. Simply reach out to us and we’ll find a time talk.

Republished with author's permission from original post.

Joseph Michelli, Ph.D.
Joseph Michelli, Ph.D., an organizational consultant and the chief experience officer of The Michelli Experience, authored The New Gold Standard: 5 Leadership Principles for Creating a Legendary Customer Experience Courtesy of The Ritz-Carlton Hotel Company and the best-selling The Starbucks Experience: 5 Principles for Turning Ordinary Into Extraordinary.


Please use comments to add value to the discussion. Maximum one link to an educational blog post or article. We will NOT PUBLISH brief comments like "good post," comments that mainly promote links, or comments with links to companies, products, or services.

Please enter your comment!
Please enter your name here