In the first post of this series, I laid out a set of principles for standing up a truly differentiating analytics capability in an enterprise analytics Center of Excellence:
- Business driven not exploratory analytics
- Investing in analytics as a portfolio
- Agile methods for analytics
- Creating operational strategies for every analysis
- Coupling analytics and reporting to drive data democratization
- Making collaboration a core capability not an add-on
- Creating virtuous cycles inside analytics
I’ve covered the first two (sorry about missing last week – it’s really busy around here in the here pre-Christmas rush both at EY and on the family side. The combination makes for a tough blogging environment!) and today I’m going to dive into using Agile methods in analytics.
Agile is a programming methodology that’s become very popular in the last ten years – particularly in the digital realm and with commercial development shops. Since my background is in programming, I’ve had a first-hand view of the long history around development methodologies and the rise in popularity of Agile. So I’ll start with a quick and probably grossly oversimplified look at the last forty years of development methodologies.
Back when the personal computer revolution shook up the world and brought a flood of enthusiastic and entirely un-educated (in programming – we were otherwise a fairly well educated bunch) developers into the business, methodologies were almost entirely lacking. A huge amount of development was done by lone individuals. That worked okay, because the project’s codebases were much smaller. Software systems lacked much in the way of GUIs, memory was tight, and programs were in their first generation or two. Smaller codebases make it possible for a single programmer to hold nearly all the code in their heads at one time.
With the introduction of graphical interfaces and the evolution of projects into larger, 2nd and 3rd generation efforts, things changed. First, the field was inundated, greatly lowering the overall quality of developers. Star developers became rare and harder to find. But it’s also true that projects simply got too large for star developers to execute them individually. Even if the core code was written by one or a small group of individuals, all of the ancillary code was written by teams. The move to teams created demand for better documentation and for more structure in the programming languages that allowed teams to work more effectively.
As systems evolved and IT became increasingly commoditized, these trends continued. The move to offshoring further drove the demand for rigid, detailed, exhaustive requirements.
In theory, these trends created more efficiency in development, fewer mistakes, and the ability to leverage lower-cost resources. In fact, they combined to make programming cycle-times impossibly long and programmers totally un-responsive to and divorced from actual user needs.
Enterprise IT organizations might live with that brand of efficiency but commercial developers can’t afford it – and with the growth in digital systems where cycle-time became of paramount interest, development methodologies began to evolve in more interesting fashions. That’s where Agile comes in.
The idea behind Agile is fairly simple. Instead of a huge design process followed by a massive coding and debugging phase, Agile breaks up the delivery of software into much smaller chunks. Each chunk is designed to have a very short cycle time (sometimes as short as a single week) and simple, clear, prioritized functional requirements based on easily understood and user-focused stories. User journies are divvied up into teams with each team being responsible for delivering the story and the team typically consisting of all the resources necessary to execute their tasks.
Agile is designed to create very fast cycle times to make sure that programmers are staying on schedule and on task. The team structure is designed to create broader knowledge sharing and to allow for the rapid communications necessary to such quick turnarounds. It’s also built around the concept that by delivering incremental chunks the design can be tweaked and tuned as testers and users actually experiment with the project. Instead of massive requirements documents, it’s driven by relatively straightforward user-stories – making it extremely business focused. By doing this, it creates the opportunity for virtuous feedback loops in the development cycle – something that’s turned out to be absolutely critical in digital systems.
The application of Agile to analytics in a CoE is surprisingly natural. Many of the parallels are obvious. Like development, analytics has been largely star driven – with single individuals often carrying the entire analytics load for a large enterprise. We seen a tremendous influx of new people and the introduction of offshoring. We’ve also seen dramatic growth in the complexity of analytics as we move from SaaS and traditional stats tools to big data platforms. Big data platforms not only create more complex projects, they create more diverse rolls as data manipulation and programming become important skills in the analyst’s toolbox.
For the CoE, therefore, we recommend a heavy dose of Agile. Keeping analytics projects small with rapid turn-arounds and multiple cycles is a huge advantage. It ensures much faster time to value and delivers, over the same period of time, deeper and more business responsive analysis. Creating small teams with key skills distributed lowers the bar on specific individual expertise (making it easier to avoid the data scientist trap), and creates rich opportunities for sharing, mentoring, and skill-building.
Unlike most “me-too” applications of methodology, Agile actually works even better in analytics than in development. There’s a couple of reasons why. First, analytics, even more than programming, rewards knowledge sharing and team building. Creating small team environments where new analysts can be mentored in hands-on application of analysis is a huge win – far bigger, in my opinion, than in programming. It also creates a natural method of integrating consulting expertise with in-house expertise. That’s less important in the very mature world of software development than in the still maturing universe of analytics.
I also think that big-data analytics (which is often the norm in a CoE) provides more differentiated roles than comparable development projects. The simultaneous need for closely collaborative but clearly differentiated roles is a major drive of Agile and is the situation where its advantages as a methodology are most apparent.
Lastly, the creation of organic, virtuous cycles of improvement is easier, at least in some cases, in analytics than in many development projects. The results of analytics projects can be fed directly into testing programs and consumed and evaluated by stakeholders. The results are often more directive than in software development. Frankly, as I look at most Agile software projects, there is relatively little use of the feedback mechanisms that the methodology offers. And, of course, the focus on user-stories is a natural extension of the business-driven, portfolio methods I’ve described in the last two posts as essential to effective analytics.
If you look back at the list of CoE principles I’ve enumerated, you can see how the first three principles are deeply entwined and create a distinct and powerful approach to analytics. Creating small, agile teams is the perfect operational counterpart to the portfolio approach to analytics. And the agile method, with its focus on tight feedback loops, specific goals, user-stories, and team execution pretty much demands the formal business-driven approach that I described as the first principle of CoE analytics.
Business-driven analytics prioritization, a rich portfolio approach to finding and executing analytics projects, and the use of Agile methods to deliver fast, team-based turn-arounds of projects combine to deliver world-class Center of Excellence analytics.
With the holiday’s upon me I expect to have some time to knock out a few more posts before year end. Next week I may continue this series or break out for a post I’ve been wanting to write on customer life-cycle and campaign frameworks. I also plan to do a traditional year-end “best of” post than captures and summarizes the blogs from the past year that I think are particularly worth reading. Heaven knows I understand how hard it is to keep up with any blog and I’m always flattered that people find time to read anything. So I figure it can’t hurt to point to a few of the really good ones that people might have missed.
Till then, have a great holiday!
