Recent high-profile production outages and security breaches have drawn attention to a longstanding but under-reported risk to modern business: A growing dependency on IT systems that aren’t fully under control.
It’s no secret that IT is now at the heart of many modern business models. In many cases, IT systems are the business itself—the basis for revenue and value creation. Front- and back-office processes are increasingly automated and integrated, end-to-end. Web, social and mobile applications are increasingly the frontline for brand engagement, commerce and customer experience.
Adding layers and layers of technology has made modern business more efficient, profitable … and, perhaps, more susceptible to catastrophic failures.
Much like the fictional antihero Victor Frankenstein, many organizations have unwittingly invited a level of complexity and consequence they don’t fully fathom.
Yes: It’s alive! Now what? Which leads to my question: Is IT a house of cards?
The past weeks are rich with examples of IT failures bringing business to its knees. In the two most prominent failures, manual configuration errors and outdated software are to blame. Simple oversights by human beings, unintentional conflicts created by hidden dependencies, and the fear of change all put IT systems—and therefore business operations—at risk.
Amazon EC2 experienced a multiday outage caused by a single manual configuration error that had a cascading butterfly effect across a massive network of complex interdependencies. That single configuration error crippled dozens of web-based companies running on Amazon’s infrastructure.
It could have happened to anyone. More to the point, if it happened to Amazon—one of the smartest and best run IT shops on the planet—it will happen to others.
And it does happen, all the time.
Sony Networks is the other recent example of obsolete software wreaking havoc on an IT service. In this case, failure to patch software in combination with a firewall issue exposed major security vulnerabilities, which allowed hackers to pilfer account information for more than 100 million registered users.
That’s equivalent to one-third of the population of the United States.
Today, our tolerance for flaky IT practices is stunningly high. Imagine if automobiles or airplanes were built or managed like an enterprise IT system?
Perhaps we haven’t viewed the consequence of an IT failure in the same light—defective planes and automobiles are more viscerally scary.
But what’s happening in many data centers today is pretty darn scary itself. The US government seems to agree, having recently launched a Congressional inquiry into the cause of the security breach at Sony. These are certainly new days.
The lesson here is that the current scale, scope and proximity of IT systems to modern business models demands increased management rigor. It demands automation of manual processes and a deep understanding of configuration and dependencies up and down the stack. It demands absolute assurances around what software is deployed and a model for consistently and reliably updating software and configurations that takes inherently error-prone human beings out of the critical path. It demands the full industrialization of IT practices.
Anything less makes IT—and, consequently, business itself—a house of cards.
