Managing Change is Managing Risk
What if the bug fix was for the Web portal which fed information to the ERP system, and the maintenance of the electrical generator also included a switch over to a backup system that shut down the data center for a few minutes? What if all of these changes were scheduled on the same day, at the same time, and the teams associated with these changes did not know about the other events?
If this collision occurred and caused the inevitable production outage, it could be a real disaster for the CIO.
Many organizations adopt ITIL (information technology infrastructure library) for communicating and identifying environmental changes. It does not matter if your business is small, medium or large; as mishandled changes, regardless of size, duration, or complexity, can drastically hurt any size company. What follows are several examples where miscommunicated production changes seriously impacted an organization, and in one case, was a factor in the dissolution of a company.
The first example is a university, where five significant Web outages were experienced within a four week period. The technology group did not have a formal change management process in place and was unaware of simultaneous implementations and the disastrous collisions: online courses became unaccessible, grades were not posted, and students became, as you would expect, irate.
Citing older technology as the issue, the CIO was able to convince management to make a sizable investment in a new hardware platform. The technology group then embarked on a project to manage future outages, including the creation of a hot-site data center, a communication plan to notify users of an outage and implemented ITILs change management. Even though outages are now a rare occurrence, the CIO and his team lost political capital and reputation of the IT group declined substantially.
Small Business to No Business
Next, we look at a small retail company that suffered for years with up to 20 Web outages per day. Each of these outages impacted its ecommerce site and revenue. Many of the outages were attributed to older hardware, bad coding practices and frequent implementation collisions between the application and infrastructure teams. Using a three-pronged approach, the company purchased new hardware, fixed the application and introduced ITIL change management.
Unfortunately, it was all a little too late: customer satisfaction levels were already low and the company's revenues continued to decline with the ongoing loss of clients. These outages were clearly a factor in the ultimate demise of the company.
Big Business Works
A nationwide insurance company, having difficulty managing over 1M changes per year successfully implemented ITIL change management, complete with release and configuration management. The company used a top-down approach, with ongoing support from the CEO, CFO and CIO and implemented the processes across the entire organization, which included seven divisions and over 30,000 users. The organization is now able to effectively manage changes through the ITIL process and has increased the success rate for production implementations.
A major retailer, trying to upgrade their Web platform, experienced a disruption that lasted over a week. Used as the primary sales channel, this outage resulted in a $750,000 loss to the company's bottom line. Using a SWOT analysis to determine the root cause, it was determined that the outage was caused by a lack of communication between IT support teams that led to an inappropriate number of staff on hand to support the upgrade. Implementing ITIL led to a higher level of communication between the application teams and reduced the number of production outages.
Lastly, IT employees at a nationwide healthcare provider inconsistently used the change management process, and saw a direct correlation between poor communication and production outages. These outages stopped transmissions between the remote sites and home office systems and led to patient information becoming corrupted and lost. The CIO stepped in, demanded the change management process be used 100% of the time, and the system disruptions subsided rapidly.Unfortunately, there are thousands of stories such as these, and many of these issues could be minimized or avoided, if change management was used. In each of these companies and situations, using a simple method of communicating changes may not have circumvented the disruptions, but it definitely would have minimized the downtime.
All businesses experience periodic disruptions and when they occur, CIOs must rapidly respond. While utilizing change management doesnt guarantee an elimination of problems, it definitely helps to avoid them. In preparing for this paper, Internet searches brought up many sites where outages were posted and resolutions sought. This research shows that many companies now know the importance of a robust change management process.
When considering the implementation of a change management process, there is one key principle to follow: make the process easy to use and IT resources will adapt quickly. Having been personally involved in many ITIL change management implementations, I can share the following lessons with you:
- Announce your expectations and support for the implementation of change management.
- Put the effort into learning about ITIL and spend the time and money to train the IT team.
- Look for those individuals that are eager to get started and give them the authority to build a process that works for the organization.
- Designate a change management leader: someone who can champion, sell and win support for the process.
- If your organization is large, create a change management department to coordinate the changes impacting all production systems.
- There are many software packages on the market that provide change management functionality. Find one that is easy to use and supports wide-scale changes.
- Eliminate bureaucracy and let the people that build the systems, support the changes at the change meeting.
- Consider implementing release schedules to make the change management process even smoother. Remember to keep it simple.
- Change management is about communicating changes to avoid disruptions and to increase customer satisfaction levels.
There is much more information that can be shared on this topic but this article provides basic information on how to get started. Form information visit the official ITIL site or CIOUpdate's sister site ITSMWatch.com. They are great reference sites for the different approaches to service level management and IT service management.
Sue Bergamo is the former CIO at Aramarks WearGuard & Galls companies. She can be reached at firstname.lastname@example.org.