Why IT Service Level Management Fails (And How to Fix It)

By Hank Marquis

(Back to article)

IT managers think IT services are different from other services. Not so. To deliver any type of service follows a few identical processes. Not understanding this is the reason for much of the pain and suffering born by IT providers and their users today.

Consider: would the owner of a quick oil-change company measure his or her service quality in terms of oil viscosity, oil splashed (or not) on the floor, choosing the right oil filter, assembling the rubber O-ring correctly, dispensing the right amount of oil, and remembering to put the oil cap and plug back in? Of course not, yet this is precisely how the average IT service level agreement (SLA) reads.

Oil change franchises measure their quality based on getting the job done fast and right. But ask yourself one question—what is the job? Oil change franchises don’t market fast oil changes. According to the websites of several top franchises they market “preserving the health and value of your vehicle” and “keep your car running well and running longer.”

What are you measuring with your SLA? Does your customer care about whatever you are measuring? Do you know what your job is or what your customers really want?

The average quick oil change service focuses on getting the consumer in and out so they can do what they really want to do. For these customers, an oil change is the means to an end. He or She needs their car to operate so they can get where they need to go. We in IT need to realize that we too are nothing more than the means to an end, and that end purpose is what we need to measure to determine IT service quality; not the bits, bytes, speeds, feeds and tasks it takes to deliver a service. Of course, monitoring the physical properties of the hardware and software are critical to IT service delivery, just as is the viscosity of the oil and remembering to put the oil cap on.

However, customers expect these things, and these things are not the primary measure of customer satisfaction. They are not even very interesting to customers. One can even argue that customers pay us (service providers) a premium to shield them from all that complexity in the first place. What they really want is to get where they are going more quickly and easily. That is, they are paying us so they can do something else better.

The oil change providers measure of quality is customer satisfaction with regard to the right job, done quickly, because this is what their customers really want. No one wants on oil change for the sake of an oil change. Yet, IT often pays only lip service to customer satisfaction, and does a haphazard job at best in trying to ascertain it. Fewer still use customer satisfaction as its primary service delivery quality measure for service level management (SLM.) The reason is not that there are other more successful SLM methods, but more often it’s fear.

SLM could and should be the voice and means for cost controls, quality improvements and competitive advantage. Yet there are too many failed SLM initiatives to count in the IT workplace, and the cost of these failures is large because it includes not only money wasted on countless software solutions, work hours and loss of focus, but also because of the opportunities lost. The problem is that services are not products. This single realization is the key to effective SLM.

Products are durable goods. The ways to measure product quality are many, and perhaps best described by Garvin’s 8 dimensions of (durable) product quality: Performance, Features, Reliability, Conformance, Durability, Serviceability, Aesthetics and Perceived Quality.

Qualities of products are also easily measured. For example, reliability can equate to “uptime” and serviceability can relate to “mean time to restore service”, and so on. Moreover, this is exactly what we see in SLA—the “operationalization” of metrics surrounding the qualities of physical goods used to create and deliver IT services. And this is precisely what causes SLA efforts to crash and burn.

Any SLA that attempts to operate in this manner is doomed because services are, as I elaborate below, intangible, heterogeneous, perishable, and inseparable:

·      Intangibility - IT services are performances not objects: services cannot be inventoried, patented, readily displayed or communicated, and pricing is difficult.

·      Inseparability - IT services are simultaneously created and consumed: customers participate in and affect the transaction; customers affect each other; employees affect the service outcome; decentralization may be essential; and mass production is difficult.

·      Perishability - IT services have no buffer on supply: it is difficult to synchronize supply and demand with services; services cannot be returned or resold; cannot rework services; and impressions are lasting.

·      Heterogeneousity - Every IT service and service encounter is unique: unique constraints, resources, and requirements.

The unique characteristics of IT services means that IT service quality may only be measured external to the IT organization in the form of satisfaction of the customer with the service after they consume it.

For an SLA to be effective it must address not only the mechanics of the systems performance, features, reliability and so on, but also the perception of quality. This is the root of most SLA failures. In large part, the reason for failures with ITIL, BSM and SLM occur because IT staff and management think “operationally” with regard to the bits and pieces of hardware and software they know comprise an IT service. Customers and users are mostly unaware of these intricate details and simply consider the IT service the means to an end.

IT and customers usually have two very divergent views about how to measure service quality. Arguably, they only view that matters is the one held by the customer, since IT exists solely to accelerate the work of the enterprise. The reason customers outsource IT service delivery to IT in the first place is to reduce risk and improve utility. Customers consume IT services for one and only one reason, and that is to assist in selling, supporting or delivering enterprise products to end-customers.

Thus, the true measure of IT service quality, and therefore what SLM ought to be measuring and reporting to customers, is not the operational characteristics of physical products, but rather the ability of the user to sell, support and deliver enterprise products. As mentioned, customers have specifically paid us to mask them from internal production details so they can sell, support and deliver without distraction.

How to Measure Quality

Measuring these abilities requires a fundamental shift in thinking on the part of IT managers. From behind the IT curtain, IT services are composed of durable products, but that is not how consumers perceive and consume IT services. Instead of the durable product model of Garvin, a better model is SERVQUAL and its five dimensions of service quality: Tangibles, Reliability, Responsiveness, Assurance and Empathy. The research behind SERVQUAL has substantiated these dimensions across many industries. The primary means to obtain the values for these dimensions is a specially constructed survey instrument.

The benefit of this approach is that it does not tell customers how IT is doing, but rather reflects how customers say IT is doing. This is a very import customer-centric change in approach. By its very operation, SERVQUAL requires participation of customers and users on a regular basis. When combined with traditional “speeds and feeds” type metrics used “behind the curtain” SERVQUAL can transform the business IT relationship. Isn’t that the goal of BSM in the first place?

SERVQUAL is a continuous quality improvement tool designed for the service industry and first described in the late 1980’s. It provides a model for improving service delivery that directly addresses the root of poor service quality. SERVQUAL starts with an analysis of customer expectation as the basic formula. It defines service quality (q) as:

service (q)uality = (e)xpectation – (p)erception

Figure 1. Service Quality Formula

Perception is reality with regard to service performance, and “q=e-p” is the “e=mc2” of every service industry except IT. The larger the negative gap between expectation and perception, the poorer the service quality. Higher positive values indicate meeting or exceeding customer requirements. Too high and you have identified an opportunity to cut costs or rebalance resources.

Good as that sounds, right away the average IT manager may start to feel itchy in that SERVQUAL seeks to determine customer needs and wants (expectations) first, and then asks customers to determine how closely the service meets their needs. Many in IT feel very uncomfortable with the idea of measuring their performance based on subjective analysis, and from customers and users of all people! The truth of course is that “it takes a village” to deliver quality service and measuring individual elements can only loosely approximate service quality, and can never measure customer satisfaction.

Now you can see why some individuals in IT think speeds and feeds are the thing to measure. They have some direct control over them. However, it takes a well-managed team to effectively deliver IT services, and this what is so frightening to many in IT an organization notorious for its “gun slinger hero” mentality.

SERVQUAL measures precisely the gaps in team workflow responsible for failing to deliver a quality service. SERVQUAL is a proven model, and even with its inevitable detractors, SERVQUAL remains the de-facto standard for measuring the quality of services.

Quality is precisely what your customers tell you it is, and SERVQUAL lets you know loud and clear what your customers say. SERVQUAL goes beyond describing how to measure quality to include a built-in gap analysis model that shows where the supplier organization induces the poor quality. SERVQUAL defines five (5) gaps and aligns very nicely with ITIL, Six Sigma, CMMI and most quality and customer oriented management frameworks required to succeed with BSM. SERVQUAL includes four (4) internal gaps and one (1) external gap.

Enter the Zone of Tolerance

SERVQAL gathers three data sets: minimum, desired, and perceived service quality. These three data sets combine to create a “zone of tolerance.” Think of the zone of tolerance as the boundaries representing the upper (desired) and lower (minimum) boundaries of service quality. Plotting the third data set, perceived quality, in relation to the zone of tolerance lets IT suppliers and consumers instantly visualize service quality. It also shows where to improve, and can function as a continuous improvement model.

Consider the image in Figure 2 below. It shows how to use SERVQUAL to show IT performance based on the job IT needs to do as measured by customer requirements.


Figure 2. Example SERVQAUL SLM Report


Figure 2 makes it is very easy to understand IT performance. Even non-technical business managers will have no problem at all understanding IT performance. There is no translation from speeds, feeds, MIPS and so on into human terms. The value of the measure means nothing; it’s the relationship of the measure to the boundaries that is meaningful.

SERVQUAL offers two metrics: measure of service adequacy (MSA) and measure of service superiority (MSS.) MSA represents perceived quality less adequate quality while MSS perceived quality less desired quality. These metrics are internal to the provider. MSS > 0 means the provider is over-servicing based on requirements. MSA < 0 means the provider is under-servicing, as shown in figure 3.


Figure 3. MSA and MSS Scores


This model drives IT operations based on customer and business requirements. Using MSS and MSA values help the IT service provider prioritize work, balance resources and select improvement targets. Select for improvement those services not operating within the zone. Initiate corrective action for services moving down and out of the zone. Market services that move up in the zone. Stop investing when balanced in the zone.

This model also provides a roadmap for improvement and remediation. The four SERVQUAL gaps identify where poor service quality (Gap #5) originates. It also shows where you don’t have to focus—an important and often overlooked item.

Using Figure 3 as an example, it would seem we probably need to focus on Assurance, defined as “knowledge and courtesy of employees and their ability to inspire trust and confidence,” and Reliability or “ability to perform the promised service dependably and accurately.” Managers can quickly see they ought to reallocate and balance resources, perhaps moving funding or resources from “Tangibles” to “Assurance.”

Based on MSS and MSA values, examine Gaps 1 to 4. Shrinking some gaps requires training, others usually require software support tools, and others require modifications to process. The resolution depends on the gap, and the tools required depend on the resolution. Here are some examples viewed from the perspective of the service supplier. Gaps 1 to 4 reflect where in the supplier organization the poor quality arises. Gap 5 is the “Service Quality Gap” measured as q=e-p.

SERVQUAL not only provides a roadmap to measure and improve service quality, it also provides a model to present service quality metrics, and a means for quantifying and prioritizing supplier operations and improvement projects.

Figure 4. SERVQAUL Mapping to ITIL v3

Gap #1 – The “Market Information Gap”

Gap 1 occurs when there is a discontinuity between customer expectations and managements understanding of customers expectations. Reasons here include insufficient research into, or understanding of, customer needs; inadequate use of the research; lack of interaction between management and customers; insufficient communication between staff and managers; etc.

Resolutions include conducting research, making senior IT managers interact with customers, making senior managers occasionally perform customer contact roles, encouraging upward communication from customer contact employees, and so on. ITIL, Six Sigma and other solution sets can help here, along with good old-fashioned attention to customers. ITIL v3 Service Strategy directly addresses this gap.

Gap #2 - The “Service Standards Gap”

Gap 2 arises within the provider organization when there is a misunderstanding between management perceptions of customers’ expectations, and service quality specifications used by provider staff. Causes of this gap include: inadequate management commitment to service quality, absence of formal process for setting service quality goals, inadequate standardization of tasks, and a perception of infeasibility or that customer expectations cannot be met.

Gap 2 resolutions include using tools like CMMI and ITIL to define process, clarify roles, and to document and measure service delivery goals and performances. ITIL v3 Service Strategy, and the SLP (Service Level Package) passes to Service Design, and the SDP (Service Design Package) Service Design passes to Service Transition directly address this gap.

Gap #3 - The “Service Performance Gap”

This Gap appears between service quality specifications and service delivery. Key factors here are lack of teamwork, poor employee/job fit, poor technology/job fit, lack of perceived control by contact personnel, inappropriate evaluation and compensation systems, role conflict and ambiguity among contact employees.

Ways to address Gap 3 include: investing in employee training, supporting employees with appropriate technology and information systems, giving customer-contact employees sufficient flexibility to respond, reducing role conflict and ambiguity, recognizing and rewarding employees who deliver superior service. ITIL v3 Service Transition, and the early life support provided to Service Operation directly addresses this gap.

Gap #4 - The “Internal Communications Gap”

Gap 4 comes from miscommunications between actual service delivery and external communications about service delivery to customers. Gap 4 occurs with inadequate communication between supplier and consumer; inadequate communication between advertising or service descriptions and what operations can actually deliver; differences in policies and procedures across branches or departments; and the general tendency to “over promise and under deliver.” Aside from resisting the temptation to “say anything” to please customers, Gap 4 resolution requires effective internal and external communications. An IT service catalog and a Service Portfolio Management effort go a long way to eliminating this gap. Service Strategy, and Continual Service Improvement address this gap.

Gap #5 - The “Service Perception Gap”

Gap 4 occurs when the service provided does not meet customer requirements for any reason. Service Operation directly addresses this gap.


Most IT managers would love to remove “99.99%”, “two minutes to repair”, “2.5 second response” and all the like from their SLA’s. Customers hate this techno-babble, too. They have no idea what the mumbo-jumbo means, but over the years, IT has trained them to think that is how you manage IT. The problem is neither group knows how to do a better job. The solution is to operate and deliver IT services within the SERVQUAL zone of tolerance.

SERVQUAL positions IT to finally achieve not only SLM, but also BSM. It offers a new way to measure IT, and at the same time, it provides a self-correcting system for knowing what to do, how much to do (when to stop), and why to do it—all in business terms. SERVQUAL presents a framework for utilizing all the other IT tools including CMMI, ITIL, PMI, etc. Finally, a framework (SERVQUAL) for choosing and using other tools, a framework for using the other frameworks BSM requires … Brilliant.

Oh, and it this is exactly what it takes to improve IT service quality, align with business and control costs, in other words, it just might be how to “do BSM.”

Most recently, Hank Marquis was the director of IT Service Management Consulting at Enterprise Management Associates based in Boulder, Colo. Marquis has more than 25 years of hands-on experience in IT operations, management, governance and operational frameworks. Visit his blog and podcasts at www.hankmarquis.info.