The High Cost of Expertise'

By Brian Cook

(Back to article)

It is a widely held belief that business analysts (BAs) must have a number of years experience in a domain they are to be discovering and documenting requirements in to be effective in that domain. It turns out that that assumption is usually false.

An IAG study, Business Analyst Benchmark – 2008, identifies the following types of business objectives from the least risky to the most risky with the average failure rate for 68% of companies without high BA skills:

  1. Regulatory compliance (27%)
  2. Information and access to information (41%)
  3. Communications with employees, suppliers or customers (46%)
  4. Launch new products (47%)
  5. Reduce direct costs (59%)
  6. Significantly re-engineering the business processes surrounding application (67%)
  7. Reduce transaction and indirect costs (68%)

Failure Rate Correlations

The rate of failure correlates to the type of the project without regard to whether the BA is a business domain SME or not. In other words, using a BA who is a SME in that domain on a regulatory compliance project would fail on average 27% of the time. However, using the same BA with the same SME knowledge in the same domain would fail an average 59% on a project to reduce direct costs. Conclusion: being a SME does not help the BA reduce risk for this type of project.

IAG defines two approaches for BAs to mitigate risk: they may rely on personal subject expertise; but must rely on professional requirements discovery techniques.

Personal subject matter expertise can be effective for the first two business objective types, regulatory compliance and data access. Professional requirements skills expertise should be used to mitigate risk for business the other five objectives, types 3 – 7. A BA SME can be advantageous for only the two least risky of the seven types of business objectives.

When a BA is too familiar with a domain the BA will share the same thought processes and concepts as the rest of the team for better or for worse. However, a BA with expert level skills who is new to the domain is much more likely to ask the “dumb” question that will uncover misaligned assumptions early on when they can be addressed at the lowest cost.

Additionally, expert level BAs are much more likely to enable development of an innovative project that delights stakeholders and helps the company succeed in the marketplace. Their projects result in higher customer satisfaction.

Financial Impact

IAG states that:

Companies using best requirements practices will estimate a project at $3 million and better than half the time will spend $3 million on that project. Including all failures, scope creep, and mistakes across the entire portfolio of projects, this group will spend, on average, $3.63 million per project.

Companies using poor requirements practices will estimate a project at $3 million and will be on budget less than 20% of the time. Fifty percent of time, the overrun on the project both in time and budget will be massive. Across the entire portfolio of successes and failures, this group with poor requirements practices will, on average, pay $5.87 million per project.

Companies without high level requirement skills spend $2.24 million more per project than companies with high level requirement skills. Unfortunately, many companies recruit BAs primarily because of their business domain knowledge and use the same BAs for all projects. Those companies pay a heavy price for that decision.

Case Study

At a major global company some time ago, executives determined that significant growth could be achieved by cross-selling services to existing customers. The service was one of the most profitable in the entire company. Over a two year period, a development team had tried to make this happen but it just wasn’t working because of the wrong understanding of what was needed to cross-sell properly.

While using a simulation approach (see previous CIOUpdate article, Using the Right Medium Gets Executive Buy In), the business subject matter expert (SME) for the project simulated a related part of the department’s process and was presenting it in a group meeting with both business and IT representatives. As the business SME was walking through the process under discussion, he stopped with an “Aha!” look on his face and said “I know why we have not been able to link [the service].” He then explained his insight to the group.

As a direct result of using interactive visual simulation, the root misconception that had plagued the team for two years—connecting to the wrong use-case—became clear. Additionally, a solution also became clear immediately. Everyone in attendance agreed they could now write accurate requirements and get this functionality into production quickly. They would then be able to cross sell this highly profitable service.

Consider: What was the cost of the failed attempts? How much revenue was lost due to missed cross selling for two years? How large was the opportunity cost due to the benefit stream from other ideas that could not be implemented as a result of not achieving this functionality the first time?

While we do not have enough information to calculate real numbers, we can conclude that these three items combined represented a significant negative impact to the bottom line for this company.

So, why did simulation enable the discovery of this insight? The main people involved in this project were the same over the two years. The insight was not the result of adding a new person. I was invited to participate in this project because the program manager believed that a simulation approach would help this stalled project. I trained and mentored the team in the approach and the tool we used.

However, and this is significant, I did not have any detailed prior knowledge about either the application this team was developing or the service they wanted to cross sell. I had a 50,000 foot view of the department, but that was all. Additionally, on the day of this “Aha” moment, I did not know about the two year history which I learned about afterwards. I emphasize my lack of prior knowledge in this domain because I will shortly challenge a widely held assumption about business analysts (BA).

Let’s go back to the SME’s “Aha” moment. The SME discovered a new way to connect the cross selling opportunity on his own. You could argue that the SME had all the facts already and you would be right. However, the way those facts were organized, as industry standard textual system requirement specifications, had not help the SME to connect the dots.

On the other hand, the new approach enabled the SME to quickly see those very same facts in a new light. The simulation itself facilitated the “Aha” moment by visually connecting previously unconnected or wrongly connected facts. So, this is why I’m challenging the assumption that BAs must be domain SMEs to be effective.


Cross-functional BA teams organized into centers of excellence are more effective in documenting requirements than SMEs documenting requirements. The IAG study states, “The idea of a center of excellence for business requirements is gaining momentum particularly for larger companies with a need to deal with complex projects. In the absence of this structure it is harder to manage the requisite competency base of the corporation, and optimize the use of elite analysts across the enterprise.”

The ability to move across the enterprise works because subject matter expertise is not a statistically significant predictor of project success for business objective types 3-7. However, having superior competency in BA domain skills is statistically significant. In other words, a BA must have a certain level of requirements development domain skill to be successful. Higher levels of skill are needed as the risk type of the project increases.

To manage risk, companies should develop or bring in elite analysts to succeed in higher risk project types. Failure to use rightly skilled BAs will result in high failure rates. The center of excellence approach can provide skilled resources to assess the risk of a project using the scale above and to supply elite analysts to engage in higher risk projects.

One of the burning organizational questions in many companies is “Do BAs report to the business or to IT?” Once the decision to form a center of excellence is made, the IAG findings are clear – the center of excellence should report to both the business and IT. An organization with a jointly owned center of excellence significantly outperforms an organization where requirements are the responsibility of only one group no matter which one it is.

BAs can rely on subject matter expertise for only projects with the least risky types of business objectives. However, projects seldom have only one business objective. Rate the relative importance of each relevant objective to determine the overall project risk and determine which skill level the BAs should have. If the optimal expertise is not available in-house, consider getting outside resources.

Ideally, in-house BAs should be organized into a cross functional center of excellence that is jointly owned by the business and IT. The center of excellence should focus on improving the skill level of in-house BAs by training and hiring.

The rewards for using rightly skilled BAs are high: over $2 million per project. It is an understatement to say that you could reap significant financial benefits by rethinking how your company discovers and documents requirements.

Brian Cook was a Senior Software and Systems Architect for Zurich Insurance for six years and prior to that an Information Architect, Information Modeler and trainer for Perot Systems for four years. Currently, Brian is President of Cook Enterprise Corporation whose primary mission is to help clients discover, document, simulate and validate processes, scope and requirements visually so they are understandable to non-technical people. A description of the Building Requirements Consensus Methodology can be found at Brian’s website: www.building-requirements-consensus.com.

“If a picture is worth a thousand words, a simulation is worth a thousand pictures.”