Enterprise application integration projects often seem equally interminable and produce questionable outcomes, with failure rates running as high as 95%, according to the Standish Group. But it doesn't have to be that way.
"Companies in the past have purchased an integration tool or solution without an enterprise-wide integration plan or strategy in place, said Michael Kuhbock, chairman of the Integration Consortium, a non-profit industry body responsible for influencing the direction of the integration industry, and CEO of K-Bear Corp.
John Schmidt, president of the Integration Consortium, has worked in IT since the 1970s. For the past dozen years he has managed major integration projects for financial, telecommunications and retail firms in North America and Europe.
"Since the 1990s, people have been on a treadmill trying to deal with the complexity of functionally siloed applications, most of which had been implemented without a master plan, said Schmidt.
"These are not well understood, difficult to change, expensive to manage, unpredictable in production and generally a nightmare for any organization that wants to become agile."
But integrating these systems requires more than just a technological familiarity with the systems involved. Integration is a specialty in and of itself, with its own set of tools and procedures. It also works best when it has its own dedicated personnel.
"Organizations should establish integration competency centers (ICC)," said Schmidt. "Integration is its own discipline and you need a group in the company to do it."
He said that having an ICC concentrates those personnel who enjoy integrating systems into a focused team, while letting others get on with the business of innovating or operating specific business applications and functions.
The ICC staff can then concentrate on coupling independent systems, extending the life of legacy systems and establishing a common framework so changes to one aspect of the infrastructure can proceed rapidly without disrupting other activities.
Establishing a fully competent ICC won't happen over night, however. But doing so will guarantee a higher success rate for future integration projects.
Schmidt lays out a five-level capability maturity model (CMM) for integration which is modeled after the CMM for software:
Schmidt advises organizations try to achieve at least level four.
"While level five is generally considered the ultimate and most desirable target, level four is the point where the best practices and technology come together to provide a 'sustainable' infrastructure," he said. "Once an organization achieves level four, integration becomes normal and 'routine'."
Even if you do build a level five ICC, you still won't have all the expertise you need for every project. A mature integration process includes finding the right experts, documenting what they know and then incorporating that knowledge into future projects as you move up the integration CMM scale.
Often, some of these experts will be within your own company. This is particularly true when linking legacy systems, whether custom built by a contractor or by your own staff.
But when setting up new systems, you have to look outside for that expertise, working with the vendors and outside consultants.
"What IT managers, CIOs and CSOs are often looking for is an organization they can partner with that can bring a broad base of expertise in a particular area that their in-house staff can't be expected to have," said Liz Mann, managing director of Mycroft, an identity management and security firm.
The key is not to simply have someone else do the work and give you a turnkey system. Yes, doing that may be quicker, but it keeps your organization from adding expertise to its ICC and will hamper efforts down the road.
"Customers today really want to understand what they are deploying," she continues. "Whether they leverage a company like us to do the whole piece of work or not, they are not as distanced from the project as they might have been in the past."
The model she recommends is to break down major projects into three-month chunks, each with its own set of deliverables. The consultants then work alongside the in-house team.
"The best advice is to use the systems integration firm in the largest way in the beginning, and to design into the plan a declining level of involvement over time after some infrastructure has been built," she said. "Take advantage of the expertise in the market to build the expertise in house."
By following this methodology, she said, a company can establish the skill sets it needs to make integration a routine and ongoing action, rather than a disruptive process of questionable success.