Why Software Development Projects Fail, Part IV: Release

By Drew Robb

(Back to article)


This series of five articles takes a closer look at software development, the reasons for failure and how to avoid them.

For project success, it is best to involve users early in the testing process and continue to engage users until it is time to go live. In theory, once the software has been built it is time to test it and roll it out to the end users. In reality, it is not that simple. Successful deployments require that planning the testing and deployment be started during the requirements stage, and move along, step by step, with the software development.

The most common mistake in rolling out software is showing the system only when it is supposed to be ready,” said Mikhail Bykov, CTO of Luxoft, an international IT services firm headquartered in Moscow. “The users should get hold of it as soon and possible; they have to get used to its look and feel.”

Testing Best Practices

The testing process should actually begin before the first line of code is written. “When we are talking to customers about their design requirements, we try to get what will be their validation criteria or test criteria,” said Abid Ali Neemuchwala, VP and head of Global Process Excellence for Tata Consultancy Services (TCS). “If they start defining the test specs too late in the game, the chances of meeting those test criteria come down.”

Having the tests included in the requirements gives the developers a definite set of criteria they can use when building the software. There are two main tests that are needed. The first are the technical and load tests that are routinely conducted during the development process to make sure what is written performs as intended. These can be executed running automated scripts. When doing the testing, however, don't expect to see perfection. If you don't find any errors, something is wrong with the testing process and those errors will plague later steps in the development process or be discovered by the end users.

With a very process-mature organization, you should understand that there are a certain number of defects you introduce,” said Abid. “Good testing means you can identify and eliminate those defects early in the lifecycle.”

TCS has developed models based on historical experience with thousands of projects that enables it to predict how many defects it should find at each phase of development. When it doesn't find the expected number of errors, instead of thinking that the developers did an unusually good job, it starts investigating the test procedures.

We have had situations where we have put projects back into testing mode if we had not discovered enough defects,” said Abid. “We then had a delighted customer because they had not found as many defects as they expected based on their history.”

While testing the individual of software units can be done by developers, they shouldn't be the only ones involved in tests of the entire system. They will typically find that everything performed the way they designed it to perform. Rather, system testing should be conducted by a third party, or preferably some end users. It is best to engage the same people who specified the requirements in the first place, otherwise you are testing against a different set of criteria than originally expected. Like software functional testing, user testing should be done throughout the development process rather than waiting until the end.

If you can break your application down into manageable chunks and you use those chunks to do your testing user acceptance testing events, you can get your feedback early in the projects on things such as your GUI design,” said Jeff Monteforte, president and co-founder of Exential, an IT project management consulting firm in Cleveland, Ohio. “Your subsequent tests then take into account that early feedback since you have altered that design already.”

Taking this approach also makes it easier to modify the software.

It is a bad practice to bundle the whole solution and ask the end user testing group to test an entire application, with 30+ screens, 50+ reports and 12 significant work flows,” Monteforte continued. “It becomes a monolithic behemoth to test and it will cause huge amounts of rework that you could have short circuited if you had broken it into half a dozen testing events.”


Jon Hughes, VP of the Technology Solutions Group for Robbins-Gioia, recommended getting lead users from different parts of the organization to come in and do the acceptance testing. Not only do they provide the viewpoints of different user groups, but they can then act as ambassadors to their fellow employees, easing the final deployment.

If the test is successful, they will go back to their coworkers and friends and peers and say that when we get the system, life will be so much easier, it will improve productivity, we won't having to worry about data integrity and walking all these papers around,” said Hughes. “You need that kind of energy going back out into the organization because it is the users who will make or break a deployment, despite executive buy in.”

Out the Door

Finally, the software is ready to roll out to the end users. Even if another part of the organization will be conducting the roll out, that doesn't absolve the developers of all responsibility. “What we realized is that the end to end experience determines the customer perception,” said Abid. “Your customer will not be happy with a very well-developed system which got bungled up in roll out.”

The deployment can be done either as a “big bang” deployment, or by taking a phased approach, gradually issuing the functions or rolling out the software to one group of users at a time. If you have successfully involved users in the requirements and testing phases, users should be looking forward to getting to use the new system.

A very good practice is to have a focus group of users working with you closely throughout the project,” said Bykov. “You will not only get great feedback from them, but also get supporters and trainers for other users.”

Letting them see and click around in prototypes is an easy way to gain familiarity, but there is a risk in doing this too soon.

If you start giving the users early builds containing lots of mistakes, you may get disappointment and lose their trust,” Bykov continued. “So, give builds often, but only with ready functionality; all non-implemented or buggy parts should be disabled.”

Even if users are looking forward to the new system, their enthusiasm can be crushed if they aren't properly trained before they have to use it for production. The education should be ongoing during the development process, and it should take into account the current skill level of the employees. While most are now used to web-based applications and pull down menus, not all are.

Avoid a big bang deployment unless it is a very intuitive system and dedicate the necessary time to training,” advised Hughes. “There will be users who are resistive and very negative and you just have to show them they will be better off with the new system.”