Why is Application Security so Elusive?

By Ed Adams

(Back to article)

Consumer and business products that we use every day — calculators, lawnmowers, stoves — are very robust and meet or even exceed industry and legal standards. And, although they might fail from time to time, I remain comfortable using them.

So why then does software lag in terms of overall quality compared to these devices? Why don’t we use a comparable quality assurance model for software so that we can relish in the thrill of having applications that aren’t fragile when put into production?

The reasons are many, but two of the primary ones are:

  • People don’t consider (or even know about) all the aspects of software quality.
  • Application reliability and security expertise is a rare and valuable commodity.

    Using the familiar framework, the food pyramid, allow me to discuss why security and reliability defects persistently exist.

    I won’t bore you with lots of analogous statements to the food chart because at a micro level, comparing software applications to cuisine is like, well, comparing apples to oranges (pun intended).

    However, at a conceptual level, this proposed software quality pyramid and the food-group pyramid are similar because both are designed to set guidelines for the general population looking for a well-balanced approach and they are de-facto categories set by independent assessors for the betterment of their respective industries.

    Let’s start at the bottom where you have your compulsories — your breads and cereals. These are heavily abundant and the lion’s share of recommended daily intake.

    As you work your way up through the fruits, vegetable and meats and finally to dairy, the recommended allowance changes based on a meaningful algorithm of value to the body.

    Some are healthier than the others; some are much harder to live without; some are more difficult to obtain because of cost or accessibility. Regardless, the significance remains the same: the food chart is a pyramid because all food is not created equal, and finding the proper balance is critical.

    Software quality is no different. However many software teams have chosen to eliminate security from their process for building and deploying reliable and secure products.

    The application quality stack (Fig. 1) groups five aspects of software quality in a hierarchy based on several factors. Continue to think of it as a software quality food chart, a general guideline for good software quality intake with five basic components: functionality, system test, performance, reliability and, the elusive, security.

    Figure 1: The Application Quality Stack


    This is the pinnacle component of the software quality stack because: it is the most challenging of all the aspects of software quality, it is the most potentially damaging aspect, and it is the aspect of software quality that is the least understood.

    When testing for functionality and performance, you’ve got a nearly linear relationship: remove 30% of your functional bugs and your application is for the most part 30% better than it was and your customers or you are 30% happier.

    This is not the case for security, where you could spend that same 30% testing an application, and that single security defect that you missed can cause a major breach.

    You know what happens next: you see it every day in the news! As is the case with reliability issues, we are almost always finding security problems “under the covers” of an application; in the pipelines of its conversations with the database server, Web server and other applications it depends on to operate.

    So, the solution ought to be simple, right? Let’s incorporate security into our daily intake of good development and testing practices, and let’s purchase some sophisticated tools to assist us in our efforts.

    However, unlike other software quality test tools, automation for security assessment is dangerous and fraught with accuracy challenges. Both false-positives and false-negatives plague existing security tools.

    Automated tools are great at validating that functionality does work because they know exactly what to look for and it is easily programmable because it is GUI-driven. An automated tool is mostly clueless, however, when it comes to environment testing as it will find itself in a foreign territory where human knowledge is the only efficient interpreter of clues.

    The most effective security assessment tools are the highly-specialized tools that do only a single thing and do it well. Even there you still need a lot of human interaction and interpretation for the application under test and the tool.

    Reliability and security defects also emote themselves in dependency testing. Applications operate in a highly co-dependent environment in which they may load dozens of libraries and interface with several third-party or OS components.

    It is in these interdependencies where reliability and security defects hide. It is here that the crafty assessor needs to look and it is why they are so elusive. And even if reliability and security requirements were outlined meticulously, they can still be implemented in ways that cause insecurity.

    This highlights the importance of thoroughly testing “negative requirements” and asking the what-if questions constantly during your software quality assessments.

    Thorough Software QA

    We see the extremely damaging impact of reliability and security defects to businesses all the time. The mind-boggling part is that so many tools vendors still call themselves quality assurance companies when they don’t deliver any solutions for reliability and security. How can that be? We’re building, using and buying applications with which we entrust our personal information, and many of which have modest security gates — at best.

    We as consumers assume that software we use has gone through the software quality stack with all aspects covered. How is it that we know so little about the true quality that went into the building and testing of them?

    Imagine the same principle applied to a different industry. Would we ever allow General Motors to rant and rave about the quality of their car without reviewing their safety rating? Would we be satisfied knowing that the breaks were tested only prior to assembly, and not after it left the assembly line?

    Most other industries require vendors to demonstrate that all aspects of quality have been addressed or documented in the product they release to the public. That day will come for software, but I’m impatient, so I’ll get the ball rolling with a few specific, easily obtainable recommendations:

  • Let’s ask our QA tools vendors why they are not building tools for security at the development and tester level like they do for functionality, system test and performance.
  • Before we buy products, let’s ask vendors if all five components of the application stack have been met and how.
  • Ask software vendors what their patch release strategy is when a security defect is found.
  • Seek expert advice if you are lost or worried about your application quality. There is lots of help out there.

    Ed Adams is CEO of Security Innovation, an independent provider of application security services that include security testing and training.