Transforming IT: Application Flow Management

By Dennis Drogseth

(Back to article)

Probably none of you have ever heard of application flow management or AFM. And if you have, or think you have, you most likely associate it with network management technologies like NetFlow, for capturing application “conversations” across the network. While this isn’t entirely off the mark—paired flow analysis of traffic volumes across the network is a part of AFM—it’s very far from being the whole picture.

For a few years now, EMA has been assessing a series of capabilities that tend to be orthogonal (set at right angles) to most approaches to network performance management; and to performance management in general. Traditional approaches, which are still the most dominant in the market, and especially dominant among platform vendors, have focused on monitoring events and collecting SNMP statistics through polling network and systems devices. These solutions use this information to isolate points of failure in the infrastructure on a component level that may or may not impact application service performance.

These types of traditional monitoring tools are, as a class, getting better due to better analysis across network and systems devices. In many respects, they represent the next generation of platform centric performance monitoring tools for integrated network and systems performance. But, as a class, they are still far from offering the complete picture and have two very significant limitations.

First of all, they depend on polling for information, and so cannot do real-time analysis (most still collect data every 10-or-15 minutes). Such a low frequency of information gathering makes sense since polling for performance information can in itself cause congestion, and in one or two instances EMA has documented private networks where better than 50% of the network performance issues came from enterprise performance management traffic. But this type of polling provides only limited visibility into service performance, and critical intermittent problems can sometimes go completely undetected.

A second problem with traditional performance management tools is then don’t monitor actual application traffic. They monitor the infrastructure supporting it and through various levels of intelligence (and here the quality ranges vastly) deduce what the application impacts are on a specific component. This is something like studying storm systems by monitoring the trees—certainly not irrelevant, but not the most direct approach.

Assessing AFM

AFM is the flip side of this approach. This is EMA’s term for all those technologies that directly address application traffic at various layers (especially layers three-through-seven and above in the OSI stack). These include a whole host of technologies, but a short list would include:

Analytics for assessing the Quality of Experience (QoE) of applications and transactions from the end-user perspective. QoE should be able to provide a governing metric for assessing the impact of service performance from the point of view of the end user.

Transaction analysis at the data center , or other end of the infrastructure, to monitor transactions within the application server with interfaces into the database server and the performance of other data center elements such as storage. This is an area where a parallel and complementary set of capabilities come into play to round out the visibility of the application’s performance through the heart of the data center to the end user in remote locations.

Analysis of flows from servers to end devices over IT infrastructure, while supporting packet-level drill-down analysis. In other words, end-to-end latencies of application traffic across the infrastructure with packet and protocol drill down—both for real-time analysis and for network forensics targeted at persistent and difficult-to-diagnose problems.

Analysis of traffic volumes through capabilities such as NetFlow or sFlow, or jFlow or IPFix, or through other types of adapters of probes. This helps to understand how applications are impacting the infrastructure in terms of capacity and potential congestion, and can also expose inappropriate practices such as backing up servers remotely in the middle of the work day, or even security breaches.

Route analytics , or real-time visibility into the actual routed path of application flows. Route analytics is a powerful enabler not only for automating more traditional types of component analysis, but also for being able to more realistically project actual application traffic issues across the routed network.

Application dependency mapping are flow-based technologies combined with more detailed configuration information that can enable more real-time awareness of application-to-infrastructure interdependencies.

Support for application life cycle planning by providing tools with common data sources to support both the management of applications over the network and the needs of application developers seeking to design for real-world distributed infrastructures. These tools can enable new modes of collaboration, such as in application network reviews (ANRs).

Support for financial planning as application consumption can be monitored more dynamically and accurately in context with its impact on the total networked infrastructure.

Support for security in terms of anomalous behaviors both in terms of IT governance (who’s using what application and where), and intrusion detection.

The list is not complete, but it should give you an idea. The truth is that while these seem to represent a whole host of fragmented or different tools, the reality is that many, and in some cases almost all, of these requirements are coming together in the portfolios of management innovators taking more of a “flow” or “bloodstream” approach versus a more traditional component-by-component approach.

Some of the vendors to watch are, in alphabetical order, Apparent Networks, Cisco, Compuware, Coradiant, EMC, e.g. Innovations, Fluke Networks, Netcordia, NetQoS, NetScout, Network Physics, OPNET, OpTier, Packet Design, Shunra, Solar Winds and Wild Packets.

In the end there are several reasons why you, as executives, in particular should care:

  • The management of distributed Web-based service-oriented architectures (SOA) will not be possible without AFM. Both because of its real-time nature and because of its potential ability to deal directly with application components as they interact with each other.
  • In a Q3 2007 study of how IT managers and executives prioritize what they need to manage applications end-to-end over a distributed infrastructure, traffic performance, traffic volume and anomaly detection came out on top. All three are pillars of AFM.
  • AFM has political and process implications more than many traditional approaches because it can provide a unifying system for analyzing application performance problems in context with a whole host of factors. In other words, AFM should speak directly to executive buyers seeking to promote better collaboration across their teams.
  • Also in our Q3 research we learned that if IT is to make a cohesive investment in true end-to-end AFM, you, the C-level executives, are the ultimate buyers. This is not the heritage of this market, which is grounded in more technical decision makers such as network engineers. But if AFM is to come into its own, you will have to play a role especially as AFM emerges beyond being a resource to empower professionals to being a resource to enable better cross-IT effectiveness.
  • EMA has already seen striking examples of value brought to IT organizations via AFM capabilities; including some with explicit organizational and process-directed changes. Among the more dramatic is 50% reductions in MTTR for critical application services, as well as teaming between application developers and network managers to ensure that new applications can perform correctly across the infrastructure.

    Probably the key words in grasping the impact of AFM as a continuum are “visibility” and “context”. This is why EMA believes in the coming two-to-five years, AFM will redefine performance management and ultimately force platform vendors, as well as the industry more broadly, into new approaches for assimilating and integrating application flow information as a unifying resource for managing business services.

    Dennis Drogseth is vice president of Boulder, Colo.-based Enterprise Management Associates, an industry research firm focused on IT management. Dennis can reached at www.enterprisemanagement.com.