Survey says: Bad alignments hamper app management

NetForecast find a significant gap between what gets measured and what's important

A recently completed NetForecast/Network World survey of application performance management (APM) practices has uncovered serious gaps between the performance attributes IT managers cite as important to measure, and those they actually do measure.

Application performance management: Keeping an eye on the end-user prize

These gaps point to a misalignment between needs and practice, and should be corrected to ensure good performance outcomes. In particular, end-user page response time and server query response time showed enormous gaps between their perceived importance and the frequency with which they are measured.

A recently completed NetForecast/Network World survey of application performance management (APM) practices has uncovered serious gaps between the performance attributes IT managers cite as important to measure, and those they actually do measure.

Application performance management: Keeping an eye on the end-user prize

These gaps point to a misalignment between needs and practice, and should be corrected to ensure good performance outcomes. In particular, end-user page response time and server query response time showed enormous gaps between their perceived importance and the frequency with which they are measured.

In the survey, 364 participating IT managers were asked to rate the importance of 12 application performance attributes, and to tell us whether or not they measure each of the attributes.

We considered needs and practices to be aligned if an attribute is cited as important and is measured, or if it is cited as unimportant and is not measured.

Needs and practice are misaligned when an attribute is described as important and is not measured, or if it is described as unimportant and is measured.

A performance attribute that is important yet not measured suffers from a measurement gap. And an attribute that is unimportant yet measured exhibits what we dub measurement squander.

In aggregate, we found that 60% of the response combinations were misaligned, and only 40% were aligned.

Of those that are misaligned, 37% show a measurement gap - in other words important attributes were not measured. And 23% of that 60% were squandering their measurement efforts because they were measuring things they considered unimportant.

The top five application performance attributes respondents most often describe as important to measure are network availability, server availability, end-user page response time, server query response time, and bandwidth utilization.

Of these five attributes, a serious measurement gap of more than 50% exists for end-user page response time and server query response time -- compared to modest gaps of under 30% for network availability, server availability and bandwidth utilization.

Click to see: Chart showing percent of enterprises not measuring important factors

We surmise that end-user page and server query response times fare poorly compared to the other important attributes because they are harder to measure, and there are fewer and less established tools available to measure them.

Measuring response times takes synchronization, which is not easy to pull off. You need the virtual equivalent of a stop watch to determine start and stop times, and you need automated report-generating capabilities. But response times are very important because they measure what matters most to end users and are where the rubber meets the road. Everyone talks about how important it is to ensure a good end-user experience, but more often than not response times go unmeasured, making it impossible to know how well you are doing on that front.

In contrast, network availability, server availability and bandwidth utilization are well-established, easy-to-gather metrics, reported by tools ensconced in corporate network operations centers.