Volume No. 65

How Many Measures Are Enough?

As is typical in any installation of an enterprise-wide Performance Management framework, managers and employees often balk at the volume of measures and data that are represented by the selected KPI’s for the business.

Such was the case for me in a recent project that spanned six major business areas, from Energy Supply to Delivery, and also included all of the company’s administrative, customer, and infrastructure support areas. In total, the number of KPI’s selected were just over 100 and represented both “level 1” indicators (business unit goals and top level result areas) and “level 2” indicators representing a suite of KPI’s that reflected the individual performance of each area (roughly 4-6 each). For the client, that appeared to be an overly heavy dose of data and information to absorb.

In part, the client is right, this is a lot of information, being that each metric will require numerous data elements to assemble, and further complicated by how many different ways the client will want to view the data (by business unit, component, region, etc.). So yes, this is a lot of data. But is it too much?

As with most questions like this, there are a few dimensions to the answer.

1. First is the question of sheer volume. As a matter of comparison, there are clients of mine that started with a single business unit that had over 1400 KPI’s, and others that started with only 5-10 at the entire company level. Yes, 1400 is too many (and to call them KEY Performance Indicators is clearly a stretch). And yes, 10 for the entire enterprise is too few. But by sheer numbers, 100 would fall on the lower end of the spectrum, yet broad enough to be representative of the business in general.

2. Second is the architectural aspect of the measurement framework. We are not talking 100 metrics to be consumed in “one sitting,” but something designed to be managed by a collection of individual managers – in this case between 30 and 50 managers depending on the level of management. Any good balanced scorecard will have a “line of sight” or pyramid aspect to the architecture, typically flowing from the mission to the key result areas, to the supporting objectives, and finally to KPI’s and metrics. So the question of “too many” or “too few” really depends on what level we are talking about. At the key outcome level, 100 would be ridiculously high, and at the KPI level it would be just as ridiculously low.

3. Third is what I will call the soundness or “sniff test” element – the question of whether there is unnecessary duplicity, redundancy or inconsistency in the universe of measures selected. The test I apply here is what I call “complete and discrete.” For each area being measured, does the set of KPI’s adequately measure (80% or more) of what the function or process is there to produce? And are each of the measures somewhat mutually exclusive (i.e. not redundant)?

4. Finally, the test of relevancy does have to come into play. This one is tricky because you need to strike a balance between being relevant to everyone and relevant to a particular process owner. Often, you may elect to include a few measures that are in the proverbial “grey area” so as to not disenfranchise a key business unit leader or process manager. Sometimes, a judgment call.

Using these three tests, we would normally conclude that the 100 measures selected would be reasonably appropriate given the size of the enterprise and breadth of business units at play.

Here’s an analogy to consider. Think about a football coach who meets with the team at halftime of the big game. The most significant outcome is whether they are winning or losing. On the surface that is what really counts. Keeping it simple, one might also be able to add in some evaluation of offence, defense and special team’s performance. Four measures that in total pretty much tell the story. Simple enough right?

But is this enough to drive a real understanding of what is really going on in the game? Not really. As the team breaks out into their individual units, simply telling them that they had 150 yards of total offence reveals little if anything in terms of what needs to change. With just that information, the conversations would be very short and somewhat pointless in terms of managing performance.

That is why offense, for example, is often broken down into metrics like number of first downs, time of possession, yards per carry, number of “touches” per key player, etc. So extrapolating out, there may be 2-3 dozen measures for an 11 man football team required to effectively manage a 60 minute contest.

My point to you is this: It’s ok to aspire to simplicity. We all want to keep the message simple and not confuse the troops. But let’s also remember that we are managing a business that does have some complexity to it. We are often talking several thousand employees and a business strategy that transcends many years. While 100 metrics may sound daunting to a company at first, it is really just scratching the surface in terms of the volume of drivers and levers at play in a comprehensive EPM framework.

The true test of whether the volume of measures is right is how it stacks up against the tests outlined above, and how well your overall framework holds together.

That notwithstanding, some good rules of thumb to follow for an enterprise with multiple business units:

- 2 to 3 broad business goals (usually things like revenue, growth, profit, etc.).

- 5 to 6 outcome areas (perspectives that need to be managed, like Financial, Customer, Operations, etc.).

- 2 to 3 objectives within each outcome area (e.g. Customer satisfaction, Customer retention, etc.).

- and a collection (usually more than one and less than five) measures (whatever is necessary) to adequately reflect performance of each objective in a meaningful way.

This is certainly not a hard and fast rule, but should give you some parameters to go by.

Author: Bob Champagne - UMS Group, Inc., a privately held international management consulting organization specializing in Performance Management tools, systems, and solutions. Included in UMS Group's product portfolio are a wide variety of performance tracking, reporting, and benchmarking solutions, as well as customized performance assessments and diagnostic services. UMS Group has consulted with hundreds of companies across numerous industries and geographies. Visit UMS Group at http://www.umsgroup.com or contact us directly at +1 973.335.3555.