Watch out for bad KPIs

Bus Stop Board

Metrics are everywhere these days. Data is a way of life, and we expect to get it! But what happens when it’s used badly?

I recently took a trip to London with my son, and was waiting at a bus stop to catch a number 10. I noticed that the bus stop gave me the amount of time I would need to wait. Great KPI! I only have to wait 2 minutes for the next number 10.

But then 2 minutes came and went.

And another 2 minutes.

Then another 10 minutes.

Finally, the bus showed up 15 minutes later than the board had suggested. How could the computer have got it so wrong?

We need valuable information

The information I was reviewing at this London bus stop was not valuable. It gave me false information – which could not be relied on. How many times have you seen data or information in your organisation which has a similar profile. How many times have you made decisions on bad KPIs such as this, which have caused you and your organisation issues?

Be bold. If you are responsible for producing data or information, make sure it is valuable and correct. If it is not, stop doing it.

Find the right metric

I have a theory about the bus data.

The buses are all fitted with GPS systems, so the central computer always knows where they are in London. Therefore the computer know how far away the bus is from each stop on the route (distance).

The system then uses the average speed of a london bus is used to convert distance into time (time = distance / speed). So when I’m sitting in Oxford Circus with my son (probably the slowest road in London), this formula is far too ambitious. In other areas with faster roads, perhaps the buses arrive sooner.

So what is the right metric? I would give the customers the distance of the next bus, in kilometres or miles. So for example, rather than seeing 2 mins, you would see 0.4km. As I sit at the bus stop, I feel comforted that a bus is coming, although I can see that the road is very very slow. As the distance reduces, I get a feel of when the bus is coming, which is never wrong. This is a valuable metric which keeps my expectations managed.

Applying this

If you reflect on this example, and apply it to your organisation, can you think of any similar examples?

As a finance reporting professional, I feel that often too much focus is placed on EBITA variances to the month, or year to date. These historic figures only tell you what happened, not what is going to happen.

My favourite KPI is distance to go (DTG) Vs. prior year. This shows you the future remaining months in your EBITA forecast, Vs. the same period in the prior year. Even in rapidly changing businesses I find this KPI gives me really strong insight into the likelihood of what we think is going to happen. It also highlights errors in forecasting, or ‘sandbagging’ (hiding of upsides from prior months in the final month).

Do you have any examples that spring to mind?

Leave a Reply

Your email address will not be published. Required fields are marked *