If you can’t count what’s important, you make what you can count important
Is a famous quote regarding the Vietnam war where success was determined by “body count”. The single minded focus on this metric drove the military to kill indiscriminately because they could add civilians to their “count”, encouraged faking of numbers and generally took the focus away from the real important issues which was winning the hearts and minds of the people.
That’s a dramatic example but this kind of thing happens on projects all the time. Key Performance Indicators (KPIs) are something that get a lot of focus on projects by management and stakeholders. What does not get enough focus is whether they are the right KPIs in the first place. There is a whole science behind choosing KPIs but I’d like to use a simpler framework here with some questions to answer for every KPI:
- Do they actually give important and actionable insights?
- Do they create perverse incentives in the project team?
- Are they verifiable and difficult to fake?
The KPI test
1) Important and actionable insights
A KPI is only useful if it can generate insights that useful decisions can be made with. Usually this is by highlighting where the project is failing which then prompts the management to drill down into the details of what is going wrong and fixing the issue.
A very common example that I really don’t agree with is reporting the number of safety cards submitted. This doesn’t tell you anything about project safety if the leadership is enforcing safety card submissions resulting in 90% of them being about how bad the canteen is.
This metric is significantly more valuable if tweaked a little bit to measure the amount of safety cards of high quality. This is further enhanced by categorising them to find themes of concern which can highlight systemic issues.
2) Perverse incentives
KPIs generate perverse incentives if not designed correctly which can influence project behaviour to actually make the project perform worse.
Mega projects often like to keep track of measured quantities installed. The problem with this is that it incentivises the project team to focus on the easy wins to get a good number in the period rather than what is most important to deliver the overall project in the optimal way – areas where it is really easy and efficient to lay down large amounts of material. Unfortunately this often means working out of sequence and takes focus away from the complex parts of work that are critical pinch points. This was rampant on the crossrail project in London and I have no doubt contributed to the famous delay.
A better way to do this is to track what percentage of the quantities that were installed this period were supposed to be installed this period based on the planned activities? This encourages working to the plan and highlights potential issues which are causing the plan to change last minute.
3) Verifiable and difficult to fake
Many KPIs are difficult to verify and are often fudged by the project team to cover their failures or for contractual reasons. They should be generated from datasets which enable analysis into root causes and performance.
A typical example is project end date variance. This is why projects are often reporting to be running on time until a few months before they are supposed to be complete and then all of a sudden there is a “rebaseline” with a significant delay.
A better way to do this is to measure the performance of intermediate milestones and how they are trending. If your intermediate milestones are following the pattern of remaining on track until close to their planned date and then suddenly jumping into delay you have a systemic issue with how the programme is being reported.
A Final thought
Not everything that counts can be counted
It is very important to keep a holistic view on what is going on with your projects. I think we should take the view that KPIs are only a small part of the picture when it comes to project performance not the entire picture as they are frequently treated. Otherwise we can be working to meet KPIs only and take our eyes off the bigger picture.
