Agile Enterprise Part 4 - Measurement

agile measurement.png

So far in this series on the agile enterprise I have covered

  • organising for agility;

  • funding envelopes as an alternative to project funding;

  • enabling customer experiences with agile platforms.

In part four of the series I will explore an important foundation of the agile enterprise, a measurement system that is focused on change speed, productivity and quality. 

You’ve probably heard the saying “What gets measured gets done.”  Sales teams are a good example where the performance measures are usually very clear, and they are often linked to remuneration outcomes.  For sales teams, if management were to put in place a one-dimensional measurement system that solely focused on new sales, then they shouldn’t be surprised if sales people aren’t that motivated to help with after-sales service.  A one-dimensional, new-sales focused reward model would also provide no help with encouraging front line staff to only sell to customers for whom the product or service is a good fit.

The Australian Prudential Regulatory Authority (APRA) raised this exact issue in its Prudential Inquiry into the Commonwealth Bank of Australia published in April 2018.  This APRA report referred to the imbalance that the review panel observed between a strong “financial voice” (i.e. revenue targets) and much a weaker “risk voice” and “customer voice” in the way decisions were being made by front line teams selling credit insurance.  As a result, while sales targets might have been hit, tens of thousands of customers were sold a credit insurance product that they couldn’t use.          

My point here is that even in well understood functions, like sales teams, in companies with very experienced and capable leaders, in industries that have been around for thousands of years, we can still find measurement systems that are incorrect, incomplete and/or out of balance at times, generating undesirable results.  It should therefore be no surprise that the relatively young and very fast-moving technology change teams in our organisations may not have the right measures in place.  


A common problem I see with change investment measurement in organisations that rely on traditional project-based accounting systems and processes is they tend to have a one-dimensional focus on whether or not the projects are delivering to the original plan.  Too much focus on on-time and on-budget tends to encourage people to minimise reported cost and schedule variances leading to:

  1. Good news reporting with delivery issues showing up very late in the project, and

  2. Large up-front padding of project cost and schedule estimates with contingency.

Is minimising cost and schedule variances between actual and plan really what the owners of the organisation want?

I put to you that the owners of organisations, private or public, for-profit or not-for-profit, don’t actually want a one-dimensional focus on delivering to the original business case.  Rather they want the best return they can get on their investment within their risk appetite.

Think about it.  If a team member two months into a twelve month project discovered a way to deliver the project at half the originally estimated and signed-off cost, would owners want take advantage of this opportunity?  Indeed they would.  A measurement system which only measures the variance to the orginal plan can’t tell you whether or not a positive variance to plan was because the team outperformed by improving their delivery productivity or because the original estimate was just high! 

If you break it down, rather than just measuring actual against plan, what we need is a measurement system that focuses on all three dimensions of the change investment:

  1. Owners care about the cost of the change and whether or not it can be done more efficiently, therefore we need to continuously measure change team productivity.

  2. Owners care about the outcomes delivered, be they economic or social outcomes, because those outcomes drive the short, medium and long-term success of the organisation. Therefore, we need continuous measurement of the relevant customer and business outcomes; and

  3. Owners care about the risks, both delivered and delivery risks. They need to know how likely it is that the outcomes will be achieved and how likely is it that the costs will vary. Therefore, we need continuous measurement of risk factors like change failure rates.

When enterprises adopt agile ways of working I often see and hear about the changes to processes like “backlog grooming” and using “Kanban Boards”, changes to meetings like introducing “stand-ups” and “retrospectives” and we certainly hear a lot of new language.  An area often overlooked is the introduction of this new continuous measurement model for the way we measure change team performance.  If you’ve started your journey to agile ways of working and rather than feeling more in control of your change teams it's feeling a bit like agile is code for anarchy, then I suggest having a look at what new measures of change performance have been put in place for your teams’ performance.    


I’ve found working with more mature agile teams as a senior executive is dramatically different to working with traditional project teams.  Agile teams spend almost no time obsessing about whether or not the actual cost or schedule will match the original plan because these dimensions of the delivery model they are working within are fixed.  Rather they spend their time obsessing about how to deliver more customer and business benefits faster as well as how to drive productivity.      


If you need help with your operating model for agile change feel free to reach out to me on


About the author: David Boyle’s IT career over 30 years spans both the buy side and sell side of technology services.  He’s worked with Accenture, EY, Commonwealth Bank of Australia and until recently was the Group CIO at NAB.  David is now the Managing Director of CAP2ITS, a technology advisory firm focusing on technology strategy, performance and risk.

David Boyle