We'll call you and arrange a demo




Blog

Measuring Forecast Accuracy – An Introduction to Performance Analysis

Cashflow Forecasting Accuracy
Posted by Martin Gillespie January 29, 2016

Conversations about forecasting accuracy happen at a number of different levels within finance teams. They can be between treasury and the CFO or treasury and entity controllers. Although the conversations can vary in nature, the issue discussed is often similar, how accurate are our cash forecasts and how do we improve it in order to make better working capital and funding decisions?

The first step in improving accuracy is measuring accuracy. In this article we discuss an accuracy measurement methodology called ‘Performance Analysis’. This methodology provides a consistent and easy to explain basis for both measuring and improving forecasting accuracy.

Start with Actual vs Forecast Analysis

Performance Analysis sounds complicated but, don’t be put off straight away, it’s simply a continuation of analysis you probably already carry out. Most people will be familiar with actual versus forecast analysis. This type of analysis measures the accuracy of a forecast by comparing it to the actual cash flow movement or balance at a point in time.

The actual versus forecast accuracy calculation is pretty straight forward – in the example below we used forecasted receivables to demonstrate the calculation.

Forecasting Accuracy

On its own this simple actual versus forecast comparison is interesting but it tells the Treasurer or CFO very little about how accurate the forecast has been leading up this point or, in other words, how the accuracy of the forecast has “performed” historically. Without understanding historical accuracy how can we rely on our forecast? Is the fact that the forecasts are very accurate or inaccurate just a once off?

Being able to understand trends in accuracy will allow you to build confidence in the information you use for decision making purpose.

In Practice

Say, for example, you operate a six week forecast that rolls over every week and we are at the end of the sixth week of the year (i.e. we’ve collected six forecasts to date). As part of the process actual cash flow data is also captured that allows you to measure the accuracy of previous forecasts. This means every week, using this week’s actual data, you can analyse previous forecasts to understand how accurate they were. The table below shows the accuracy, for each weekly period, of the very first forecast of this year ‘Forecast 1’. We had been able to measure the accuracy of the first five weekly periods using actual figures from previous weeks. The accuracy highlighted in red was calculated using this week’s actual data.

Per Period Analysis
 1 Week2 Week3 Week4 Week5 Week6 Week
Forecast 192%90%77%61%57%51%

In this example we can see in Forecast 1 our one week forecast accuracy was 92% but our four week accuracy fell to 61%. Therefore in the short term our forecasts were very accurate but in the medium term accuracy decreases significantly. While it is useful to know this, it doesn’t provide a huge amount of value as we don’t know if these accuracies are a once off or consistent with historic trends.

This is where Performance Analysis comes in – if this type of actual versus forecast analysis is carried out every week the results can very quickly provide a telling insight into the true accuracy of your forecast.

Performance Analysis

We’ve expanded this example to include analysis carried out in the weeks following Forecast 1. By combining this analysis with analysis carries out in subsequent weeks you start to build a far richer view of forecasting accuracy – a view that shows historic accuracy trends across all time periods in your forecast.

Performance Analytics
 1 Week2 Week3 Week4 Week5 Week6 Week
Forecast 192%90%77%61%57%51%
Forecast 296%87%79%65%54%
Forecast 395%89%75%60%
Forecast 494%91%76%
Forecast 593%81%
Forecast 695%
Average94%89%77%62%56%51%

The table above shows the actual versus analysis we’ve carried out on each forecast period, for each of this year’s six forecasts. The numbers highlighted in red are the accuracies we’ve calculated this week using this week’s actual cash flow data.

How is this analysis used?

Performance Analysis shows trends in forecasting accuracy. If we have information to hand that shows us how good we’ve been at forecasting in past, we’ll have far greater confidence using current forecasts to make decisions. For example, we can see from the analysis above that we consistently forecast, one week into the future, with about 95% of accuracy and, two weeks into the future, with about 90% of accuracy. As a CFO or Treasurer, this then allows you to make cash or debt management decisions two weeks into the future with a high degree of confidence.

On the other side of the equation we can see that the accuracy of forecasts decreases significantly around the three to four week mark. Using the results generate by this Performance Analysis we can approach business unit controllers with proof that their forecasts have been consistently inaccurate or use it to alter our own forecasting assumptions.

Performance Analysis provides the basis for both more confident decision making and accuracy improvements. It should be carried out by anyone managing a forecasting process.