Not all items are created equal. Not all customers are not created equal. We know this to be self-evident. If this is true, why do we assume our accuracy should be the same for all items and customers?
As Demand Planners, it’s not enough to just forecast; we need to understand the underlining forecastability for each item. Forecastability may not be in Webster’s dictionary, but it should be a word we are all familiar with in demand planning. It’s how we measure the underlying uncertainty of a particular item so we can know what kind of accuracy we can expect from it, and what resources are worth allocating to a particular item or category.
To get some insight on this important topic, I invited Sujit Singh, CEO of Arkieva, to discuss this topic on the IBF On Demand Podcast. The following is based on that conversation.
Are All Items Forecastable?
If something has sold only 1 time, then we can agree it is impossible to forecast. Similarly, items with a long tail are not always possible to forecast. Items whose demand is impacted by external variables may not be forecastable either.
Any item that is stable with sufficient data points is forecastable; we can apply a range of techniques to generate a forecast, and in that sense it is forecastable. But the proof of the pudding is in the accuracy – regardless of the data points available or the techniques used to generate the forecast, if the resulting accuracy isn’t sufficient to help plan the business, it is not forecastable.
Items Are Getting Less Forecastable
We have all heard about long tail demand where demand is getting divided into more and more products whereby the portion of demand that is fundamentally unforecastable has increased. Sujit says that with this in mind, we can expand the definition of ‘forecastable’ by generating a forecast range (instead of a point forecast) and as long as we’re inside the range, the forecast is ‘accurate’ and therefore forecastable. In so doing, we can make these tricky-to-forecast long tail items more forecastable.
Reducing Forecast Error During The Pandemic
Given the current demand disruption caused by COVID-19, forecast accuracy is inevitably lower than that we which might have enjoyed prior to the pandemic. Should we increase our tolerance for forecast error? Sujit says if we give more weight to recent observations, isolate certain history and identify certain factors impacting demand, we can still get decent outputs from our time series. Of course, when demand assumptions change, forecast engines aren’t aware. A forecasting system doesn’t know a plant closed down, but your sales team will. With the right information we can then update the models and maintain some degree of accuracy.
Methodologies To Determine If Items Are Forecastable
Error is the main metric to identify which of your times are forecastable, but Sujit recommends another simple (yet useful) metric – Coefficient of Variation. The idea is we calculate the standard deviation of a time series and divide by the mean. Very often, the cutoff point forecasters use for forecastability is a CoV of 0.5 (the lower the number, the better). It’s effective but not a perfect measure of forecastability.
We can also use intermittence i.e., gaps between observations. Let’s say we are looking at data in monthly buckets with a sale in month 1, nothing in months 2 and 3, and a sale in month 4. Calculating the average delta between the non-0 sales. If you are more than 1.2, your series is considered highly intermittent and therefore difficult to forecast. You could still reach some forecast accuracy using specialized methods like Croston’s model, but it’s a challenge.
What Do We Do With Problematic Tail Items?
We all have items we struggle to forecast and would rather forget about. In such cases, once we’ve exhausted other methodologies like getting qualitative inputs from sales, Sujit recommends grouping these items and forecasting at higher levels of aggregation so the individual items ‘inherit’ some of the properties from the top level, thereby making them more forecastable. Let’s say you have 7 products, each of which are unforecastable individually and share some commonalities. Forecast at the group level then disaggregate, applying the weights to each item.