“Shallow men believe in luck or in circumstance. Strong men believe in cause and effect.” – Ralph Waldo Emerson
It would be easy to see the demise of Sears Roebuck retail stores as a triumph for e-commerce. In actuality, if you were to look at newspaper advertising compared to annual revenue of Sears, you will notice an obvious pattern emerge. When we look at the steady increase of newspaper revenue since the 1950’s, it tracks both the growth of Sears and its decline. The fortunes of the newspaper industry and Sears peaked around the turn of this century with a small resurgence in 2006, only for the former to decline, and the latter forced into bankruptcy.
Looking at this data, it is clear that what Sears must do to rejuvenate itself is pour all of its remaining marketing dollars into newspaper print advertising to boost newspaper revenue to save Sear’s from pending liquidation.
Obviously.
If you did not pick up on my hint of sarcasm and how this is an example of the difference between causation and correlation, don’t feel too bad. Newspaper revenue and Sears’ revenue are only an obscure correlation much like linking ice cream sales to the murder rate in New York City.
My point is that humans are evolutionarily predisposed to see patterns and psychologically inclined to gather and fill in the blanks of information to make connections.
The Human Brain Looks For Correlation, Even If There Is None
The question of cause versus correlation, has haunted science and philosophy from their earliest days, and still dogs our heels for numerous reasons. You would think with the vast number of articles warning of its perils, this would not be a problem anymore, but we still get fooled.
We confuse coincidence with correlation and correlation with causality
We confuse coincidence with correlation, and correlation with causality. Just think of the last time something great happened when you had those special socks on – admit it, they are now your “lucky” socks even though you know deep down it had to be coincidence.
Unfortunately, we run into problems with this in business planning and forecasting as well. Of course, if we stock out or have too much inventory the “cause” is the forecast. With poor planning we may see a correlation between forecast error and missed sales, or between long supplier lead times and excess inventory – in both scenarios, the cause is most likely not the forecast.
Forecasts do not cause excess inventory, uncertainty does (supply and demand). In addition, forecast error is not always the result, or caused by good (or bad) forecasting but is indicative of the uncertainty of the demand you are actually measuring.
Forecast accuracy is ultimately limited by the nature of the behavior we are trying to forecast
Forecasts Do Not Reduce Demand Variability
Forecast accuracy is ultimately limited by the nature of the behavior we are trying to forecast. Accuracy is the degree of closeness of the statement of quantity to that quantity’s actual (true) value. Whilst I accept that one’s ability to create an accurate forecast is correlated to demand variability, we also need to remember that an accurate forecast does not reduce demand variability. Demand variability is an expression of how much the demand changes over time, and, to some extent, the predictability of the demand.
Analysis suggests that whatever we can do to reduce volatility in demand for our products, the better we should be at predicting the forecast. Unfortunately, most organizational policies and practices are designed to add volatility to demand rather than make it more stable. We continue to contribute to SKU proliferation, shorter life cycles and more complex channels with dynamic marketing – and still our business partners still attribute missing the forecast as demand planners’ fault.
Historical Patterns Always Repeat, Right?!
There are other ways we can become victim to the deception of causation. Inexperienced forecasters, and those outside of forecasting, may assume that history will tell us exactly what will happen in the future. If we sold 2,000 units last year, then we will sell 2,000 this year as well. If we are using sophisticated software and the fitted model is showing 10% error, then we should expect no more than 10% variation. If there is an obvious historic pattern, then it will obviously repeat itself with the same predictability.
And we are surprised when this doesn’t happen. This mindest incorporates bad assumptions because we fall into the trap of correlation verse causation.
Know The Limitations Of Time Series Forecasting
Time series analysis looks for correlations in successive observations, not causation. One of the 5 laws of demand planning are that you can’t expect to have the same results from forecasting if there are some changes in environmental conditions. Seems obvious but we too slip into unwittingly thinking seasonality will cause a lift, not remembering the seasonal index is only a correlation and the underlying cause could be dynamic weather conditions or holidays that have whole other dependent variables.
We wear our historic data like a pair of lucky socks
We wear our historic data like a pair of lucky socks without ever understanding the cause and stating it as absolute. We become comfortable and do not do enough to look more at the cause than just the correlation.
In today’s business environment, changes in the marketplace are swift, sudden, and may not follow the historical correlation. Just looking at historic shipments alone may not give you what you need or tell the whole picture.
No matter how many articles are written on causation versus correlation, people are wired to focus more on descriptive analytics and making real or imaginary connections to what happened. But we come from a different background – one that must use the data to infer what is going to happen next. To accomplish what we need, many times, we may not fully understand the “why” but use algorithms to help reveal “what” the future holds. We just need to remember that one may not always cause the other, but just rhyme and move in similar direction and manner.
That said, I too will most likely have my lucky orange stripped socks for the next special executive S&OP meeting or if I am speaking at a conference.