Models Alone are Not Enough for Improving Forecasts

jain-chaman_v4

Chaman L. Jain, Ph.D

Statistical models are important, but they are not the be all and end all of forecasting. Forecasts can be further improved if:

1) Data is properly analyzed and treated before using a model

2) Have a process in place for obtaining data/information/feedback from various stakeholders, both within and outside the organization

3) Have a procedure that is well established for monitoring and revising forecasts

4) Set goals and metrics that are well defined for measuring performance

5) Ensure that the people involved are awarded for their performance.

At times, intuitive adjustments in statistical forecasts are needed to account for elements, which have a bearing on forecasts, but cannot be quantified.  This is especially so for information that is not available at the time forecasts were generated, as well as when forecasts just dont make sense. For example, if a major competitor goes out of business or a natural disaster occurred in one of your major markets?  This information would not be incorporated within a statistical forecast.  Furthermore, it is not unusual for a forecaster to find certain computer generated forecasts that don’t make sense. They are either too high or too low, and the forecaster knows from his or her experience the numbers are far from reality. To take care of them, he or she has to adjust them intuitively. So, statistical models are fine, but they would never replace human judgment. They must work together.

Nike, for example, in 2000 implemented a new forecasting system, i2 Technologies, where nine months later it issued a press release saying it lost $400 million because of poor forecasts attributed to the system. It badly over-forecasted their products, and under-forecasted others. In this case, you cannot just blame the forecasting system and models residing within the system? If a proper process was in place that called for monitoring forecasts every month and efforts were made to find the source of error, Nike would have detected the problem much earlier, and mitigated the damages. It appears it used the forecasting system as a black box, and took the forecasts at the face value.  This is a dangerous approach.  You can read more about forecasting & planning in the IBF’s Journal of Business Forecasting.  Your thoughts and experiences on this topic or other topics are welcome!

IBF’s Journal of Business Forecasting (JBF) | St. John’s University – New York USA
Chief Editor | Professor
Dr. Jain is Professor of Economics at St. John's University based in New York USA, where he mainly teaches a graduate course on business forecasting. He is also Chief Editor of the IBF's Journal of Business Forecasting. He has written over 100 articles, mostly in the area of forecasting and planning, and has authored/co-authored/edited nine books, seven in the area of forecasting and planning. His new book, "Fundamentals of Demand Planning and Forecasting," is the basis of IBF's body of knowledge. In a consulting capacity, he has worked for many large multinational companies including Hewlett Packard, Union Fidelity Life Insurance Company, Prince Manufacturing, CECO Doors, and Taylor Made Golf. He has conducted workshops on business forecasting and planning for various organizations including Sweetheart Cup, Eastman Kodak, Jockey International, SABIC, Saudi Aramco, DU-Emirates Integrated Telecommunications Co. -Dubai UAE, and Symbios Consulting Group-Egypt, Goody-Saudi Arabia, Al-Nahdi Medical. He has made presentations on business forecasting and planning at IBF conferences / workshops, Council of Supply Chain Management, Informs, DMDNY in New York, John Galt Solutions and SAS. He has been invited by various institutions to speak on business forecasting & planning including University Technology Malaysia, School of Future Studies & Planning, Devi Ahilya University, India, and Apeejay Svran Institute of Management, India. He is the recipient of 1994 award of the Direct Marketing Educational Foundation for his best paper.

3 Responses to Models Alone are Not Enough for Improving Forecasts

  1. Hi Chaman

    These are great points you bring out about forecasting. Fundamentally what it comes down to is that we can never incorporate all the influences on demand nor can we build models that represent these influencers accurately. Human judgment must be brought to bear because the human brain has a great ability of understanding contextual nuances, machines do not. Most companies that are doing a reasonable job of forecasting understand that the statistical forecast is a starting point, not the end point, becuase they understand that the forecast can never be 100% accurate.

    I have to admit though that I find the demand side of the house to be more in tune with the idea that “life is fuzzy”. The supply side of the house, on the other hand, tends to believe that the output from their ERP and associated planning systems are accurate and, even worse, optimized. The results of these systems are taken as “marching orders” for the supply side, even though the primary driver for the supply side, the forecast, is understood not to be a 100% accurate representation of demand. First of all, it may be optimized, but for demand that never materializes. In addition, while undoubtedly manufacturing efficiency has improved greatly over the past 20 years, there are still machine failures and quality problems from tools breaking, raw material defects, incorrect inventory counts and a host of other reasons that make the output from a manufacturing site variable.

    Yet faced with this situation of variable demand and variable supply, many companies assume the output from their planning systems is optimized and accurate. I think they should be heeding your advise on forecasting and applying it to the supply side too. The plan produced by ERP and planning systems should be treated as a starting point to which human judgement is applied.

    Regards
    Trevor Miles
    Kinaxis

  2. Thanks Trevor for your kind remarks. I 100% agree with you. In the summer issue of Journal of Business Forecasting we have one article, “Worst Forecasting Practices In Corporate America and Their Solution– Case Studies,” by Lad A. Dllgard, a consulant. In this article, when Lad confronted the CFO of one large company, and told him that the company does not have a forecasting and planning process in place. His response was:

    “How can you say we don’t forecast? We already have a great forecasting capability in our ERP system! We simply plug in the product we need to build, and it blows out each item we need to buy or build and even lead-time.”

    I guess this kind of statement coming from someone from a large corporation won’t surprise you. We have made a progress over time with respect to forecasting and planning, but still a long way to go.

    Chaman
    St. John’s University

  3. Hello Chaman,

    My experiences in a variety of industry settings certainly support the position that models and statistical methodology are important but not sufficient to create reliable forecasts and plans. The models and statistical methodology address the systematic issues of the past inherent in the historical data. They can bring a factual basis important to a systematic evaluation and discussion of the future outlook in the forecasting and planning processes. But they have to be supplemented with market information, judgment, and business assumptions about the future. This can only be done with a well functioning, cooperative, and participative process. The companies with which I have worked that are having difficulties in their forecasts and plans may be experiencing technical difficulties, but most often are being affected by process problems that are overwhelming all of their other forecasting and planning activities. Process is more important than statistical methodology and systems, although one needs all working together and in the right proportions to get the best results. The process is the more challenging element of the forecasting and planning mix because of the many human interaction factors and qualitative issues that can be affecting it. A lethal problem, of course, is a process which does not have management support. I quite enjoy the technical side and would enjoy nothing more than to spend most of my time there. But as important as models and methodology might be, one cannot afford to underestimate the importance of a well designed, well managed, and well performing business planning and forecasting process.

    Mark Lawless
    Managing Principal
    Marlaw Business Advisory Services

Leave a Reply

Your email address will not be published. Required fields are marked *

WordPress Anti Spam by WP-SpamShield