Even before the pandemic, foward-thinking retailers leveraged AI and machine learning technology (ML) for demand forecasting. In the post-Covid world, however, those ML models have failed to provide accurate predictions because they don’t know that the data they use is now obsolete due to changing demand patterns. How can we upgrade them to the new reality?
There are six possible ways to get a more accurate forecast:
1 – Gathering data on new market behavior: As dynamics within the new market stabilize, use that data set to create a new model for forecasting demand.
2- Use a feature engineering approach: Track external data sources like price indices, market states, latest news developments, exchange rates, and related financial/economic factors. Using these, models can generate more accurate predictive outputs.
3 – Factor in up-to-date POS data: Analyzing recent POS data can allow us to observe and react to real-time shifts in patterns of demand, improving forecast reliability. Depending on a given product’s classification, the appropriate range for a POS data set might be between a month to two months.
4 – Use the transfer learning approach: If we possess any data sets relating to historical pandemics or behavior based on similar principles, we’re able to use that data within the context of this present-day pandemic.
5 – Utilize a model for information cascades: Merge the cascade modeling with current POS data sets to create a demand forecasting model that is able to recognize aggregated consumer behavior patterns and predict herd patterns for future sales.
6 – Leverage NLP (natural language processing) technology: NLP analyzes actual consumer comments and posts from an array of social media sources, from media platforms to popular social media sites. NLP can use sentiment analysis algorithms to collect and analyze conversations and discussions from real customers. This gives an unfiltered look at consumers’ behavioral patterns, preferences and attitudes.
If you are looking for a way to improve your current ML models and thinking of building a demand forecasting feature from scratch, this will help you to choose the best approach depending on your business type.
A data scientist generally works with historical data, and it’s impossible to predict such drastic changes as a worldwide pandemic. But as a general rule, you should prioritize flexibility in retraining your models, add more external factors as predictors, and account for a short-term perspective as long-term models become less relevant.
We tackled this problem with pre-Covid models for a restaurant business. Here is an example from our dashboard:
Forecasting dashboard showing normal revenue before the pandemic.
Forecasting dashboard revealing revenue post-lockdown.
In this case, we rebuild our models from scratch, losing a degree of accuracy. At this point, the historical data are not relevant anymore and we wait for new statistics and patterns inside the data.
Not every demand forecasting article on the internet fits the particular needs of your business and industry. Effective approaches vary dramatically depending on business types – here are some of the distinctions and varying ways to address demand forecasting:
Small vs Large Businesses
Small and large businesses should be approaching forecasting in completely different manners. First of all, the acceptance criteria for huge businesses is significantly smoother than for small businesses. We can have a higher error for prediction in quantity, as high sales volume allows for greater tolerance for error. When it comes to historical data, large businesses have a higher volume of collected data, making it easier to identify patterns in customer behavior. With small businesses, it’s often necessary to test your hypothesis to prove the correlations between sales volume and predictors.
Demand for a product at a large business, showing clear demand patterns.
Demand for a small business showing erratic sales patterns.
Online vs Offline
Online sales allow for a greater range of predictors and external factors in your modeling. It’s not necessary to have a POS (point of sale) system, as you’re able to collect all relevant data from the website. You know more information about customers and collect their historical purchases. With different predictors, you can apply a wider range of machine learning models: Gradient Boosting, Random Forest, SVR, Multiple Regression, KNN, etc. With offline sales, you are often limited to historical sales only. The best approach here is to use Time Series Analysis.
The USA vs Europe
Regional differences play a huge factor in predictive modeling, as varying locations will have different behaviors both in specific sales and general cultural factors. It’s worth noting that certain regions’ consumers are influenced to different degrees by marketing campaigns. Additionally, holidays vary by region, and you’ll need to decide whether to add this feature to the model or not. Take into account different legal constraints (limit for product amount) etc.
Perishable vs Non-Perishable Products
Finally, it’s crucial to factor product type within your demand modeling. With perishable products, you need to set up the right metrics to penalize the model when the prediction is much higher than the real value, as the consequences for excess inventory are significant. You need to be careful with data preparation and work with outlier detection because this prediction should be extremely sensitive to any changes.
Conclusion
The pandemic has made us all hyper-aware of the limitations and constraints of forecasting models, but the best practices in forecasting are the same whether we’re in a pandemic or not. You should always be striving to fit your particular business model and business needs to your forecasting tools, to attract new customers, increase your revenue, and expand your market share.
To find about more about practical applications of machine learning models, pick up a copy of Eric Wilson’s new book, Predictive Analytics For Business Forecasting & Planning. Written in easy-to-understand language, it breaks down how machine learning and predictive analytics can be applied in your organization to improve forecast accuracy and gain unprecedented insight. Get your copy.