Selecting the right model plays a very important role in predictive analytics and forecasting. Use the wrong model, and you might as not have bothered at all. Use the right one, and you have a robust forecast you can plan your business operations around. So how do you choose the right one?
Each model captures a specific data pattern, each model has a shelf-life of its own, each method yields unique results, each model reacts differently for different time horizons. There are hundreds of variations of baseline methods that can be combined into thousands of models with unlimited steps and inputs you can choose from. Finding or building the perfect one may be a one in a million proposition.
Finding the needle in the haystack, or the best method or model, goes back to all the work we put in prior to arriving at this step. It is about understanding the problem (whether it be for descriptive or anomaly detection or clustering or regression or predictive,) categorizing the inputs and outputs, and knowing your data and its limitations. It comes down to a combination of business need, specification, experimentation, and time available.
Keep the following specific points in mind when finding, building, using, or analyzing any model or method:
One Size Does Not Fit All
The one thing we know for sure when it comes to modeling and predictive algorithms is that despite all the possibilities available, there is no one approach that caters to all your problems. Even the most experienced data scientists cannot tell you which algorithm will perform the best before experimenting with others. The good news is you don’t need to get it right first time. You can pick or build an algorithm that nearly solves your problem and then, over time, customize it to improve it to solve your particular problem.
Keep It Simple
It is easy to get lost in the details or think bigger is better, but it is best to select simple methods initially and use simple procedures unless you can clearly demonstrate that you must add complexity. Complex methods may include errors that propagate through the system or mistakes that are difficult to detect. The more complex and the more features there are, the more specialized techniques you need. If can’t explain it, then you probably can’t use it properly. Start with what you know and what you can do and then experiment, adjust, and iterate models over time.
Forecasting/Analytical Models Should Meet The Situation
The predictive or analytical model should provide a realistic representation of the situation. You need the right test of models, inputs, parameters and situations to match the current problem. You’re looking for the right balance between accuracy and the potential for overfitting. You also need enough time to develop and train/tune the model.
There Is No Magic Bullet For For Forecasting Models
Because we know that no model in the world works in every situation, it is important to look at different ones. By combining forecasts, you can incorporate more information than you could with one forecast. Studies have even show that combining methods reduces error by 12.5 percent compared with a single method. Combining can also have the potential to reduce risk due to effects of bias associated with a single method.
Forecasting Models can get old
Just because it works the first time does not mean it works every time. Patterns change, data changes, features change, and reactions within models change. Update models frequently as the underlying data or environment changes or revise parameters as new information is obtained. Make sure you use quantitative approaches and objective tests of assumptions and models.
Bottom Line
There is no “one fits all” algorithm or model. Choosing the right one depends on several factors including:
- Purpose,
- Data size, quality and diversity;
- And resources available.
There are also additional considerations like accuracy, training time, volume, parameters, data points and much more. This is where we come in, and it is the demand planner’s role to help choose the right model that fits the data and the underlying truths, utilizing our experience and professional knowledge.