IBF spoke to Dr. Eric Siegel, author of the best-selling book Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die, and host of the Dr. Data Show on YouTube. He reveals why predictive analytics has become a critical best practice for all businesses and how – when misused – it can further marginalize society’s most vulnerable groups.

 

What is predictive analytics?

Let me give three definitions. The first is that it’s the business application of machine learning. The second, more common functional definition, is that it’s technology that learns from data to make predictions about an individual person or case in order to improve company performance. The third definition—where predictive analytics meets forecasting— is that it’s the deployment of machine learning methods such as decision trees, log-linear regression, ensemble models and neural networks. These methods serve to predict demand. When applied for forecasting, rather than generating a prediction for an individual customer, corporate client, or transaction, which is what predictive analytics is usually employed to do, it simply provides a forecast for the item and time range in question.

How does predictive analytics work?

Imagine you have rows of data available from which to learn. In the case of a classical business application, each row corresponds to a customer and contains the outcome or behavior about the customer you’d like to predict—such as whether they committed fraud, whether they canceled their subscription, whether or not they turn out to be a good credit risk, whether they defaulted on their debt, or whatever outcome is relevant to your business.

In the case of forecasting, each row includes or summarizes everything you knew up to this point in time about a particular SKU or whatever you’re trying to predict demand for, along with the outcome, i.e., what the actual demand ended up being over the next week or next month. These rows compose your training data from which correlations are discovered between the data points you hold about the product and the outcome.

Are machine learning and predictive analytics synonyms?

It depends on the context, but, when you’re talking about these kinds of business problems, they’re entirely synonymous. A lot of confusion surrounding this technology is due to nomenclature. A few years ago, the industrial world started using the word machine learning. Before that, the term was only common within academia and research and development.

When it comes to this technology and demand forecasting, you can just as well call it machine learning as predictive analytics. Whether we’re predicting the likelihood of somebody defaulting on a loan or predicting demand for a product, the same exact core analytical methods apply.

To complicate things further, what about prescriptive analytics?

Prescriptive analytics is a superfluous term, which has never been agreed on as far as its definition. The way it’s usually used is to imply that predictive analytics is not enough and now you need additional technology in order to translate what you’ve predicted into actual action to be taken. But predictive analytics has all along been intended to directly inform actions to be taken such as whether to make marketing contact, whether to audit for fraud, or approving someone for a credit card application. The term prescriptive analytics implies there’s an entirely new field or technology. There isn’t. Predictive analytics as a field already includes field deployment, i.e., the translation into action.

How does predictive analytics differ to more traditional statistical modeling?

Machine learning is statistical modeling. It’s just that we don’t use the word statistics because it sounds boring and it kind of undersells it because the core methods have advanced so much since the term statistical modeling originated. Predictive analytics is just the use of those methods for business applications. They all make use of historical learning cases, i.e., data, and they all find patterns that map from the things we do know to the things that we’d like to predict.

What impact has predictive analytics had on businesses over the last few years?

Predictive analytics has become a critical best practice for all businesses that operate at scale. It empowers businesses to execute on mass-scale operations more effectively. Business is a numbers game. Mass marketing, for example, is mostly just junk mail. Investigating fraud is mostly looking at transactions that are not fraud. Most resources and time in that sense are intrinsically wasteful.

The question is, how can you make all these mass operations more effective? By predicting. If you can predict for each individual case, that is, put a probability on it, then you know whether or not to risk spending $2 to send somebody a marketing brochure, or the risk of not auditing somebody for fraud. So predictive analytics generate probabilities, which allows us to manage risk. If you take a marketing campaign and improve its return on investment by a factor of three, that’s a valuable business outcome.

How can companies get started in predictive analytics?

E: There needs to be a champion at the company who becomes familiar with the field by reading or undergoing some preliminary training. Then it’s about defining the right pilot project. Where’s the lowest hanging fruit, the greatest opportunity? What’s the large-scale operation that occurs repeatedly— whether it’s a forecasting function or marketing or what have you—that could stand to benefit? Then you just need to define the scope for the project, do a preliminary analysis of the data, and, usually, engage with an external resource like professional services to get that first project off the ground.

What is big data and why is it valuable for businesses?

Big data and data science are simply umbrella terms that mean we have a lot of data and let’s do something useful with them. Big data did originally mean data that is too big for traditional database technology, but that’s not how it’s commonly used now. These days, both big data and data science allude more to a culture than any particular technology or value proposition. When we talk about using big data, we’re just saying, “We’ve got smart people who are working on using data to derive value”.

What kind of data do companies typically hold that could be used for predictive analytics?

The good news is that anything you do repeatedly is the exact thing that creates the data you need to improve that process. In marketing, it’s simple—you’ve got all the demographic profile and behavioral data about that customer, like what they’ve bought up to that moment and other data points. For forecasting, it’s very similar. Everything that’s been going on with the sales of this particular product up to this moment in time, as well as other external factors which you believe could influence demand, can be leveraged by predictive analytics.

One of the most famous cases of a company benefitting from predictive analytics was when Target marketed to women they identified as likely to be pregnant. How did they figure that out?

That’s an unusual case because normally in marketing applications, you’re predicting for a particular product or a set of products. This is a little different because they’re trying to predict something about the current state of the customer rather than their future behavior. If we think about training data with all the things we know about the customer, Target shored that up with data from customers who voluntarily told Target that they were pregnant. Then the analytics process ends up just being the same thing as for any of these other problems we’ve been discussing. They analyzed all the demographic and profile information about the individual customer and the purchases that they’ve been making, especially more recently, to identify which customers are likely to be pregnant. Then they market appropriate products to that group.

When it comes to implementing predictive analytics, is the challenge not so much building or accessing the technology itself but building a culture around it?

Yes. That’s a parallel challenge and is often the greater challenge. Predictive analytics needs to be seen as an organizational process and the business value needs to be the leading motivator. The actual deployment has to be an integral part of the project from inception. To that end, you need to lead with that value proposition. Let’s say we’re selling this idea internally to marketing. We tell them we’re going to market to only the top 30% predicted most likely to buy and in so doing, we’ll save 70% on marketing costs and here’s our estimated ROI.

These metrics are why we’re doing it—you don’t lead with machine learning or the analytical methods, you lead with a value proposition, the functional purpose of the project. Then, to the degree they need to how the underlying technologies work, you explain the technical aspects that we’ve been discussing.

Do businesses need Chief Data Officers?

I’m actually agnostic about that. It certainly can’t hurt in terms of improving visibility of the analytics function and encouraging a general culture of using data and finding ways to do it. Of course, it depends on the individual organization. The very first Chief Data Officer was at Yahoo. It turns out the job title of Chief Data Officer was first suggested as a joke and everybody in the room laughed. But it stuck and it became a whole new thing.

In your book you say it’s impossible to predict with high accuracy. In business, what kind of accuracy do you need?

Well, accuracy is not the right measure. Rather, you want measures that tell you how much better than guessing your predictions are. As previously mentioned, business is a numbers game.  You want to tip the odds in your favor by predicting better than guessing. That can have a dramatic effect even if it’s not “accurate” in the conventional sense.

For example, I can identify the 20% of customers that are three times more likely than average to buy, but it might be that those people overall have just a 6% chance of buying. I don’t have high confidence that any one customer’s definitely going to make a purchase. But knowing that group is three times more likely to buy than the average tells you that the model has learned something. Instead of prediction accuracy, we should ideally measure the value in business terms like return on investment or overall profit of the project or marketing campaign. If you can translate it into those terms, which you often can, that’s the number that matters, not the predictive performance of the underlying model.

There are now off-the-shelf predictive analytics/machine learning tools that are very cheap. Are these black box systems and can they be trusted?

Most of the software products out there are not black box. There are well-known standard machine learning methods such as the decision trees and neural networks methods we have mentioned. The software products have lots of different bells and whistles and variations to them, but more or less they use known methods or algorithms, and they don’t hide how they work. There are exceptions. There are some machine learning products that build branding around having a “secret special sauce.”

For more technical users and experts, including myself, that creates suspicion and raises questions regarding credibility. After all, if you want to sell a product to experts in the field, it doesn’t make sense to keep the core analytics a secret and market the product as some magical tool. Transparency, as well as cost, user interface, and how well it integrates with existing operational systems and databases should be the selling points.

Many people are still forecasting in Excel when these tools are now easily accessible. What do you think is stopping widespread adoption?

There’s a lot of complexity in moving from Excel to proper machine learning. It’s a completely new endeavor and you have to get together training data, apply methods that may be new to you, and then be able to interpret the outcome, results and performance. It’s a big project. The thing is, there has to be a business reason or incentive to do it. How much ROI would there be if your forecasts were 10% better, for example, and how much is that worth to you?

What is the persuasion paradox and why is it important?

The paradox is that, although we can’t perceive or record individual cases of causation or influence, we can predict it. You can’t conclusively observe persuasion, even in retrospect. If you look at a customer, why did they buy the product? Was it because of your marketing initiative? You can never be entirely sure because they might have bought the product anyway. But we can predict the likelihood of causation over large numbers of individuals.

On the other hand, in forecasting, observing influence is generally is not a concern. We just want to know how many widgets are going to be sold next week. We don’t care why we’re going to sell a lot of widgets next week. We just want an accurate estimate of sales.

What are some examples of unethical uses of data?

The first one that comes to mind is using a protected class such as the race, gender, or religion of an individual in order to derive a consequential decision such as whether they get approved for a loan or sentencing in prison. Let’s imagine that a manager is interviewing a candidate for a job, and they use predictive analytics to gauge the candidate’s likelihood of success in the position. The manager might reject the candidate based on the results provided by the system. When the candidate asks the manager for the basis of their decision, they would have to say, “Well, for one thing, our predictive model penalized you by seven points because you’re black.” Normally, protected classes are disallowed as direct inputs, but there are experts, including university professors involved in the legal justice system, campaigning to purposefully introduce then as direct inputs. The risk is real.

What excites you about predictive analytics?

I’m a former academic and I think that technology that learns from examples and, therefore, from experience to draw generalizations that hold in future cases is the most interesting kind of any science and technology. It’s the most exciting science period. The fact that it naturally ends up being valuable only makes it that much more exciting.

To read the full interview with Dr. Eric Siegel, download the latest issue of the Journal of Business Forecasting. When you become an IBF member, you’ll get the Journal delivered to your doorstep quarterly. Get your IBF membership.