Read This Before You Use Response Models

Response models are used in industries such as telecom and retail to do targeted marketing. The thought process in building a response model goes like this:

• I want to predict who would churn (leave my services) or who would respond to my marketing offer.
• Okay, seems like a classification problem, so why not build a classifier. Let’s build a classifier then.
• I have a classifier, let me use it to obtain probability scores: P(Response|attributes)
• I have the scored file, I do a Pareto analysis on it and figure out what percentage of high probability customers (scores obtained using the classifier) constitute the people who actually responded to the offer.

My pareto analysis tells me 30% of high probability customers constitute 70% of the people who actually responded to the marketing offer. Heee haaaa, I’ve got a good model!!!!!!

Wait a minute!!! Let’s control the excitement for a while. Even if my pareto analysis was as good as mentioned in the last bullet point, it doesn’t mean very much. Now some of you would ask what’s wrong with the analysis?  Well, quite a few things. Let me explain with an example:

I am a very heavy user of taxi services provided by a taxi aggregator, let’s call this taxi aggregator as aggregator “O”. I often get promotional offers from this service. I do respond to their offers not because offers were made to me but I would have any ways availed of the services because I “like” the service. So should I be sent these offers? I honestly think that the company is wasting money by providing me offers as I would anyways avail of their services irrespective of the fact if the offer was made to me or not.

On the other hand my friend who is always on the lookout for “deals” would avail the services of this taxi provider only if the offer makes sense to him. Also from the stand point of the effectiveness of any marketing campaign a campaign should be considered effective if it was able to target the people who would have acted solely because of the marketing efforts and not due to any personal biases!!!

Now think about response models, a response model is going to compute P(Response|attributes) while ideally it should compute the probability of response due to marketing efforts alone.

So doing response modelling is never going to tell me what the incremental impact of the marketing activities was. I might end up targeting people who were loyal customers and would have continued to avail my services irrespective of the fact if the offer was made or not. So the problem of optimally spending marketing budget will not be solved by a response model. What should I do then?

The solution lies in creating an uplift model. Here the idea is to find the probability of incremental impact of a marketing activity using some sort of baseline. Usually a carefully chosen “control group” serves the purpose of figuring out the “incremental impact”. An uplift model is going to give you the probability of response due to marketing efforts alone.

Many statistical tools such as SAS and R give you the capability to create an uplift model. In the next post we will discuss how we can do uplift modelling in R.. 

Suggested Read:

The Wonderful World of Geospatial Visualizations Explained in 2 Minutes

Geospatial Visualizations- Extracting Data with lat-long Information From Shape Files