Data Misinterpretation: Why Faulty Conclusions and Bad Outcomes can Happen

There are three components required to make an expert business decision based on data :

1. Statistical knowledge/ Quantitative aptitude
2. Domain Knowledge
3. Business Context

To make data driven decisions using a mathematical approach, it is important to have a perfect blend of all the above factors. One can create an extremely robust model where the results are statistically significant but they may not be applicable to the business world.  Let’s take a look at some possibilities of when that can happen:

1. 1. Correlations – It is important to understand that correlations are not causations. If we run correlations on two columns, correlations are going to tell us if they are moving in the same or inverse directions. We cannot assume that one causes the other. For example – If I have data for ice cream sales and discounts on cornflakes. Even if they show a positive correlation we cannot conclude that the discounts led to increase in sales of ice cream.

1. Insufficient Domain Expertise – Statistically a model may suggest that if we increase the price of a product by 30%, it will lead to increase in revenue. In reality we may not be able to do so. Business cannot make drastic decisions given the reality of competitor existence and the attrition one may witness after implementing such a decision. Understanding of the domain along with the right context of the business for the study at hand is very critical. No two projects can be approach in a standard way and can have different perspectives depending on the context.
2. Lack of Data- Many a times, we may not be able to capture all the factors in a model. For example competitor data is not always available to us but maybe influencing our target. Or there might be factors that may not be captured accurately in the model due to not having access to the right data. For example we may not always have the data on clicks on a website so instead of that we may capture only for its presence or absence which may not give the right picture.
3. Business context – Statistically we may get a great fit, but if some very important variables are not staying in the model, we cannot apply it to the business. For example – If TV does not stay in the model despite the fact that the business has invested a lot on it, we cannot conclude that it has been ineffective. We may look at what we are using in the model – spending instead of TRP (approximation for reach),  weekly reach instead of long term stock (Adstock) variable, etc and fix it accordingly.
4. Aggregations and Interactions – A lot of factors come into play when we are building a statistical model. It is rare that only one or two campaigns over the year tmay influence the target. Since many factors are involved, we may end up into collinearity issues. In such cases, we create interaction variables and study the combined impacts instead of isolated one. It may be difficult to segregate how individually these factors maybe affecting.
5. Visualizations – When we present the statistical output to the business, its important that correct interpretations are done with clarity and simplicity. We may also have to check for right scales and right alignment of the data else it may mislead the interpretations.
6. Updating the results periodically – Once a model is run, we need to check the results for their validity over a period of time. Statistically the model can run and even give results but their validity has to be tested over time to ensure that economic and business changes are taken into account and the basic assumptions hold true.

So, hope these examples have helped you understand the importance of using data to make effective decisions that can actually impact the business profitability. If you found this interesting, why don’t you go on and explore  another article that talks about which facets of Data are more important? Quality, Quantity or Both? Here Susan Mani explains that while a certain minimum quantum of data is required to run models, which can sometimes be a bottleneck while modeling, more often than not, quality tends to be a bigger problem.

Suggested Read:

Open Data frameworks can Positively Impact the Economy, and Find You the Perfect Parking Spot

The Hidden Biases in Big Data