Evolution of the Data Scientist Through the Decade: What’s Changed

I was invited to be a part of a panel discussion at the recently conducted Cypher Analytics Summit by Analytics India Magazine. While the topic of the discussion was ‘Key skills for a data scientist”, it was interesting to note that all of us (the panelists) spent a fair bit of time discussing how the skill requirement has changed over time. I think this change in skill requirement reflects the evolution that has happened in the last 10 years in the field of analytics and would make for an interesting read for anyone in the field or looking to get into the field.

Hiring Trends in 2006

I have been a part of the analytics industry for over 15 years now. I started building analytics teams back in 2005. Back then, building a new team meant hiring a bunch of statisticians, economists and mathematicians. We would hire people who had a comfort level with numbers and some basic understanding of statistics. We would then teach them business analytics. This meant a quick refresher of statistical concepts like chi-square, p-values etc. followed by training on SAS (in most cases) or R/Weka/SPSS (in a few cases).

We would then teach the new recruits to do specific analytics tasks. Marketing Analysts would learn to build market mix models or price promotion models based on the team they were part of. Financial Analysts would learn to build credit risk models or fraud identification strategies. People would focus on learning the specific task that they would be doing in their role and get started. Over time, analysts would move across teams, across clients and across domains, and this would broaden their skill set over time.

Everyone was called a Data Analyst or a Business Analyst. Similarly, there were BI analysts or Reporting Analysts or MIS Analysts who looked after the reporting piece.

Hiring Trends in 2011

Over time, businesses realized the importance of domain knowledge in analytics. The analytics industry at the time was dominated by people with strong quantitative skills that were not necessarily complemented by deep business knowledge. Analysts were working on complex analyses for products they had never used, for markets they had never been to and for customers they could not relate to. This caused obvious problems in the industry and businesses started to pay special attention to providing the right level of business understanding to the analysts. There were many changes that came about because of this.

Domain knowledge became a strong part of the analytics training. I remember the first time I took my team to visit a Walmart store. They could not believe the scale of the store in terms of size and assortment. Suddenly, all those discussions about space planning and aisle management started to make sense to them.

We also started hiring more MBA graduates as well as people from a domain background. For ex: a retail analytics team will have a non-analytics person who understands the retail business very well.

Analytics slowly became a domain with an eclectic mix of people – MBAs, Engineers, Physicists, even Psychology graduates.

Disclaimer: This website may include links to third party websites & you maybe redirected there as the user of this website. UNext is not responsible for the content of any third party websites; thus the views and opinions expressed in the publications/ third party websites are those of the authors and do not necessarily reflect the views or positions of any entities UNext represent.