In the year 2001, William S. Cleveland published a book called “Data Science: An Action Plan for Expanding the Technical Areas of the Field of Statistics.” This was the beginning of a new kind of statistics, brought about as a result of the new, data-heavy internet era. Professor and Statistician Jacob Zahavi is quoted as saying “Conventional statistical methods work well with small data sets… [sic] Today’s databases, however, can involve millions of rows and scores of columns of data.”‘
Statistics had to turn to computing to handle the increasingly weighty databases that needed processing. Thus, Data Science was born, and in the years following the field has gone through rapid changes.
In the early days of Data Science, statisticians were using their old, on-paper mathematical models primarily, just beginning to integrate automation in the form of computing to handle the loads. Increasingly, in recent years, more and more tools exist to replace those old methods.
The modern Data Scientist doesn’t need to know nearly as much in order to perform the same tasks as before. This property of software tools is often referred to as the”black box” – it performs a certain function, and the scientist does not need to know how it does that. The process is quicker and easier, but less flexible as the scientist can only use it in the way it was designed to be used.
The focus of Data Science in recent years has been in understanding machine learning as opposed to understanding the deep statistical theory. As the job has gotten easier and more lucrative, it’s also gotten more popular. Data Scientists are in high demand in big companies now, though corporate understanding of the field has yet to catch up. Many companies have different expectations of what a data scientist will offer their company.
Still, more and more people in the field are experiencing satisfaction and success. Three years ago, 67% of polled data scientists claimed they were at least happy in their job, but in 2017 the number rose to 88%
Not to mention, the increasing demand for a data scientist in the technology industry. In CrowdFlower’s 20017 Data Scientist Report, it was revealed that about 90% of data scientists are contacted at least once a month for new job opportunities.
The field is experiencing a boom that will continue for many years. Data Science today is more efficient, better understood, better appreciated, and more technological than ever before.