Data Science & Data Engineering

I was very happy to participate in an NRC workshop on “TRAINING STUDENTS TO EXTRACT VALUE FROM BIG DATA“.  The discussion was very interesting and there was a terrific mix of very impressive participants.  Near the end, there was a slight contention. One dimension of that was what is perhaps a useful distinction that we should make when talking about Data Science generally.  Many of us talk about databases in Data Science, however, some people think of that in very ambitious, technically advanced and interesting ways.  On the other hand, practitioners (and people thinking from the data analysis perspective), typically think of using databases in a very simple-minded manner, and perhaps as being something dictated by the provider of the data.  As a result, many topics important in database design and implementation are not as relevant to consumers of databases.  So perhaps it is important to think of two types of data scientist – the consumer of data focusing on data analysis, and a different group who are “data engineers” and who design data products and implementations and who can architect important frameworks for the analysts to consume.

So data science may be better described as having sub-categories of data analysts and data engineers.