Tech teach-in: Data Scientist vs Data Engineer

It’s all about data – collecting data, storing data and analysing data. With data at the heart of the technological revolution, finding the right talent to handle it is crucial.

Two of the roles that have been singled out as emerging are data scientists and engineers. But how do they differ?

A data scientist is both a computer scientist and a mathematician. This is definitely a growth area with the rise of big data. As companies get overwhelmed with data, they need a data scientist to help them make sense of it and come up with actionable insights.

Companies have been investing heavily in data hubs to understand more about their end-users and customers. A data scientist can help interpret the data and to give meaningful feedback. Skills needed involve machine learning, data science and python (programming language).

Meanwhile, a data engineer, like all engineers, is interested in the ‘’how to’’. This means they are in charge of pipeline, data workflow management and ETL processes (Extract, Transform, Load).

Data is only as good as the database it comes from, so a data engineer is constantly building, testing and maintaining the processing system. Data insights can now be delivered quickly, especially in cities which have lightning-fast connectivity. And so a data engineer has to think and act fast.

Top skills needed include Apache Spark, data engineering, Scala and Hive.