Big Data Engineering for Machine Learning

Data engineering teams are nothing new, they have been around for several decades. Their role has been most recently extended from building data pipelines that only support traditional data warehouses to also building more technically demanding continuous data pipelines that feed today’s applications that leverage Artificial Intelligence and Machine Learning algorithms. This document covers the most popular engines used to build these pipelines. It delineates the synergies between data engineering and data science teams.