A Decade In Data Engineering - What Has Changed?

The concepts and skills data engineers need to know have been around for decades.

However, the role itself has really only been around for a little over 10 years, with companies like Facebook, Netflix and Google leading the charge. 

Throughout those ten years, there were significant breakthroughs and tools through hype, and general acceptance became standard (if even only for a few years). Each of these tools and practices shifted how we as data engineers operate. 

On top of that, various regulations and legislation reignited a need for better security and governance policies that have always been crucial in data management.

Each of these various turning points pushed data engineering as a discipline to the next level. In this article, we will be going over the 10+ years of data engineering history that got us to where we are today.

2011-2013:  Why Did Data Engineering Become Popular - The Rise of Big Data and Hadoop