Operational Data Stores Vs Data Lakehouses And All The Other Data Management Methods

A few months back, I posted a video discussing data warehouses, data lakes, and transactional databases.

But there are so many other methods technical teams use to manage their data, especially as AI and machine learning are having another resurgence. I wanted to go over the various ways companies are managing their data.

After all, without well managed data, all of the fanciest algorithms, models, and LLMs aren’t going to perform well (this is why I referenced the creation of data engineering being an indicator in the 10 years of the DE article). In many ways, data engineering and data management will continue to grow in importance in the near future as companies try to wrangle data sets that are growing in all forms of complexity. 

Since you’ll likely need to pick a data storage paradigm for your company, I wanted to go over a few common terms you’ll hear. I also wanted to provide definitions and illustrate how they’ve been useful to me in the past.

ODS - Operational Data Stores

I wanted to start by describing a less commonly referenced data storage solution–ODS, also known as an operational data store.

Unlike traditional data warehouses typically used for long-term storage and historical data analysis, an ODS focuses on providing a current, integrated, and consistent view of operational data from multiple sources. It acts as an intermediary layer between the operational systems (such as transactional databases, CRM systems, or ERP systems) and the data warehouse or data mart.