That's why we need to advocate for lean data practices: pull only the data you need, use it until you don't need, then discard it. By adopting lean data practices, we can minimize data waste and optimize our data operations.
Excellent points. Sometimes you don't realize how bad the data in your source system is until you try to extract into a data warehouse to load . Worked on a team in the early 2000's to build a data warehouse from the source system. We discovered a lot of issues with source system we didn't know were there. So it helped identify and clean up issues in the source system too
That's why we need to advocate for lean data practices: pull only the data you need, use it until you don't need, then discard it. By adopting lean data practices, we can minimize data waste and optimize our data operations.
Great comment! How do you recommend teams implement lean data practices!
Just wrote a bit more about it today, sharing our use case, hope you find it interesting: https://leandata.substack.com/p/embracing-lean-data-practices-minimizing
Excellent points. Sometimes you don't realize how bad the data in your source system is until you try to extract into a data warehouse to load . Worked on a team in the early 2000's to build a data warehouse from the source system. We discovered a lot of issues with source system we didn't know were there. So it helped identify and clean up issues in the source system too
Oof...I have seen these types of projects.
Infrastructure for the sake of infrastructure