"If you don’t have an automated way to track dependencies, get ahead of the problem. Ask every team that relies on your data warehouse to submit a list of what they need migrated—set a deadline, and hold them to it."
Alternatively, look carefully at whether a metadata data catalog will address a good portion of the dependency analysis that you need, one-demand and in realtime. Depending on the technology and the catalog vendor, often a catalog alleviates a good amount of the documentation burden on dependency analysis and understanding end-to-end.
This is a great articles, but can you tell me the way how can we compare data in 2 database systems? Which tool should we use ( great expectations, soda, etc.)? And how we manage to run .in a different systems?
Great article! It reminded me of a migration I worked on from Redshift to Snowflake. We encountered some discrepancies when using the same query, though they weren’t always significant. For instance, one of our metrics was supposed to be 23.8, but it showed as 22.9. In such cases, what would you recommend? Do you think an acceptable threshold or range is appropriate? When we discussed this with stakeholders, they mentioned setting a range, with anything within that range being acceptable. I’m curious to hear your thoughts on this approach! :)
That’s very true for data migration projects, another big factor is not buying in business users that lead to adaption failures to new platforms.
"If you don’t have an automated way to track dependencies, get ahead of the problem. Ask every team that relies on your data warehouse to submit a list of what they need migrated—set a deadline, and hold them to it."
Alternatively, look carefully at whether a metadata data catalog will address a good portion of the dependency analysis that you need, one-demand and in realtime. Depending on the technology and the catalog vendor, often a catalog alleviates a good amount of the documentation burden on dependency analysis and understanding end-to-end.
This is a great articles, but can you tell me the way how can we compare data in 2 database systems? Which tool should we use ( great expectations, soda, etc.)? And how we manage to run .in a different systems?
Great article! It reminded me of a migration I worked on from Redshift to Snowflake. We encountered some discrepancies when using the same query, though they weren’t always significant. For instance, one of our metrics was supposed to be 23.8, but it showed as 22.9. In such cases, what would you recommend? Do you think an acceptable threshold or range is appropriate? When we discussed this with stakeholders, they mentioned setting a range, with anything within that range being acceptable. I’m curious to hear your thoughts on this approach! :)