Really enjoyed this! It’s interesting to see how, despite all the new tools, platforms, and AI capabilities, the core challenges in data engineering, like data quality, schema design, query performance, and aligning with business priorities, remain largely the same. Your point about business alignment is so true: technology alone isn’t enough if it doesn’t deliver real value. A great reminder that strong fundamentals still matter most.
You can post this every year and will be spot on every time 😅 I think that with new hype driven trends like databricks vs Snowflake, dbt vs SQLMesh, AI and the MDS speech we are always finding an excuse to avoid working on the right things, which are listed here and they will rock any other shinny project that involves AI or similar
LOL at the meme (soooo true) and a note on #3, definitely an art that needs to be applied thoughtfully as you said. Thanks to suggestions by LLMs I've seen coders indexing everything as a first pass on troubleshooting with no performance improvements... just more clutter to the database
Really enjoyed this! It’s interesting to see how, despite all the new tools, platforms, and AI capabilities, the core challenges in data engineering, like data quality, schema design, query performance, and aligning with business priorities, remain largely the same. Your point about business alignment is so true: technology alone isn’t enough if it doesn’t deliver real value. A great reminder that strong fundamentals still matter most.
You can post this every year and will be spot on every time 😅 I think that with new hype driven trends like databricks vs Snowflake, dbt vs SQLMesh, AI and the MDS speech we are always finding an excuse to avoid working on the right things, which are listed here and they will rock any other shinny project that involves AI or similar
LOL at the meme (soooo true) and a note on #3, definitely an art that needs to be applied thoughtfully as you said. Thanks to suggestions by LLMs I've seen coders indexing everything as a first pass on troubleshooting with no performance improvements... just more clutter to the database