Your Data Warehouse Isn’t Integrated Just Because the Tables Are in One Place
Centralization is not integration!
Hi, fellow future and current Data Leaders; Ben here 👋
Today I wanted to talk about one of the major issues I still see companies face in 2026.
Data integration.
Before we jump into today’s article, I want to emphasize how important it is to connect technical solutions to real business outcomes.
That is the gap our team at Codestrap helps companies close. We start with the outcomes the business wants to improve, then combine data and software engineering services with our software factory to deliver solutions faster and more consistently.
Feel free to reach out if you’d like to check out what we are working on.
Now let’s jump into the article!
Many companies have successfully “centralized” their data without actually integrating it.
Centralization is not integration!
Integrating data across systems, teams, and business units remains a challenge. It’s also where a lot of the value of building a data warehouse comes from.
A warehouse is not integrated just because all the tables are in the same place, and that’s where many companies end up. They have a Salesforce set of customer tables and a product database customer table, but they don’t connect.
Now, many businesses want it all unified and integrated…but it’s just not where things end up.
For one reason or another, many businesses end up with a jumble of tables. So let’s talk about how many companies get there and how to avoid it.
The Original Promise of the Data Warehouse
One of the great things about working at Facebook was that its data was so well integrated across systems.
Prior engineers spent time developing systems that made it very easy to integrate from one table to another. There wasn’t even much you had to do as a data engineer in terms of getting disparate data sets to talk to each other. They’d done that in the application layer.
Which, in my mind, is the best layer. The less the data team has to do to fix application data and make it answer key questions, the better.
That way, when a business executive asks:
“Can we split up customers by, say, type of industry, or buying habits”
You can answer easily because you have a single customer table that you’ve taken and joined multiple source systems’ data into one, all thanks to there being a single customer ID that is across all systems.
This was actually the main focus of a recent project. There were several key entities that were spread across multiple systems.
But many data warehouses end up being a sandbox of multiple data source-specific tables. The marketing team has its user table, the operations teams have theirs, and of course, the data science team has its own.
The mistake many teams make when thinking of a data warehouse is that it’s supposed to just collect data and centralize it. It is also supposed to reconcile data across systems. It was supposed to create a shared enterprise view of the business.
That takes time. Data teams across the organization need to agree on everything from what ID will be used for the user to how to label specific entities. And most companies skip past that.
What Happens In Most Companies
In most companies, the data warehouse slowly becomes a reflection of the org chart or business units.
Enterprises in particular have multiple business units that might even all have their own C-suite with different marketing and sales departments that would benefit from integrating across business units. But whether due to politics or just simply because of a lack of communication, the systems don’t talk to each other.
There might be a dozen different data warehouses between the various units, all answering overlapping questions.
Here are a few other causes, and of course, most companies deal with more than one of these issues:
Mergers and Acquisitions
This comes up frequently in my work with private equity firms and growing organizations. Companies often merge, split, and acquire new businesses. Mergers and acquisitions, in particular, require integrating systems. Usually, the goal is to centralize operations onto one CRM, ERP, and other core business tools. But this takes time, and sometimes, it never happens. Just look up ERP migration failures.
SaaS Sprawl
Every team these days seems to have some SaaS product that has overlapping functionality. It makes sense how this happens. The sales teams have Salesforce, the customer success team might be able to use some parts of Salesforce, but maybe they prefer Gainsight, finance has to have NetSuite(no matter how much the data team complains about it), and so on. Every SaaS comes with its own version of customer, user, account, and other concepts that overlap. And all those teams have questions they want answered that involve a bit of data from each system.
No One Wants To Own The Decision
Not making a decision is still a form of making a decision. So if you don’t decide who owns the customer entity, multiple teams likely will make that decision for themselves. Each team will, in turn, define the customer entity differently and base it on a different source system.
None of these definitions is necessarily wrong. They are just built for different purposes.
The problem is that each team then starts building reporting, processes, and metrics around its own version. Over time, those definitions take on a life of their own. They find their way into dashboards, dbt models, and, of course, Excel spreadsheets.
Then one day, leadership asks what sounds like a simple question.
“How many customers do we have?”
And there is no simple answer.
Vertical Vs Horizontal Integrations
This is another interesting issue I often run across. Many businesses don’t just need to integrate data horizontally. That is to say, join customer data from Salesforce, HubSpot, and your internal product. Sometimes they also need to integrate data vertically. Meaning, you’ve got 4 different systems that have order data and need to stack them on top of each other, or 7 different places where patient claims data comes from. Then you’ve got to line it all up and be ready to add in future data sets. Sometimes this is due to mergers; sometimes it’s just the way the business is.
The Pressure to Deliver Quickly
I think it goes without saying that one of the common reasons data warehouses aren’t well integrated is that there is a never-ending pressure to deliver quickly. The challenge here is that you can often move fast initially. If all you need to do is answer some basic questions about the current system you’re working on. Great, pull out the data, ingest it into your data warehouse, and you’re done. But as soon as you ask more complex questions, well, now you have problems.
Whatever the cause, eventually, I find someone from leadership who finally pushes back. They are tired of having inconsistent numbers or having to meld multiple Excel-based P&Ls together to see exactly how their business is doing.
So they ask, what will it take to integrate across organizations?
Practical Advice For Fixing Integration
Integrating data across a large enterprise is hard. There is a reason many companies have ended up with not only multiple data warehouses, but also different data platforms.
Snowflake, Databricks, Postgres, BigQuery, and likely a few others.
So not only are there three different customer tables in the same data warehouse, but there are multiple instances of those same data sets across multiple platforms.
How can you fix this, and are there certain things you won’t be able to?
And I don’t believe it’s realistic to say, stop everything and take on a two-year enterprise data warehouse project. So here is where I’d start.
Start With the Business Questions
I wouldn’t start by saying, “We need a canonical customer table.”
That might be true. You probably do need one eventually. But starting there usually turns the project into a theoretical data modeling exercise. Suddenly, everyone is in a room arguing about what a customer is in the abstract. And although those conversations matter, they can quickly become disconnected from the business problem you were trying to solve in the first place.
A better place to start is with the questions the business is already struggling to answer.
Which customers are profitable? How much revenue did we actually generate across business units? Are we double-counting customers? Which products are growing? Which accounts are active in the product but not paying enough? Which customers are costing us the most in support?
Those questions will tell you where the integration problem actually is.
Pick the Core Entities That Matter Most
One of the mistakes I see large enterprises and medium-sized businesses make is that they try to model their entire company in one sitting.
This is a herculean task. By the time you’ve found every possible entity…guess what, your company has changed some part of its business, and what you’ve mocked up is no longer valid. Every company has a core part of its business that likely can be modeled in a reasonable time frame.
Think customer, account, product, employee, vendor, transaction, order, facility, location. These in themselves will have plenty of arguments and discussions. So start with your core model, the ones that could actually impact your business, and model that.
Assign Ownership for Each Core Entity
It’s difficult for a single data team to manage every possible entity that exists in a large organization. What I believe worked well at Facebook and other organizations is when specific data teams own specific core entities.
The People data team owns Employee tables. Depending on the type of business, Customers might sit under GTM, Finance, or a Billing data team, which might own revenue, invoices, payments, or accounts receivable. There will always be some crossover, but with the right incentives, tooling, and principles, I’ve found that this allows teams to build reliable data models.
Decide Which System Wins When Systems Disagree
One of the first projects I took on at Facebook was cleaning up the People data model. One of the issues was that there were 3-4 IDs that could be used to connect employee information.
That’s great, right?
Every table has multiple ways to join to others. No, it was far from great. It was confusing and opened the door for end-users deciding to remove IDs when they feel like building out their own tables. So we had to pick one ID to propagate into production tables so everyone knew exactly which ID to use.
Be Careful with AI on Poorly Integrated Data
One final point before my final thoughts.
AI, in many cases, is revealing how poorly many companies’ data integrations really are.
Put a text-to-SQL or conversational analytics tool on top of poorly integrated data, and it may generate a syntactically correct query that produces the wrong answer.
The tool does not automatically know how a Salesforce account, a Stripe customer, and a product workspace relate unless those relationships have been clearly modeled.
Be careful, there are a lot of ways data can look right, pipelines can technically work, and still give you bad results.
Final Thoughts
It’s easy to get all your data into Snowflake or Databricks. It’s hard to actually integrate it unless your data is integrated in the application layer(which, if it is, thank the data gods).
More than likely, if you work at a large enterprise, it’s not, and it’ll take some effort to get to a point where you can join all your data easily. But start with your core entities and key business questions. That will help you build a core data model that much of the rest of your data warehouse can be built on.
As always, I’d love to hear your thoughts here. What has worked in the past for you in terms of integrating your data? What hasn’t?
And thanks for reading.
Video Of The Week - Your Data Pipeline Is Lying To You - 5 Ways Your Data Pipelines Fail Silently
Articles Worth Reading
There are thousands of new articles posted daily all over the web! I have spent a lot of time sifting through some of these articles as well as TechCrunch and companies tech blog and wanted to share some of my favorites!
Most companies think they’re building a software factory. They’re actually just shipping bugs faster
Industrialized factories changed how the world produced physical goods: more output, lower costs, faster than anything that came before. Now a similar shift is happening with software.
LLMs have lowered the barrier to writing code, increased individual output, and pushed organizations to think about software development as a production system. The standard software development lifecycle and CI/CD practices that have held for decades won’t hold up under that pressure. That’s where the software factory comes in — and like physical factories, it needs more than speed to actually work.
The idea of a “software factory” started to solidify over the past year. Luca Rossi’s“The Era of the Software Factory” made the case plainly: AI is not just changing how fast people write code — it’s changing the whole production system around software.
Building Trust and Credibility is the New Moat for Engineering Leaders
We hear the word “moat” a lot these days, especially in the context of AI.
“Good judgment is the new moat for engineers”. “Harness is the new moat for AI products”. “Loop engineering is the new moat”.
So, in the spirit of a trending word, I’ll be sharing what I believe is the new moat (competitive advantage) for engineering leaders in this article.
The insights are based on what I am hearing and seeing regularly from talking with different engineers and engineering leaders across the industry, and what I recommend they do in order to become valuable at their company and grow in their careers.
End Of Day 222
Thanks for checking out our community. We put out 2-5 Newsletters a month discussing data, tech, and start-ups.
If you enjoyed it, consider liking, sharing and helping this newsletter grow.
When starting a new consulting project, I like to survey executives: "How many customers does your company have?" The range of answers (guesses) says a lot!
ZH
thank you for confirming that creating the CRM of my dreams IS possible.
🥹Be blessed.🙌🏾