Seattle Data Guy In 2023
Reviewing my client trends and your favorite articles
On December 17th, 2021 I left Facebook. My consulting business which I have been running for the past 6-7 years finally started taking off. January and February were a little slow but by the time March came around I was in full swing.
Requests of all kinds were coming in.
By the end of the year I:
Wrote 40 proposals and worked for 20 different clients
Filmed 54 videos
Wrote 60+ articles and newsletters
Gained 25,000 new newsletter subscribers
And recently I just past 50,000 Youtube subscribers
But enough metrics.
Let’s review everything from the type of client work I did to the most popular articles and videos I put out. We’ll cover:
Client Trends - I will review the types of clients I took on, both in terms of size and type of work.
Most Popular Articles - I am going to take a page from the Pragmatic Engineer and reference both your favorite articles as well as a one of my own personal favorites.
Most Popular Videos - In many of my newsletters I will share a video from the Seattle Data Guy Youtube channel. Here were the most popular from 2022.
1. Client Trends
I have reviewed my past year's clients and categorized them. Making a list and checking it twice as it were.
In a past article, I claimed that 60% of my client work was fixing key person issues ( described later). This was recency bias as I was incorrect about this claim.
A little over 30% of my client work in 2022 was key person issues. Whereas the other major categories were green field projects and solutions and design consulting. These are my bread and butter in terms of projects.
Below you will see the rest of my breakdown as well as a quick explanation of each of these categories.
Here is a quick definition of each of the types of consulting.
Key Person Issues( 32%) - Many companies rely on key persons/teams to develop their data infrastructure. This works for a while. Problem occurs when these individuals leave and there was no hand-off/no documentation that allowed the next team to pick up where they left off. Perhaps the company decided not to even rehire a data team for a while until major issues started to arise. These projects often require a lot of reverse engineering, reading through limited documentation, and a whole lot of patience from everyone involved.
Green Field( 31%) - For those who haven’t heard this term before, green field refers to projects where a company has no prior infrastructure. These projects require a combination of solutions architecture work as well as hands-on work. The focus is building a complete data stack from the ground up.
Solutions And Design( 20%) - Several organizations last year asked me to look at their current systems and help them find the right solutions to migrate to or redesign them from a high level. These often didn’t require me to code and instead were far more about making sure I look through all the options, run interviews with stakeholders and understand the company's requirements.
Marketing( 10.5%) - This category isn’t consulting-based. Instead, this comes from Newsletter sponsorships such as this and other one-off sources like ad-sense.
Product Analysis( 6%) - There are so many new products in the data space and last year I had several reach out and ask me to take a look at their overall solution.
Coaching( ~.5%) - I limit how much coaching I do. But I took on a few people to help them adjust to their new roles in 2022.
2. Popular Articles
I only recently committed to writing 3-4 articles a month in 2022. It was a bit of a bumpy start and some months I only got two articles out. But I have found a stride. Here are some of your favorites.
A Zero ETL Future - AWS and Snowflake have both made moves to remove the amount of data movement from source to data warehouse. Is it possible to completely remove the ETL?
How To Start Your Next Data Engineering Project - Following step-by-step guides are a good place to start with data engineering projects. But eventually, you’ll want to come up with your project ideas. Here is where you can start.
Let's Move Fast And Get Rid Of Data Engineers - Data engineers, in all their current and past forms, have often been the bottleneck in companies' process to getting value from data. Why don’t we just get rid of them?
Different Types Of "Data Engineering" Teams - Data engineering, like data science, has continued to specialize over the past decade. Where it was once expected that a data engineer would be a jack of all trades, I am starting to see specialized roles arise. Here is their breakdown.
The Next Generation Of All-In-One Data Stacks - Several tools outside of the point solution world of the modern data stack started to gain a lot of popularity in 2022. They were landing deals with enterprise customers and people were noticing. So I wanted to review several of these all-in-one data stacks and see what they had to offer.
My favorite article from 2022:
Should You Use Apache Airflow? - One of the types of projects I enjoy is gathering a large swath of other smart people’s opinions. This is exactly what I did for this article. Apache Airflow has a solid footing in the data engineering world. Yes, there are a lot of new contenders like Dagster, Prefect, and Mage. But overall, Airflow remains a popular option. However, if you use Airflow long enough, you will run into plenty of issues. That’s what I wanted to understand. Why do data engineers love/hate Airflow?
I wanted to give my loyal subscribers a gift for the new year. Until January 1st I am offering 15% off so you can read my past articles such as A Zero ETL Future.
3. Most Popular Videos
Seattle Data Guy In 2023
First off. Thank you! Thank you so much for being a subscriber and supporting me on my various platforms. I appreciate everything and I aim to continue to provide as much value as I can.
Now let’s look at what you can expect in 2023 from this newsletter as well as all the various other platforms.
Community - In 2023, I plan to further invest in our efforts to grow the data community. This means more Data Happy Hours, where speakers and practitioners can network and share their experiences.
If you haven’t signed up for these events, feel free to sign up here.
Collaborations With Other Authors - I enjoyed my experience collaborating with The Pragmatic Engineer and I would like to do more of those in the future. I am talking with experts in fields such as machine learning and data platforms to have them share their knowledge here. The data world is vast and there are so many different types of data engineers, analysts, and scientists that I believe would benefit the community.
Live All-Day Conference - For those who haven’t been able to make it to
the live events our team has put together, you can attend our all-day conference. We have lined up several great speakers including Chad Sanderson, Mei Tao, Sireesha Pulipati, and more.If you’re interested in attending, then please sign up here.
State Of Data 2022 - Our team has been surveying data leaders and practitioners from companies of all sizes and we are quickly coming up against our deadline. We need your help to reach our goal so we can share the results with y’all! Please fill out our data survey so we can better understand the state of data infra going.
Thanks again for a wonderful year! See y’all in 2023.
From someone who is just starting their data engineer journey, just want to say that this newsletter is an amazing resource and I hope you keep this up!
Nice recap and all around great newsletter, Ben!