How to build a career you enjoy as an engineer. What I wish someone told me when I started

“We all have a finite amount of time to live, and within that mortal countdown, we devote some fraction towards our work. Even for the most career-focused, your life will be filled with many things beyond work.” - Will Larson

As Will Larson put it, we all have finite time, and yet as we become more experienced the expectation on our time becomes that we have higher “impact.”

We get fancier titles, have more meetings, and sometimes get further separated from the actual work. If you don’t pay attention to what is going on around you, you may find yourself in a role you don’t like or perhaps at a level below what you’d hope.

It can be easy to sleepwalk in your career or just expect your manager to help you pick the projects that will take you from mid-level to senior or staff.

That’s not how it usually works.

There is a need for you to take control and avoid doing work that is just provided because it's easy to find and complete. You have to eventually start looking for projects that increase in scope, complexity and likely risk.

As one of my managers used to say:

If you want to get a promotion, you don’t need to complete 1,000 tasks; you need to figure out how to eliminate the need for the 1,000 tasks.

Below, we’ll discuss how to ensure you take the proper so you’re growing and building the career you want.

Let’s start by talking about taking control. 

Take Control of What You Can

Irina Stanescu

posted a while back about her experience trying to get a promotion at Google. It’s not an unheard-of narrative, especially if you’re still early on in your career or have worked for companies that just want you to do as you're told.

I’ll let you read through her post before we get back to talking about the importance of taking control of your career.

Now not every company gives you the freedom to take control, but even when I worked at a healthcare provider, I constantly advocated for myself and pushed to try to at least have 10-20% of time allocated to projects I wanted to take on.

Eventually, this made it possible to work on projects that made it easier for me to get a job as a data engineer at the next company.

This was also the time I met a consultant who pointed out this fact.

That if you’re working on a project no one knows exists, it’s not a good project to be working on.

I do think it’s a little more nuanced than that. There sadly is work, such as glue work, that needs to happen and sometimes doesn’t get the credit it should. However, the notion is still important to call out.

You have to work on projects that are meaningful to the company(that people know you’re working on) and help you grow, whether that means it grows you because the scope of the project is bigger or the complexity is greater. 

Guess what?

Those projects may not fall into your lap; in fact, likely someone else is already starting the project they know is impactful or meaningful for themselves at your company. And as pointed out earlier, if you don’t take some control of what you work on, you’ll just end up doing the work that is asked that might not grow you into the position you’re hoping for.

Like, work that could be called snacking.

Avoid Snacking

Source

Hunter Walk recommends that folks avoid "snacking" when they prioritize work. If you're in a well-run organization, at some point, you're going to run out of things that are both high-impact and easy. This leaves you with a choice between shifting right to hard and high-impact or shifting down to easy and low-impact. The latter choice--easy and low-impact--is what Walk refers to as snacking. - Work On What Matters

Or as

Jordan Cutler

recently put it:

• Snack = Low impact, low effort
• Quick win = High impact, low effort
• Impactful project = High impact, high effort

Early on in your career, I think it can make sense to take on projects that might be lower impact and easier to take on. After all, you’re building your fundamentals so you don’t need to try to build a fully distributed MPP novel data management system.

But eventually, it’s important to start seeking larger projects where you can have a big impact. Early on, this likely means you will be the one planning, building, and delivering the work. It’ll likely be stressful and require you to think through several layers of the business.

Eventually, you’ll even start having others work on these larger projects as you get better and better at taking on projects with more extensive scope and impact.

Here is where individuals might break off; some might delve into being more of a consultant guiding 4-5 of these projects at once, or maybe you’ll start pushing to drive the overall strategy for your data teams. Still, others will continue to act as high-level software engineers, building, deploying, and solving some of the most challenging problems at a company.

Make Sure You Work On What Matters

One of the questions people often ask in many of my videos on YouTube is, “Should I become a data engineer, data scientist, etc?” Sometimes they are explicit in terms of referencing salary; perhaps others are just asking in general.

At the end of the day, most of these roles pay well, and your goal should be to focus on both roles and industries you enjoy as well as focusing on solving problems that impact the company heavily.

As Jordan Cutler put it in Becoming a go-to person gets you promoted, there are at least three dimensions you should focus on:

• Team or company impact

• What you’re good at

• What you’re passionate about

Your goal should be to try to find a balance between work you find meaningful, you’re passionate about, good at, and has a large impact on your company.

Now part of of getting to a point where you are working in a role and on projects that meet this set of criteria is that you need to go through a few years of trying things out. So don’t feel like you have to find your perfect balance day one. But always be readjust for it(as well as trying out new things early on).

Who knows, you might get so good at a skill, maybe even a skill you don’t view as heavily technical, that you decide to consult or create a company that makes far more than you would as an engineer.

Double Down - And Don’t Get Distracted

To take another point that I really liked from Jordan’s article:

“If you can get all 3, double down. Invest more in that area, maintain your expertise and expand your scope. Learn more about that area and keep making an impact.” - Becoming a go-to person gets you promoted

I think the double-down point is really important.

Otherwise, it can be tempting to see other individuals, what they are doing, and most importantly, the output or outcome and think that’s what you want to do. But generally, that is only a few percent of the actual work, and if you decide to switch to a new role because you think it pays better or has a more exciting outcome, you might miss the fact that the majority of the work isn’t something you like doing.

I’d say this is also true for people who start companies and brands. You’ve got to stick to your mission, your why.

So part of becoming an expert is finding work that you truly enjoy doing and is needed by companies, as well as sticking with it and getting really good at it. Become so good that if someone needs a specific problem solved, you become the go-to person.

Now, this is tech; you’ll have to learn several skills over your lifetime. That being said, having deep knowledge in specific areas along with general broad skills puts you in an excellent position to drive a lot of value. Especially if you can start to bridge the gap between tech and business.

You might even become the person or consultant that people go to solve a very specific industry problem if you can apply your technical skills to a domain(healthcare, insurance, financial services, sales, etc).

Once that happens, you likely will start getting involved in more impactful projects as long as people know you exist.

But we can talk about building a brand and selling your expertise in a different newsletter.

What Is Next?

Working on what matters, not just for your company but yourself, ensures you’re growing skills while also getting recognized for the work you’re putting out.

Now not every project will 100% line up with what you love doing, but the more you learn how to find the right projects, deliver them, and get recognized for them, the more you’ll grow inside the company and gain flexibility to take on more projects you enjoy.

In turn, hopefully, this starts to further align you more with the work you want to do.

Thanks for reading.

---

Video Of The Week: Using AWS Lambda As A Data Engineering - Automating An API Extract With AWS Lambda And EventBridge

---

Join My Data Engineering And Data Science Discord

If you’re looking to talk more about data engineering, data science, breaking into your first job, and finding other like minded data specialists. Then you should join the Seattle Data Guy discord! We are close to passing 5500 members!

---

Articles Worth Reading

There are 20,000 new articles posted on Medium daily and that’s just Medium! I have spent a lot of time sifting through some of these articles as well as TechCrunch and companies tech blog and wanted to share some of my favorites!

---

Data Processing in 21st Century

by

Junaid Effendi

Lets talk about the open source data processing technology from the 21st century, covering distributed frameworks to single node libraries, mature and recent development.

Before these technologies, the common technique to process data was to glue together custom solutions, e.g. scripting and piping was one approach as mentioned in the Designing Data-Intensive Applications.

Now, we have lot of tools on our plate, lets discuss briefly each tool, what they solved, and how they evolved. Some of the tools were part of Hadoop Ecosystem, some are direct replacement of another, while some are complimentary to each other.

Read More Here

Unveiling Interactive Workflows with Prefect - Sponsored

Have you ever wanted to pause a workflow to get human input? With Prefect’s new interactive workflows, you can describe the data your flow needs, and Prefect will render a form in its UI for users to enter the data. But this capability goes beyond human input—interactive workflows can communicate with each other, too.

It’s now easier than ever to write flows that pause for human-in-the-loop input, manage continuous data streams, and more. Curious about how this works in practice? Read on for a detailed example that explains how to add interactivity to your flows.

💡 Want to jump right into code? Check out our example repository for everything used in this blog post or read our guide to creating interactive workflows.

Read More Here

---

End Of Day 119

Thanks for checking out our community. We put out 3-4 Newsletters a week discussing data, tech, and start-ups.