What Does It Take To Get Promoted As An Engineer?
Expectations And Axes
60% of the respondents to my 2022 State Of Data survey aren’t confident they know what is required to get a promotion. This made me re-examine what I know about leveling and skill expectations.
Seriously, what is expected at different levels?
I have found that tech companies will break down what is expected pretty clearly(even if it’s not always followed to a T ). Different levels are based on the axes they find valuable.
This could be based on axes such as the ones stated by Omar Halabieh (Head of Technology at Amazon Payment Services) which were:
Independence
Complexity
Scope
Ambiguity
Influence
Impact
Facebook also had a similar set of axes including impact, direction, scope, people, and engineering excellence. I would say tech companies are in the minority when it comes to having clear axes and expectations (based on discussions I have had). Most people I have talked to find that their promotions in standard enterprise companies are somewhat attached to project deliverables, years of experience, and the hands of fate.
That aside, for this article, I will be focusing on breaking down the expectations at various levels for scope, influence, impact, complexity, and independence.
Juniors and Interns
Scope - Whether you’re a junior engineer or intern, your general focus is on doing clearly defined tasks. Your manager or a senior engineer may be assigning said tasks so you can focus more on doing and learning how to be a better engineer and not focusing on the big picture. You should be learning about big-picture concepts like team dynamics, business impact, and, unavoidably, company politics. But at this point, it’s about building strong technical basics.
The wax-on and wax-off of engineering: speeding through being a junior to try to focus on the bigger picture could limit your impact as you continue to move up in scope. Because like the image suggests above, everything builds on each other.
Independence - Another trend I have noticed with less experienced engineers is knowing when to ask for help. Generally, less experienced engineers tend to either ask questions too often or not frequently enough. Too often, senior engineers might feel the junior is not self-reliant, and now frequent enough can slow down projects and tasks.
Influence - At this level, you’re not expected to influence anyone. Your focus is learning, gleaning, going through code bases, and becoming solid in your technical skills.
Complexity - In terms of complexity, most of the work you will be performing is well-defined and standard programming tasks. This could be building an API end-point or a clearly defined data pipeline.
Impact - When first starting out, your work is likely not driving much impact on its own. Perhaps you are helping a senior engineer complete their larger project or maybe you’re just doing one-off tasks. Early on, the focus is more on execution and less on influence.
Mid-Level
Scope - By mid-level, you should be capable of taking several tasks and completing them in succession for a larger goal. Often this is referred to as a small project, or possibly a feature as Omar Halabieh pointed out. An example of this for a data engineer could be working with an application and data analytics team to build the tables and pipelines required to support a new product feature. This work would involve the engineer helping define what information should be tracked by the application team while designing tables the data science team can rely on.
Independence - At this point, a manager is likely assigning you somewhat ambiguous projects. Meaning you will need to spend some time talking to other XFNs (cross-functional partners) to understand the requirements and end goal.
Influence - You should start becoming comfortable voicing your opinion in meetings. Even if you’re occasionally wrong, it’s important that as a mid-level you are providing perspective, as well as backing for said perspective when it comes to design decisions and which projects the team should take on. You’re also likely starting to mentor new engineers.
Complexity - Mid-level engineers are learning how to take on projects by themselves. In itself, projects bring new complexity both in terms of dealing with dependencies as well as the number of problems you are having to solve.
Impact - Once you have been working for 3-5 years you should have some idea of how your work is impacting the business, but it’s likely sophomoric. That is to say, you might think you know, but you’re still learning.
Senior
Scope - Senior engineers are starting to take on projects that expand across multiple teams. This might mean they need to talk to the product team to make sure it aligns with a key business metric. In addition, the senior engineer is likely the one to find the project instead of being assigned.
Independence - Besides dealing with ambiguity by yourself, senior engineers start finding projects that are worth digging into. This shows signs that you are becoming more aware of the big picture and thinking about how your work could tie to the bottom line.
Influence - At this point your opinion holds a decent amount of weight. Even at companies where there isn’t explicit leveling, a senior engineer still generally has the ability to sway opinions without a Sr. by their title because they are comfortable sharing opinions and backing them up with clear articulate arguments. In addition, at this point a senior engineer is also likely mentoring several engineers underneath them.
Complexity - For senior engineers an example of complexity might be a project that involves simplifying a large code base so it is easier to maintain. This would involve a heavy amount of risk as the engineer worked to figure out all the dependencies and requires code simplification which requires solid design and problem solving.
Impact - The best tip I got about getting noticed and having large impact at Facebook came from my manager. He told me that impact is not closing 1000 tickets a quarter, it’s about finding a way to remove those 1000 tickets from existing in the future. That’s the kind of impact you want to be thinking of once you become more senior.
Quick Pause: We need your help! Our team is putting together a survey to better understand the state of data infrastructure. We will be sharing our results in this newsletter. If you work in data(whether an analyst or VP of engineering) we would love to hear from you!
Staff
Scope - Staff engineers are focused beyond driving value in a single product feature. They will help define the business alignment for organizations and products. A quote from An Engineer (that is their online name) about their experience of being a Staff Engineer covers this well.
In 6 months we had a working version and got blessing from leadership to invest in this product. We started recruiting team heavily the team grew to 12 engineers by end of the year. - An Engineer
They were making bets on the product level. Meaning they had to bring in several other engineers and get leadership buy-in.
Independence - At this point, staff engineers are defining initiatives or making large bets on products by themselves.
Influence - Staff engineers at many tech companies are considered to be the equivalent of a manger. Meaning they are often influencing multiple software engineers. Even if they are not directly responsible for the growth of said software engineers, they are often in charge of the result of their work.
Complexity - Complexity for staff engineers entails dealing with problems such as scaling your infrastructure to manage 10x more traffic than it is currently managing while being cost-efficient. For data engineers, it might involve migrating to a new data storage analytics system or data model to handle the influx of data. These projects both have a large impact as well as can prove challenging due to all of the various moving pieces and cost restraints.
Impact - A great example of the level of impact a staff engineer may have comes from Zach Wilson who referenced his role in helping roll out more transparent pricing for Airbnb. This was a very public initiative. There has been plenty of public discourse about how people wanted Airbnb to be more transparent and the CEO publicly said they were dealing with it. If the CEO is aware of your work, it’s impactful.
Some Tips To Get That Promotion
Knowing what skills and competencies you should have to get promotions is only part of the battle when it comes to getting promoted. A recent post from Carly Taylor gave some tips in terms of how to go beyond just bringing the value of your skills to get promoted.
Many engineers and data scientists, often feel as if good work should lead to promotions just for the work itself. Sadly, that’s not how many companies operate. Even on Facebook, there was a major aspect of showing value. When I worked at Facebook, there were over 1000 other data engineers, and you were rated against a cohort of said data engineers. Meaning you had to stand out.
Even when this isn’t the case, let’s say at a more standard corporate company with less clear axes and fewer engineers. You still need to justify your promotion.
There are plenty of tips on standing out. However, for this newsletter let’s look through some of Carly’s tips, and I would love it if you shared your own in the comments.
Get That Promotion!
What gets you promotions is different for every company. But, I do believe companies should attempt to provide a clear set of expectations. If not by axes, then through clear sets of expectations in terms of what projects should be delivered or what work should be completed.
Of course, I have still had discussions with disappointed engineers at companies like Amazon and Facebook because they felt like they met the requirements for a promotion and didn’t receive one.
I’d love to hear your thoughts though, what do you believe are the different expectations for each level of engineer?
Build SQL Pipelines. Not Endless DAGs!
With Upsolver SQLake, you build a pipeline for data in motion simply by writing a SQL query defining your transformation.
Streaming and batch unified in a single platform
No Airflow - orchestration inferred from the data
$99 / TB of data ingested | transformations free
Join My Data Engineering And Data Science Discord
Recently my Youtube channel went from 1.8k to 48k and my email newsletter has grown from 2k to well over 28k.
Hopefully we can see even more growth this year. But, until then, I have finally put together a discord server. Currently, this is mostly a soft opening.
I want to see what people end up using this server for. Based on how it is used will in turn play a role in what channels, categories and support are created in the future.
Articles Worth Reading
There are 20,000 new articles posted on Medium daily and that’s just Medium! I have spent a lot of time sifting through some of these articles as well as TechCrunch and companies tech blog and wanted to share some of my favorites!
Speed Up Presto at Uber with Alluxio Local Cache
At Uber, data informs every decision. Presto is one of the very core engines that powers all sorts of data analytics at Uber. For example, the operations team makes heavy use of Presto for services such as dashboarding; the Uber Eats and marketing teams rely on the results of these queries to make decisions on prices. In addition, Presto is also used in Uber’s compliance department, growth marketing department, and ad-hoc data analytics.
Using the Open Source Insights dataset to analyze the security and compliance of your dependencies
Security teams generally use one of two common approaches to evaluate dependencies for vulnerabilities. With the first approach, security teams begin by researching every single dependency and its security posture at the beginning of the application development process. When a project is underway, security teams must continue to monitor and analyze any changes to the security posture of each dependency over time. This approach has drawbacks: it is very tedious and difficult for security teams to continually check each dependency. This is why security teams frequently choose the second approach, which is to rely on commercial software that evaluates security posture. Unfortunately this approach limits the control that security teams have over how to integrate dependency evaluations into the team's workflows.
End Of Day 60
Thanks for checking out our community. We put out 3-4 Newsletters a week discussing data, tech, and start-ups.
If you want to learn more, then sign up today. Also you can check out some of my recent newsletters below!