Burned Out - Take A Break, Don't Quit

Community Update #11

Jun 23, 2021

The great resignation.

The new cultural phenomena that are a combination of built-up job transitioners, freedom seekers and remote work purist is real.

I am talking to companies around the US that are watching people leave their current role for a whole multitude of reasons.

Many were too afraid to look for a new job when times were uncertain. However, now that the ground is beginning to settle, they are looking for other roles.

Others have found new passions and side-hustles that they believe holds the key to their financial freedom.

And of course, some want to keep working remotely.

Now, typically, I focus on technology in my articles.

But in many ways, I view the giant movement of people from the company to company as well as the changing perception as a factor that will impact technology teams.

Teams will be disrupted.

Workflows and projects halted.

We will all likely need to spend time regaining our bearings as people resituate themselves.

So it’s important if you’re a manager or director to be prepared for this shift. Especially if you’re managing a software or engineering team.

These teams have so many specialist as well as so much tribal knowledge about code-bases on feature requirements.

Much of which could be lost if your best engineer or data scientist leaves.

So you will need to be ready in the coming months if you’re an engineering manager.

Here is how I recommend you prepare for the tidal wave that is already here for many companies.

Onboarding Documentation And Processes

Onboarding documentation is always hit or miss.

Some companies I have gone to have amazing onboarding documentation that make it so easy to jump into projects quickly.

Other companies I have worked with had me waiting weeks to even get close to starting real work.

If you’re having high turnover, then you don’t have time to be the latter. You need to have your onboarding process and documentation cinched up.

This means you need to have a clear checklist of what your team members will need to be given access to from day one. Instead of them having to constantly run into roadblocks as they figure out they can’t access your SQL server or BigQuery instance and now they have to put a ticket into some IT system.

Or at the very least, just have a clear checklist so they can put in one ticket from the get-go. No one-offs.

Cross-Training

Another way to mitigate the risks of team members leaving is ensuring there is some cross-training going on.

Make sure that there isn’t just one person managing a business-critical process.

Make sure there isn’t one person that is a cornerstone in a business-critical project.

This doesn’t mean you need to have two people doing one job. It just means occasionally have some team members work with others on the same work for a few days, so they are aware of what is going on.

Especially when it comes to developers, data engineers, and data scientists. If you lose one of them, tons of tribal knowledge is often lost.

Yes, documentation can help. But, having someone that already has hands-on experience can save you so much stress.

Project Re-Prioritization

Companies are seeing an increase in their overall attrition rates. Not to mention in a recent survey upwards of 40% of employees are thinking about quitting.

Meaning that projects that are currently running smoothly, are at risk.

This means if you are managing a large portfolio of projects, you should be ready, or already starting to prioritize which projects are business-critical. Because if there is a shift in your engineering staff, there will also be a delay in all of your projects. Unless you are already prepared.

Then you can at least communicate with management what projects could be at risk in case you need to shift your teams around to ensure your business-critical projects get finished.

Be Calm And Carry On

Most importantly, if you’re a manager or just an employee wondering if you should quit and find greener pastures.

Take a moment to breathe.

Yes, there will be some shakeup and the ground will shift again.

But after a while, hopefully, the workplace finds its new medium.

Good luck to all your data science and engineering managers.

Ask A Data Consultant - Office Hours

Every newsletter I open up a day or two with a few slots for open office hours where my readers can sign up and you can ask me questions. I got to answer a lot of great questions so far and hopefully, they helped provide a lot of insights for those who signed up.

Next Open Office Hours

Thanks To The SDG Community

I started writing this weekly update more seriously about 6-7 weeks ago. Since then I have gained hundreds of new subscribers as well as 5 supporters!

And all I can say is, Thank You!

You guys are keeping me motivated.

Every read, comment, like and financial subscription is amazing and I really appreciate all you.

If you want to help support this community consider clicking the link below.

Managing A Billion Dollar Analytics Team - Veronica Zhai

This week I will be interviewing Veronica Zhai and talking about her journey from working as a Trader at JP Morgan to her dive into being a leader in the data analytics space and eventually finding herself as a Principal Product Manager at Fivetran.

She’s a data enthusiast, writer/dancer/choreographer, and lover of psychoanalysis. In addition, she enjoys designing an elegant system (whether it being information platform or social network) to achieve higher level of consciousness and affect changes.

Tomorrow we will be discussing her journey as well as asking her how she manages an analytical team successfully and how she is working to scale it with the needs of Fivetran’s business.

Set A Reminder For Tomorrow Thursday 24th 12 PM PT!

Articles Worth Reading

There are 20,000 new articles posted on Medium daily and that’s just Medium! I have spent a lot of time sifting through some of these articles as well as TechCrunch and companies tech blog and wanted to share some of my favorites!

Identifying Financial Fraud With Geospatial Clustering

For most financial service institutions (FSI), fraud prevention often implies a complex ecosystem made of various components –- a mixture of traditional rules-based controls and artificial intelligence (AI) and a patchwork of on-premises systems, proprietary frameworks and open source cloud technologies. Combined with strict regulatory requirements (such as model explainability), high governance frameworks, low latency and high availability (sub second response time for card transactions), these systems are costly to operate, hard to maintain and even harder to adapt to customers changing behaviors and fraudsters alike. Similar to risk management, a modern fraud prevention strategy must be agile at its core and combines a collaborative data-centered operating model with an established delivery strategy of code, data and machine learning (ML) such as DataOps, DevOps and MLOps. In a previous solution accelerator, we addressed the problem of combining rules with AI in a common orchestration framework powered by MLflow.

Rivery.io – What Is It and How It Can Help You Develop Your Data Pipelines

There are plenty of clichés about data and its likeness to oil. But it’s far from easy to get value from data.

Companies are creating more data than ever before. At our current pace, 2.5 quintillion bytes of data are created every day. Companies went from pulling data solely from their CRM and ERP systems to pulling data from dozens or hundreds of data sources simultaneously.

Creating data pipelines for all of these data sources is expensive and difficult to maintain. Data engineers are often stretched thin, between constantly updating API connections, to building new pipelines for every data source. Without access to this data, analysts and data scientists cannot provide key insights for organizational stakeholders.

But, in the last decade, software solutions have emerged that can pull data from many different data sources without requiring companies to build infrastructure manually. These managed, low-code platforms increase efficiency and time-to-insights, while freeing engineers to work on more important projects.

Rivery counts itself among these platforms. I recently had the chance to test out Rivery for myself. Here’s what I found.

The Evolution of Data Science Workbench

In October 2017, we published an article introducing Data Science Workbench (DSW), our custom, all-in-one toolbox for data science, complex geospatial analytics, and exploratory machine learning. It centralizes everything required to perform data preparation, ad-hoc analyses, model prototyping, workflow scheduling, dashboarding, and collaboration in a single-pane, web-based graphical user interface.

In this article, we reflect on the evolution of DSW over the last 3 years. We review our journey by looking at how DSW’s usage has evolved to include and supercharge the workflows of more than just data scientists, dive into how the choices we made when designing the platform have helped us scale, offer an in-depth look at our current approach and future goals towards democratizing data science, and also share some lessons that we learned along the way.

A Fun Video You Might Have Missed

I really have been enjoying creating videos on Youtube for the last few weeks.

One video that has been doing well has been my review of Google’s Data Engineering Certificate. Everything from my flowery shirt to the occasional joke, this video was a lot more fun to make.

End Of Day 11

It’s been a busy few weeks.

From a consulting stand-point, I have put out several proposals and onboarded new clients.

In addition, I passed 5k subscribers on Youtube and my videos are starting to gain traction.

Thanks for all your support and let me know if you have any questions about big data, data engineering, data science or technology in general, then send them my way.

Good luck with the rest of your week!

Also, if you enjoyed this update, consider reading one of my personal favorites below.

Greylock VC and 5 Data Analytics Companies It Invests In And Looking At The Path To Senior Engineer

SeattleDataGuy’s Newsletter

Discussion about this post