3 Comments
Nov 17, 2021Liked by SeattleDataGuy

I made a straw poll in the locally optimistic airflow channel about these options, results were

A - How is Docker deployed (n =24)

Using Managed Airflow Services 46%

Using Production Docker Images 29%

Using 3rd-party images, charts, deployments 17%

Using Official Airflow Helm Chart 4%

Using PyPI 4%

B - Which managed services used (n=17)

GCP Cloud Composer 47%

Astronomer 35%

AWS MWAA 18%

I was surprised so many are not using managed services!

Expand full comment
author

Wow! Thanks for doing the quick research. I think 46% is higher than I thought based on my project experience.

Expand full comment

We're running Airflow inside our Kubernetes cluster, and I tell you, it's not so hard to manage and scale. So we never made a decision to move to Composer.

Expand full comment