Posted By
A R
More Posts
Ratings
0 Experts Rating
0 All Members
Find More
Resource Type
Engineering’s Big Ideas
Parent Resource373
Find and Add Resources
Posted September 16, 2016
0
Read More
Ratings Summary
{{reviewGroup.totalForGroup}}{{reviewGroup.groupName}}
{{valuesDictionary[groupValues.rating]}} ({{groupValues.noOfRatings}}/{{reviewGroup.totalForGroup}})
Leave Edit Mode
How to Run Apache Airflow on Kubernetes at Scale
https://doordash.engineering/2021/09/28/how-to-run-apache-airflow-on-kubernetes-at-scale/

PostedFriday, November 5, 2021 at 4:34 PM

Updated11/05/2021

As an orchestration engine, Apache Airflow let us quickly build pipelines in our data infrastructure. However, as our business grew to 2 billion orders delivered, scalability became an issue. Our solution came from a new Airflow version which let us pair it with Kubenetes, ensuring that our data infrastructure could keep up with our business growth.

As an orchestration engine, Apache Airflow let us quickly build pipelines in our data infrastructure. However, as our business grew to 2 billion orders delivered, scalability became an issue. Our solution came from a new Airflow version which let us pair it with Kubenetes, ensuring that our data infrastructure could keep up with our business growth.

Read More
Your Review
Your overall rating for this resource
0/ 125 chars