The Road to Airflow 2.0

The Data Flowcast: Mastering Airflow for Data Engineering & AI

This week, we linked up with Airflow release manager, core committer, and Astronomer platform engineer Ash Berlin-Taylor to discuss the Airflow 2.0 roadmap [1]. There is some great stuff in the works around performance, autoscaling, and usability that we're excited about. In this episode, Ash lends his thoughts on the design, implementation, and value-add around all of the upcoming features, including: - The Knative Executor - A modern and real-time UI - A production-grade API - Improved scheduler and webserver performance - An official production Docker image for Airflow We hope you enjoy! Please email [email protected] if you have thoughts on topics you'd like to see covered in future episodes. Separately, some good folks from the Airflow community are running a user survey that will help collect some useful information around the Airflow UX. If you have five minutes to spare, filling out the following form will help the core Airflow committers to shape the project roadmap: https://forms.gle/XAzR1pQBZiftvPQM7 [1] https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+2.0