Artwork

محتوای ارائه شده توسط The Data Flowcast. تمام محتوای پادکست شامل قسمت‌ها، گرافیک‌ها و توضیحات پادکست مستقیماً توسط The Data Flowcast یا شریک پلتفرم پادکست آن‌ها آپلود و ارائه می‌شوند. اگر فکر می‌کنید شخصی بدون اجازه شما از اثر دارای حق نسخه‌برداری شما استفاده می‌کند، می‌توانید روندی که در اینجا شرح داده شده است را دنبال کنید.https://fa.player.fm/legal
Player FM - برنامه پادکست
با برنامه Player FM !

Transforming the Airflow UI for Cloudera’s Users with Shubham Raj

22:28
 
اشتراک گذاری
 

Manage episode 493621102 series 2948506
محتوای ارائه شده توسط The Data Flowcast. تمام محتوای پادکست شامل قسمت‌ها، گرافیک‌ها و توضیحات پادکست مستقیماً توسط The Data Flowcast یا شریک پلتفرم پادکست آن‌ها آپلود و ارائه می‌شوند. اگر فکر می‌کنید شخصی بدون اجازه شما از اثر دارای حق نسخه‌برداری شما استفاده می‌کند، می‌توانید روندی که در اینجا شرح داده شده است را دنبال کنید.https://fa.player.fm/legal

Contributing to open-source projects can be daunting, but it can also unlock unexpected innovation. This episode showcases how one engineer’s journey with Apache Airflow led to impactful UI enhancements and infrastructure solutions at scale. Shubham Raj, Software Engineer II at Cloudera, shares how his team built a drag-and-drop DAG editor for non-coders, contributions which helped shape the Airflow 3.0 Ul and introduced features like external XCom control and bulk APls.

Key Takeaways:

(02:30) Day-to-day responsibilities building platforms that simplify orchestration.

(05:27) Factors that make onboarding into large open-source projects accessible.

(07:35) The value of improved user interfaces for task state visibility and control.

(09:49) Enabling faster debugging by exposing internal data through APIs.

(13:00) Balancing frontend design goals with backend functionality.

(14:19) Creating workflow editors that lower the barrier to entry.

(16:54) Supporting a variety of task types within a visual DAG builder.

(19:32) Common infrastructure challenges faced by orchestration users.

(20:37) Addressing dependency management across distributed environments.

Resources Mentioned:

Shubham Raj

https://www.linkedin.com/in/shubhamrajofficial/

Cloudera | LinkedIn

https://www.linkedin.com/company/cloudera/

Cloudera | Website

https://www.cloudera.com/

Apache Airflow

https://airflow.apache.org/

2023 Airflow Summit

https://airflowsummit.org/

https://www.astronomer.io/events/roadshow/london/

https://www.astronomer.io/events/roadshow/new-york/

https://www.astronomer.io/events/roadshow/sydney/

https://www.astronomer.io/events/roadshow/san-francisco/

https://www.astronomer.io/events/roadshow/chicago/

Thanks for listening to “The Data Flowcast: Mastering Apache Airflow® for Data Engineering and AI.” If you enjoyed this episode, please leave a 5-star review to help get the word out about the show. And be sure to subscribe so you never miss any of the insightful conversations.

#AI #Automation #Airflow #MachineLearning

  continue reading

63 قسمت

Artwork
iconاشتراک گذاری
 
Manage episode 493621102 series 2948506
محتوای ارائه شده توسط The Data Flowcast. تمام محتوای پادکست شامل قسمت‌ها، گرافیک‌ها و توضیحات پادکست مستقیماً توسط The Data Flowcast یا شریک پلتفرم پادکست آن‌ها آپلود و ارائه می‌شوند. اگر فکر می‌کنید شخصی بدون اجازه شما از اثر دارای حق نسخه‌برداری شما استفاده می‌کند، می‌توانید روندی که در اینجا شرح داده شده است را دنبال کنید.https://fa.player.fm/legal

Contributing to open-source projects can be daunting, but it can also unlock unexpected innovation. This episode showcases how one engineer’s journey with Apache Airflow led to impactful UI enhancements and infrastructure solutions at scale. Shubham Raj, Software Engineer II at Cloudera, shares how his team built a drag-and-drop DAG editor for non-coders, contributions which helped shape the Airflow 3.0 Ul and introduced features like external XCom control and bulk APls.

Key Takeaways:

(02:30) Day-to-day responsibilities building platforms that simplify orchestration.

(05:27) Factors that make onboarding into large open-source projects accessible.

(07:35) The value of improved user interfaces for task state visibility and control.

(09:49) Enabling faster debugging by exposing internal data through APIs.

(13:00) Balancing frontend design goals with backend functionality.

(14:19) Creating workflow editors that lower the barrier to entry.

(16:54) Supporting a variety of task types within a visual DAG builder.

(19:32) Common infrastructure challenges faced by orchestration users.

(20:37) Addressing dependency management across distributed environments.

Resources Mentioned:

Shubham Raj

https://www.linkedin.com/in/shubhamrajofficial/

Cloudera | LinkedIn

https://www.linkedin.com/company/cloudera/

Cloudera | Website

https://www.cloudera.com/

Apache Airflow

https://airflow.apache.org/

2023 Airflow Summit

https://airflowsummit.org/

https://www.astronomer.io/events/roadshow/london/

https://www.astronomer.io/events/roadshow/new-york/

https://www.astronomer.io/events/roadshow/sydney/

https://www.astronomer.io/events/roadshow/san-francisco/

https://www.astronomer.io/events/roadshow/chicago/

Thanks for listening to “The Data Flowcast: Mastering Apache Airflow® for Data Engineering and AI.” If you enjoyed this episode, please leave a 5-star review to help get the word out about the show. And be sure to subscribe so you never miss any of the insightful conversations.

#AI #Automation #Airflow #MachineLearning

  continue reading

63 قسمت

Tất cả các tập

×
 
Loading …

به Player FM خوش آمدید!

Player FM در سراسر وب را برای یافتن پادکست های با کیفیت اسکن می کند تا همین الان لذت ببرید. این بهترین برنامه ی پادکست است که در اندروید، آیفون و وب کار می کند. ثبت نام کنید تا اشتراک های شما در بین دستگاه های مختلف همگام سازی شود.

 

راهنمای مرجع سریع

در حین کاوش به این نمایش گوش دهید
پخش