התחל במצב לא מקוון עם האפליקציה Player FM !
Orchestrating Analytics and AI Workflows at Telia with Arjun Anandkumar
Manage episode 463948888 series 2948506
The future of data engineering lies in seamless orchestration and automation. In this episode, Arjun Anandkumar, Data Engineer at Telia, shares how his team uses Airflow to drive analytics and AI workflows. He highlights the challenges of scaling data platforms and how adopting best practices can simplify complex processes for teams across the organization. Arjun also discusses the transformative role of tools like Cosmos and Terraform in enhancing efficiency and collaboration.
Key Takeaways:
(02:16) Telia operates across the Nordics and Baltics, focusing on telecom and energy services.
(03:45) Airflow runs dbt models seamlessly with Cosmos on AWS MWAA.
(05:47) Cosmos improves visibility and orchestration in Airflow.
(07:00) Medallion Architecture organizes data into bronze, silver and gold layers.
(08:34) Task group challenges highlight the need for adaptable workflows.
(15:04) Scaling managed services requires trial, error and tailored tweaks.
(19:46) Terraform scales infrastructure, while YAML templates manage DAGs efficiently.
(20:00) Templated DAGs and robust testing enhance platform management.
(24:15) Open-source resources drive innovation in Airflow practices.
Resources Mentioned:
https://www.linkedin.com/in/arjunanand1/?originalSubdomain=dk
Telia -
https://www.linkedin.com/company/teliacompany/
https://airflow.apache.org/
https://www.astronomer.io/cosmos/
https://www.terraform.io/
Medallion Architecture by Databricks -
https://www.databricks.com/glossary/medallion-architecture
Thanks for listening to “The Data Flowcast: Mastering Airflow for Data Engineering & AI.” If you enjoyed this episode, please leave a 5-star review to help get the word out about the show. And be sure to subscribe so you never miss any of the insightful conversations.
#AI #Automation #Airflow #MachineLearning
53 פרקים
Orchestrating Analytics and AI Workflows at Telia with Arjun Anandkumar
The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI
Manage episode 463948888 series 2948506
The future of data engineering lies in seamless orchestration and automation. In this episode, Arjun Anandkumar, Data Engineer at Telia, shares how his team uses Airflow to drive analytics and AI workflows. He highlights the challenges of scaling data platforms and how adopting best practices can simplify complex processes for teams across the organization. Arjun also discusses the transformative role of tools like Cosmos and Terraform in enhancing efficiency and collaboration.
Key Takeaways:
(02:16) Telia operates across the Nordics and Baltics, focusing on telecom and energy services.
(03:45) Airflow runs dbt models seamlessly with Cosmos on AWS MWAA.
(05:47) Cosmos improves visibility and orchestration in Airflow.
(07:00) Medallion Architecture organizes data into bronze, silver and gold layers.
(08:34) Task group challenges highlight the need for adaptable workflows.
(15:04) Scaling managed services requires trial, error and tailored tweaks.
(19:46) Terraform scales infrastructure, while YAML templates manage DAGs efficiently.
(20:00) Templated DAGs and robust testing enhance platform management.
(24:15) Open-source resources drive innovation in Airflow practices.
Resources Mentioned:
https://www.linkedin.com/in/arjunanand1/?originalSubdomain=dk
Telia -
https://www.linkedin.com/company/teliacompany/
https://airflow.apache.org/
https://www.astronomer.io/cosmos/
https://www.terraform.io/
Medallion Architecture by Databricks -
https://www.databricks.com/glossary/medallion-architecture
Thanks for listening to “The Data Flowcast: Mastering Airflow for Data Engineering & AI.” If you enjoyed this episode, please leave a 5-star review to help get the word out about the show. And be sure to subscribe so you never miss any of the insightful conversations.
#AI #Automation #Airflow #MachineLearning
53 פרקים
כל הפרקים
×
1 Introducing Apache Airflow® 3 with Vikram Koka and Jed Cunningham 27:28

1 Airflow in Action: Powering Instacart's Complex Ecosystem 25:14

1 From ETL to Airflow: Transforming Data Engineering at Deloitte Digital with Raviteja Tholupunoori 27:42

1 A Deep Dive Into the 2025 State of Airflow Survey Results with Tamara Fingerlin of Astronomer 23:26

1 Airflow’s Role in the Rise of DataOps with Andy Byron 26:15

1 The Software Risk That Affects Everyone and How To Address It with Michael Winser and Jarek Potiuk 28:27

1 Building Scalable ML Infrastructure at Outerbounds with Savin Goyal 36:46

1 Customizing Airflow for Complex Data Environments at Stripe with Nick Bilozerov and Sharadh Krishnamurthy 27:40

1 Harnessing Airflow for Data-Driven Policy Research at CSET with Jennifer Melot 17:54

1 Leveraging Airflow To Build Scalable and Reliable Data Platforms at 99acres.com with Samyak Jain 25:08

1 Hybrid Testing Solutions for Autonomous Driving at Bosch with Jens Scheffler and Christian Schilling 33:45

1 Overcoming Airflow Scaling Challenges at Monzo Bank with Jonathan Rainer 43:39

1 Orchestrating Analytics and AI Workflows at Telia with Arjun Anandkumar 26:00

1 The Role of Airflow in Finance Transformation at Etraveli Group with Mihir Samant 21:19

1 Inside Ford’s Data Transformation: Advanced Orchestration Strategies with Vasantha Kosuri-Marshall 38:54
ברוכים הבאים אל Player FM!
Player FM סורק את האינטרנט עבור פודקאסטים באיכות גבוהה בשבילכם כדי שתהנו מהם כרגע. זה יישום הפודקאסט הטוב ביותר והוא עובד על אנדרואיד, iPhone ואינטרנט. הירשמו לסנכרון מנויים במכשירים שונים.