32 subscribers
התחל במצב לא מקוון עם האפליקציה Player FM !
Deploy and fine-tune LLM models on Kubernetes using KAITO
Manage episode 433011321 series 3332465
In this episode of the Kubernetes Bytes podcast, Bhavin sits down with Sachi Desai, Product Manager and Paul Yu, Sr. Cloud Advocate at Microsoft to talk about the open source KAITO project. KAITO is the Kubernetes AI Toolchain Operator that enables AKS users to deploy open source LLM models on their Kubernetes clusters. They discuss how KAITO helps with running AI-enabled applications alongside the LLM models, how it helps users bring their own LLM models and run them as containers, and how KAITO helps them fine-tune open source LLMs on their Kubernetes clusters.
Check out our website at https://kubernetesbytes.com/
Cloud Native News:
- https://azure.github.io/AKS/2024/07/30/azure-container-storage-ga
- https://github.blog/news-insights/product-news/introducing-github-models/
Show links:
- Azure/kaito: Kubernetes AI Toolchain Operator - https://github.com/Azure/kaito/tree/main
- https://www.youtube.com/watch?v=3cGmHDjR_3I&list=PLc3Ep462vVYtgN4rP1ThTJd2UlsBc2sou&index=2
- https://aka.ms/cloudnative/learnlive/intelligent-apps-on-aks/episode-2
- Jumpstart AI Workflows With Kubernetes AI Toolchain Operator - The New Stack - https://thenewstack.io/jumpstart-ai-workflows-with-kubernetes-ai-toolchain-operator
- https://paulyu.dev/article/soaring-with-kaito/
- Concepts - Fine-tuning language models for AI and machine learning workflows - Azure Kubernetes Service | Microsoft Learn - https://learn.microsoft.com/en-us/azure/aks/concepts-fine-tune-language-models
- Keep up to date on the most recent announcements by following some of the KAITO engineers on LinkedIn:
- Fei Guo - https://www.linkedin.com/in/fei-guo-a48319a/
- Ishaan Sehgal - https://www.linkedin.com/in/ishaan-sehgal/
Timestamps:
- 00:02:15 Cloud Native News
- 00:05:34 Interview with Sachi and Paul
- 00:42:08 Key takeaways
88 פרקים
Manage episode 433011321 series 3332465
In this episode of the Kubernetes Bytes podcast, Bhavin sits down with Sachi Desai, Product Manager and Paul Yu, Sr. Cloud Advocate at Microsoft to talk about the open source KAITO project. KAITO is the Kubernetes AI Toolchain Operator that enables AKS users to deploy open source LLM models on their Kubernetes clusters. They discuss how KAITO helps with running AI-enabled applications alongside the LLM models, how it helps users bring their own LLM models and run them as containers, and how KAITO helps them fine-tune open source LLMs on their Kubernetes clusters.
Check out our website at https://kubernetesbytes.com/
Cloud Native News:
- https://azure.github.io/AKS/2024/07/30/azure-container-storage-ga
- https://github.blog/news-insights/product-news/introducing-github-models/
Show links:
- Azure/kaito: Kubernetes AI Toolchain Operator - https://github.com/Azure/kaito/tree/main
- https://www.youtube.com/watch?v=3cGmHDjR_3I&list=PLc3Ep462vVYtgN4rP1ThTJd2UlsBc2sou&index=2
- https://aka.ms/cloudnative/learnlive/intelligent-apps-on-aks/episode-2
- Jumpstart AI Workflows With Kubernetes AI Toolchain Operator - The New Stack - https://thenewstack.io/jumpstart-ai-workflows-with-kubernetes-ai-toolchain-operator
- https://paulyu.dev/article/soaring-with-kaito/
- Concepts - Fine-tuning language models for AI and machine learning workflows - Azure Kubernetes Service | Microsoft Learn - https://learn.microsoft.com/en-us/azure/aks/concepts-fine-tune-language-models
- Keep up to date on the most recent announcements by following some of the KAITO engineers on LinkedIn:
- Fei Guo - https://www.linkedin.com/in/fei-guo-a48319a/
- Ishaan Sehgal - https://www.linkedin.com/in/ishaan-sehgal/
Timestamps:
- 00:02:15 Cloud Native News
- 00:05:34 Interview with Sachi and Paul
- 00:42:08 Key takeaways
88 פרקים
כל הפרקים
×
1 Database as a service with Percona Everest 1:02:44


1 Increasing AI adoption using Kubernetes 52:03

1 Monolith to Microservices using Kubernetes at Guidewire 1:06:28

1 Inference in Action: Scaling Al Smarter with Inferless 55:17

1 Container security with Wiz 1:02:33

1 Dagger.io Deep Dive with Co-Founder Sam Alba 1:06:24

1 Running Ray on Kubernetes with KubeRay 53:06

1 Building scalable data platforms using Data on EKS 1:02:20

1 Deploy and fine-tune LLM models on Kubernetes using KAITO 44:17

1 The business case for cloud-native and Kubernetes 54:24

1 Building the AI Hyperscaler with Kubernetes 54:56

1 Shifting Minds: Exploring OpenShift's AI Landscape 1:05:07

1 Training Machine Learning (ML) models on Kubernetes 55:29

1 The evolution of service mesh technologies 1:08:00



1 Open Policy Agent (OPA) 101 1:07:20

1 Ops Ops Hooray! Navigating IDPs from an Ops perspective 58:17

1 Generative AI on Kubernetes 1:15:56

1 IDPs Unveiled: Accelerating Deployment on Kubernetes 59:52

1 Running Kubernetes at the Edge using K3s 53:51

1 Running multi-tenant Kubernetes clusters using vCluster 57:58


1 Byte-sized: Exploring the Basics of AI in Plain English 1:00:18

1 Kubecon North America 2023: Highlights, Themes and Key Takeaways 57:41

1 Universal Control Planes for Kubernetes and Beyond 59:36

1 DevOpsDays Boston - Helping developers be more productive in a multi-cloud world 35:02

1 DevOpsDays Boston - Platform Engineering and Internal Developer Platforms 31:12

1 DevOpsDays Boston - Real value of community 42:41


1 How Chick-fil-A adopts GitOps and K3s at the Edge 1:19:16

1 Nodeless Kubernetes - Optimizing costs with just in time compute 1:02:20

1 Solving Multicloud with Seamless Connectivity and AI - with Rob Croteau 58:04


1 Generative AI: The New Frontier in Kubernetes Problem-Solving 1:04:58

1 From Manual to Automatic: Revolutionizing Cloud Native Stack Deployment with Argonaut 1:08:16

1 Accelerating Kubernetes Adoption: Unleashing the Power of GitOps using Kubefirst 1:00:47

1 Continuous Security: Keeping Pace in the DevOps Lifecycle w/ ARMO 1:01:44

1 Unleashing the power of KubeVirt - Running Containers and VMs on Kubernetes 1:14:39

1 Breaking Down the Diamond: A Look at MLB's Kubernetes-Powered Analytics 53:36

1 Kubecon Europe 2023: Highlights and Key Takeaways 46:35

1 Kubernetes Community Corner with Michael O'Leary: Exploring the Intersection of Learning and Collaboration 48:33

1 Kubernetes in Cloud Native Healthcare 56:31

ברוכים הבאים אל Player FM!
Player FM סורק את האינטרנט עבור פודקאסטים באיכות גבוהה בשבילכם כדי שתהנו מהם כרגע. זה יישום הפודקאסט הטוב ביותר והוא עובד על אנדרואיד, iPhone ואינטרנט. הירשמו לסנכרון מנויים במכשירים שונים.