51 subscribers
התחל במצב לא מקוון עם האפליקציה Player FM !
110. Why should you use Lambda for Machine Learning?
Manage episode 396187208 series 2980070
In this episode, we discuss using AWS Lambda for machine learning inference. We cover the tradeoffs between GPUs and CPUs for ML, tools like ggml and llama.cpp for running models on CPUs, and share examples where we've experimented with Lambda for ML like podcast transcription, medical imaging, and natural language processing. While Lambda ML is still quite experimental, it can be a viable option for certain use cases.
💰 SPONSORS 💰 AWS Bites is brought to you by fourTheorem, an Advanced AWS Partner. If you are moving to AWS or need a partner to help you go faster, check us out at fourtheorem.com ! In this episode, we mentioned the following resources.
- Episode "46. How do you do machine learning on AWS?": https://awsbites.com/46-how-do-you-do-machine-learning-on-aws/
- Episode "108. How to Solve Lambda Python Cold Starts": https://awsbites.com/108-how-to-solve-lambda-python-cold-starts/
- ggml (the framework): https://github.com/ggerganov/ggml
- ggml (the company): https://ggml.ai
- llama.cpp: https://github.com/ggerganov/llama.cpp
- whisper.cpp: https://github.com/ggerganov/whisper.cpp
- whisper.cpp WebAssembly demo: https://whisper.ggerganov.com/
- ONNX Runtime: https://onnxruntime.ai/
- An example of using whisper.cpp with the Rust bindings: https://github.com/lmammino/whisper-rs-example
- Project running Whisper.cpp in a Lambda function: https://github.com/eoinsha/whisper_lambda_cpp
- AWS Lambda Image Container Chest X-Ray Example: https://github.com/fourTheorem/lambda-image-cxr-detection
- Episode "103. Building GenAI Features with Bedrock": https://awsbites.com/103-building-genai-features-with-bedrock/
Do you have any AWS questions you would like us to address? Leave a comment here or connect with us on X, formerly Twitter: - https://twitter.com/eoins - https://twitter.com/loige
148 פרקים
Manage episode 396187208 series 2980070
In this episode, we discuss using AWS Lambda for machine learning inference. We cover the tradeoffs between GPUs and CPUs for ML, tools like ggml and llama.cpp for running models on CPUs, and share examples where we've experimented with Lambda for ML like podcast transcription, medical imaging, and natural language processing. While Lambda ML is still quite experimental, it can be a viable option for certain use cases.
💰 SPONSORS 💰 AWS Bites is brought to you by fourTheorem, an Advanced AWS Partner. If you are moving to AWS or need a partner to help you go faster, check us out at fourtheorem.com ! In this episode, we mentioned the following resources.
- Episode "46. How do you do machine learning on AWS?": https://awsbites.com/46-how-do-you-do-machine-learning-on-aws/
- Episode "108. How to Solve Lambda Python Cold Starts": https://awsbites.com/108-how-to-solve-lambda-python-cold-starts/
- ggml (the framework): https://github.com/ggerganov/ggml
- ggml (the company): https://ggml.ai
- llama.cpp: https://github.com/ggerganov/llama.cpp
- whisper.cpp: https://github.com/ggerganov/whisper.cpp
- whisper.cpp WebAssembly demo: https://whisper.ggerganov.com/
- ONNX Runtime: https://onnxruntime.ai/
- An example of using whisper.cpp with the Rust bindings: https://github.com/lmammino/whisper-rs-example
- Project running Whisper.cpp in a Lambda function: https://github.com/eoinsha/whisper_lambda_cpp
- AWS Lambda Image Container Chest X-Ray Example: https://github.com/fourTheorem/lambda-image-cxr-detection
- Episode "103. Building GenAI Features with Bedrock": https://awsbites.com/103-building-genai-features-with-bedrock/
Do you have any AWS questions you would like us to address? Leave a comment here or connect with us on X, formerly Twitter: - https://twitter.com/eoins - https://twitter.com/loige
148 פרקים
כל הפרקים
×
1 144. Lambda Billing Changes, Cold Start Costs, and Log Savings: What You Need to Know 13:22

1 143. Is App Runner better than Fargate? 42:09

1 141. Step Functions with JSONata and Variables 15:43

1 140. DuckDB Meets AWS: A Match Made in Cloud 17:38

1 139. Building Great APIs with Powertools 24:32

1 138. How Do You Become A Cloud Architect? 39:02


1 136. 20 Amazing New AWS Features 17:39

1 135. Serverless is making a comeback where you least expect it 21:46


1 133. Building Businesses in the Cloud with Fiona McKenna 28:28

1 132. GitHub Action Runners on AWS 24:40

1 131. What do you do about CloudFormation Drift? 19:44

1 130. Growing in Tech with Farrah Campbell 29:55
ברוכים הבאים אל Player FM!
Player FM סורק את האינטרנט עבור פודקאסטים באיכות גבוהה בשבילכם כדי שתהנו מהם כרגע. זה יישום הפודקאסט הטוב ביותר והוא עובד על אנדרואיד, iPhone ואינטרנט. הירשמו לסנכרון מנויים במכשירים שונים.