79 subscribers
התחל במצב לא מקוון עם האפליקציה Player FM !
פודקאסטים ששווה להאזין
בחסות


1 The Icelandic Art of Intuition with Hrund Gunnsteinsdóttir | 307 40:34
Teaching AI to See: A Technical Deep-Dive on Vision Language Models with Will Hardman of Veratai
Manage episode 459113635 series 3452589
In this episode of The Cognitive Revolution, Nathan hosts Will Hardman, founder of AI advisory firm Veritai, for a comprehensive technical survey of vision language models (VLMs). We explore the evolution of VLMs from early vision transformers to state-of-the-art architectures like InternVL and Llama3V, examining key innovations and architectural decisions. Join us for an in-depth discussion covering multimodality in AI systems, evaluation frameworks, and practical applications with one of the field's leading experts.
Here's to the link to one of the most comprehensive reference documents for VLMs prepared by Will Hardman: https://dust-mailbox-c73.notion.site/Vision-Language-Models-11b675d75dd480af994cc474a754bb26
Help shape our show by taking our quick listener survey at https://bit.ly/TurpentinePulse
SPONSORS:
Oracle Cloud Infrastructure (OCI): Oracle's next-generation cloud platform delivers blazing-fast AI and ML performance with 50% less for compute and 80% less for outbound networking compared to other cloud providers. OCI powers industry leaders like Vodafone and Thomson Reuters with secure infrastructure and application development capabilities. New U.S. customers can get their cloud bill cut in half by switching to OCI before March 31, 2024 at https://oracle.com/cognitive
80,000 Hours: 80,000 Hours is dedicated to helping you find a fulfilling career that makes a difference. With nearly a decade of research, they offer in-depth material on AI risks, AI policy, and AI safety research. Explore their articles, career reviews, and a podcast featuring experts like Anthropic CEO Dario Amodei. Everything is free, including their Career Guide. Visit https://80000hours.org/cognitiverevolution to start making a meaningful impact today.
CHAPTERS:
(00:00:00) Teaser
(00:00:55) About the Episode
(00:05:45) Introduction
(00:09:16) VLM Use Cases
(00:13:47) Vision Transformers (Part 1)
(00:17:48) Sponsors: Oracle Cloud Infrastructure (OCI)
(00:19:00) Vision Transformers (Part 2)
(00:24:58) OpenAI's CLIP Model
(00:33:44) DeepMind's Flamingo (Part 1)
(00:33:44) Sponsors: 80,000 Hours
(00:35:17) DeepMind's Flamingo (Part 2)
(00:48:29) Instruction Tuning with LAVA
(01:09:25) MMMU Benchmark
(01:14:42) Pre-training with QNVL
(01:32:13) InternVL Model Series
(01:52:33) Cross-Attention vs. Self-Attention
(02:14:33) Hybrid Architectures
(02:31:08) Early vs. Late Fusion
(02:34:50) VQA and DocVQA Benchmarks
(02:40:08) The Blink Benchmark
(03:05:37) Generative Pre-training
(03:15:26) Multimodal Generation
(03:37:00) Frontier Labs & Benchmarks
(03:47:45) Conclusion
(03:53:28) Outro
SOCIAL LINKS:
Website: https://www.cognitiverevolution.ai
Twitter (Podcast): https://x.com/cogrev_podcast
Twitter (Nathan): https://x.com/labenz
LinkedIn: https://www.linkedin.com/in/nathanlabenz/
Youtube: https://www.youtube.com/@CognitiveRevolutionPodcast
Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk
249 פרקים
Teaching AI to See: A Technical Deep-Dive on Vision Language Models with Will Hardman of Veratai
"The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis
Manage episode 459113635 series 3452589
In this episode of The Cognitive Revolution, Nathan hosts Will Hardman, founder of AI advisory firm Veritai, for a comprehensive technical survey of vision language models (VLMs). We explore the evolution of VLMs from early vision transformers to state-of-the-art architectures like InternVL and Llama3V, examining key innovations and architectural decisions. Join us for an in-depth discussion covering multimodality in AI systems, evaluation frameworks, and practical applications with one of the field's leading experts.
Here's to the link to one of the most comprehensive reference documents for VLMs prepared by Will Hardman: https://dust-mailbox-c73.notion.site/Vision-Language-Models-11b675d75dd480af994cc474a754bb26
Help shape our show by taking our quick listener survey at https://bit.ly/TurpentinePulse
SPONSORS:
Oracle Cloud Infrastructure (OCI): Oracle's next-generation cloud platform delivers blazing-fast AI and ML performance with 50% less for compute and 80% less for outbound networking compared to other cloud providers. OCI powers industry leaders like Vodafone and Thomson Reuters with secure infrastructure and application development capabilities. New U.S. customers can get their cloud bill cut in half by switching to OCI before March 31, 2024 at https://oracle.com/cognitive
80,000 Hours: 80,000 Hours is dedicated to helping you find a fulfilling career that makes a difference. With nearly a decade of research, they offer in-depth material on AI risks, AI policy, and AI safety research. Explore their articles, career reviews, and a podcast featuring experts like Anthropic CEO Dario Amodei. Everything is free, including their Career Guide. Visit https://80000hours.org/cognitiverevolution to start making a meaningful impact today.
CHAPTERS:
(00:00:00) Teaser
(00:00:55) About the Episode
(00:05:45) Introduction
(00:09:16) VLM Use Cases
(00:13:47) Vision Transformers (Part 1)
(00:17:48) Sponsors: Oracle Cloud Infrastructure (OCI)
(00:19:00) Vision Transformers (Part 2)
(00:24:58) OpenAI's CLIP Model
(00:33:44) DeepMind's Flamingo (Part 1)
(00:33:44) Sponsors: 80,000 Hours
(00:35:17) DeepMind's Flamingo (Part 2)
(00:48:29) Instruction Tuning with LAVA
(01:09:25) MMMU Benchmark
(01:14:42) Pre-training with QNVL
(01:32:13) InternVL Model Series
(01:52:33) Cross-Attention vs. Self-Attention
(02:14:33) Hybrid Architectures
(02:31:08) Early vs. Late Fusion
(02:34:50) VQA and DocVQA Benchmarks
(02:40:08) The Blink Benchmark
(03:05:37) Generative Pre-training
(03:15:26) Multimodal Generation
(03:37:00) Frontier Labs & Benchmarks
(03:47:45) Conclusion
(03:53:28) Outro
SOCIAL LINKS:
Website: https://www.cognitiverevolution.ai
Twitter (Podcast): https://x.com/cogrev_podcast
Twitter (Nathan): https://x.com/labenz
LinkedIn: https://www.linkedin.com/in/nathanlabenz/
Youtube: https://www.youtube.com/@CognitiveRevolutionPodcast
Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk
249 פרקים
כל הפרקים
×
1 Mechanistic Interpretability: Philosophy, Practice & Progress with Goodfire's Dan Balsam & Tom McGrath 1:52:52

1 The Perfect Substrate for AGI, with Replit CEO Amjad Masad 1:01:16

1 The RAISE Act: Minimum Standards for Frontier AI Development, with NY Assembly Member Alex Bores 1:43:36

1 Gemini Robotics – AI for the Physical World, with Keerthana Gopalakrishnan and Ted Xiao of Google DeepMind 1:47:38

1 Titans: Neural Long-Term Memory for LLMs, with author Ali Behrouz 2:11:25

1 Luma Labs' Diffusion Revolution: from Dream Machine to Multimodal Worldsim - Amit Jain, Jiaming Song 1:19:32

1 OpenAI's Identity Crisis: History, Culture & Non-Profit Control with ex-employee Steven Adler 2:03:13

1 AI Control: Using Untrusted Systems Safely with Buck Shlegeris of Redwood Research, from the 80,000 Hours Podcast 2:29:21

1 Blueprint for AI Armageddon: Josh Clymer Imagines AI Takeover, from the Audio Tokens Podcast 2:02:05

1 Fiverr Goes All-In on AI: Empowering Creators, Not Replacing Them, with Micha Kaufman, CEO of Fiverr 1:43:05

1 Securing Superintelligence: National Security, Espionage & AI Control with Jeremie & Edouard Harris 2:09:51

1 Is OpenAI's o3 AGI? Zvi Mowshowitz on Early AI Takeoff, the Mechanize launch, Live Players, & Why p(doom) is Rising 3:08:19

1 AI News Crossover: A Candid Chat with Liron Shapira of Doom Debates 2:29:35

1 Helen Toner: OpenAI Reflections, Adaptation Buffers, and AI in Warfare 1:29:35

1 Is a US-China Thucydides Trap Unavoidable? With David C. Kang from the ChinaTalk Podcast 1:39:16
ברוכים הבאים אל Player FM!
Player FM סורק את האינטרנט עבור פודקאסטים באיכות גבוהה בשבילכם כדי שתהנו מהם כרגע. זה יישום הפודקאסט הטוב ביותר והוא עובד על אנדרואיד, iPhone ואינטרנט. הירשמו לסנכרון מנויים במכשירים שונים.