MongoDB’s Sahir Azam: Vector Databases and the Data Structure of AI

Training Data

Player FM - Internet Radio Done Right

16 subscribers

הוסף לפני one שנה

תוכן מסופק על ידי Sequoia Capital. כל תוכן הפודקאסטים כולל פרקים, גרפיקה ותיאורי פודקאסטים מועלים ומסופקים ישירות על ידי Sequoia Capital או שותף פלטפורמת הפודקאסט שלהם. אם אתה מאמין שמישהו משתמש ביצירה שלך המוגנת בזכויות יוצרים ללא רשותך, אתה יכול לעקוב אחר התהליך המתואר כאן https://he.player.fm/legal.

Before The Chorus

1
LIVE: Before the Chorus & Open Folk Present: In These Lines feat. Gaby Moreno, Lily Kershaw & James Spaite 33:58

לפני 12 ימים33:58

הפעל מאוחר יותר

רשימות

לייק

אהבתי

33:58

On June 25th 2025, in collaboration with Open Folk, we presented our first ever live interview event in Los Angeles. As Open Folk put it: "In These Lines is a live event where three artists each bring one song — not just to perform, but to explore. They sit down with Sofia Loporcaro, host of Before The Chorus, to talk about where the song came from, what it meant to write it, and what it still holds. Then they play it. Just the song, and the truth behind it." Find Open Folk on Instagram: @openfolkla Find Gaby on Spotify: https://open.spotify.com/artist/0K9pSmFx0kWESA9jqx8aCW?si=Wz4RUP88Qlm_RKs7QTLvWQ On Apple Music: https://music.apple.com/us/artist/gaby-moreno/472697737 Instagram: https://www.instagram.com/gaby_moreno/ Find Lily on Spotify: https://open.spotify.com/artist/0p0ksmwMDQlAM24TWKu4Ua?si=Bmdg-uIUTHu-zRUc_dqL3g On Apple Music: https://music.apple.com/us/artist/lily-kershaw/526884610 Instagram: https://www.instagram.com/lilykershaw/ Find James on Spotify: https://open.spotify.com/artist/3u50TPoLvMBXNT1KrLa3iT?si=OoLoq7ZTRZyUiytQcz0FsQ On Apple Music: https://music.apple.com/us/artist/james-spaite/905076868 Instagram: https://www.instagram.com/jamesspaite/ Subscribe: ⁠⁠⁠⁠⁠⁠⁠https://beforethechorus.bio.to/listen⁠⁠⁠⁠⁠⁠⁠ Sign up for our newsletter: ⁠⁠⁠⁠⁠⁠⁠https://www.beforethechorus.com/⁠⁠⁠⁠⁠⁠⁠ Follow on Instagram: ⁠⁠⁠⁠⁠⁠⁠@beforethechoruspodcast⁠⁠⁠⁠⁠⁠⁠ & ⁠⁠⁠⁠⁠⁠⁠@soundslikesofia⁠⁠⁠⁠⁠⁠⁠ About the podcast: Welcome to Before the Chorus , where we go beyond the sounds of our favourite songs to hear the stories of the artists who wrote them. Before a song is released, a record is produced, or a chorus is written, the musicians that write them think. A lot. They live. A lot. And they feel. A LOT. Hosted by award-winning interviewer Sofia Loporcaro, Before the Chorus explores the genuine human experiences behind the music. Sofia’s deep knowledge of music and personal journey with mental health help her connect with artists on a meaningful level. This is a space where fans connect with artists, and listeners from all walks of life feel seen through the stories that shape the music we love. About the host: Sofia Loporcaro is an award-winning interviewer and radio host who’s spent over 8 years helping musicians share their stories. She’s hosted shows for Amazing Radio, and Transmission Roundhouse. Now on Before the Chorus, she’s had the chance to host guests like Glass Animals, Feist, Madison Cunningham, Mick Jenkins, & Ru Paul's Drag Race winner Shea Couleé. Learn more about your ad choices. Visit megaphone.fm/adchoices…

לפני שנה 44:26

MP3•בית הפרקים

Hosted by: Sonya Huang and Pat Grady, Sequoia Capital

Mentioned in this episode:

Introducing ambient agents: Blog post by Langchain on a new UX pattern where AI agents can listen to an event stream and act on it
Google Gemini Deep Research: Sahir enjoys its amazing product experience
Perplexity: AI search app that Sahir admires for its product craft
Snipd: AI powered podcast app Sahir likes

57 פרקים

#Tech #Business #Sequoia Capital

MongoDB’s Sahir Azam: Vector Databases and the Data Structure of AI

Training Data

16 subscribers

published לפני שנה

שתפו

MP3•בית הפרקים

Hosted by: Sonya Huang and Pat Grady, Sequoia Capital

Mentioned in this episode:

Introducing ambient agents: Blog post by Langchain on a new UX pattern where AI agents can listen to an event stream and act on it
Google Gemini Deep Research: Sahir enjoys its amazing product experience
Perplexity: AI search app that Sahir admires for its product craft
Snipd: AI powered podcast app Sahir likes

57 פרקים

#Tech #Business #Sequoia Capital

כל הפרקים

1
OpenAI Just Released ChatGPT Agent, Its Most Powerful Agent Yet 37:36

לפני 5 ימים37:36

37:36

Isa Fulford, Casey Chu, and Edward Sun from OpenAI's ChatGPT agent team reveal how they combined Deep Research and Operator into a single, powerful AI agent that can perform complex, multi-step tasks lasting up to an hour. By giving the model access to a virtual computer with text browsing, visual browsing, terminal access, and API integrations—all with shared state—they've created what may be the first truly embodied AI assistant. The team discusses their reinforcement learning approach, safety mitigations for real-world actions, and how small teams can build transformative AI products through close research-applied collaboration. Hosted by Sonya Huang and Lauren Reeder, Sequoia Capital…

1
DeepMind's Pushmeet Kohli on AI's Scientific Revolution 41:13

לפני 16 ימים41:13

41:13

Pushmeet Kohli leads AI for Science at DeepMind, where his team has created AlphaEvolve, an AI system that discovers entirely new algorithms and proves mathematical results that have eluded researchers for decades. From improving 50-year-old matrix multiplication algorithms to generating interpretable code for complex problems like data center scheduling, AlphaEvolve represents a new paradigm where LLMs coupled with evolutionary search can outperform human experts. Pushmeet explains the technical architecture behind these breakthroughs and shares insights from collaborations with mathematicians like Terence Tao, while discussing how AI is accelerating scientific discovery across domains from chip design to materials science. Hosted by Sonya Huang and Pat Grady, Sequoia Capital…

1
Mapping the Mind of a Neural Net: Goodfire’s Eric Ho on the Future of Interpretability 47:07

לפני 18 ימים47:07

47:07

Eric Ho is building Goodfire to solve one of AI’s most critical challenges: understanding what’s actually happening inside neural networks. His team is developing techniques to understand, audit and edit neural networks at the feature level. Eric discusses breakthrough results in resolving superposition through sparse autoencoders, successful model editing demonstrations and real-world applications in genomics with Arc Institute's DNA foundation models. He argues that interpretability will be critical as AI systems become more powerful and take on mission-critical roles in society. Hosted by Sonya Huang and Roelof Botha, Sequoia Capital Mentioned in this episode: Mech interp : Mechanistic interpretability, list of important papers here Phineas Gage : 19th century railway engineer who lost most of his brain’s left frontal lobe in an accident. Became a famous case study in neuroscience. Human Genome Project : Effort from 1990-2003 to generate the first sequence of the human genome which accelerated the study of human biology Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs Zoom In: An Introduction to Circuits : First important mechanistic interpretability paper from OpenAI in 2020 Superposition : Concept from physics applied to interpretability that allows neural networks to simulate larger networks (e.g. more concepts than neurons) Apollo Research : AI safety company that designs AI model evaluations and conducts interpretability research Towards Monosemanticity: Decomposing Language Models With Dictionary Learning . 2023 Anthropic paper that uses a sparse autoencoder to extract interpretable features; followed by Scaling Monosemanticity Under the Hood of a Reasoning Model : 2025 Goodfire paper that interprets DeepSeek’s reasoning model R1 Auto-interpretability : The ability to use LLMs to automatically write explanations for the behavior of neurons in LLMs Interpreting Evo 2: Arc Institute's Next-Generation Genomic Foundation Model . (see episode with Arc co-founder Patrick Hsu ) Paint with Ember : Canvas interface from Goodfire that lets you steer an LLM’s visual output in real time (paper here ) Model diffing : Interpreting how a model differs from checkpoint to checkpoint during finetuning Feature steering : The ability to change the style of LLM output by up or down weighting features (e.g. talking like a pirate vs factual information about the Andromeda Galaxy) Weight based interpretability : Method for directly decomposing neural network parameters into mechanistic components, instead of using features The Urgency of Interpretability: Essay by Anthropic founder Dario Amodei On the Biology of a Large Language Model: Goodfire collaboration with Anthropic…

1
ElevenLabs’ Mati Staniszewski: Why Voice Will Be the Fundamental Interface for Tech 59:53

לפני 25 ימים59:53

59:53

Mati Staniszewski, co-founder and CEO of ElevenLabs, explains how staying laser-focused on audio innovation has allowed his company to thrive despite the push into multimodality from foundation models. From a high school friendship in Poland to building one of the fastest-growing AI companies, Mati shares how ElevenLabs transformed text-to-speech with contextual understanding and emotional delivery. He discusses the company's viral moments (from Harry Potter by Balenciaga to powering Darth Vader in Fortnite), and explains how ElevenLabs is creating the infrastructure for voice agents and real-time translation that could eliminate language barriers worldwide. Hosted by: Pat Grady, Sequoia Capital Mentioned in this episode: Attention Is All You Need : The original Transformers paper Tortoise-tts : Open source text to speech model that was a starting point for ElevenLabs (which now maintains a v2 ) Harry Potter by Balenciaga : ElevenLabs’ first big viral moment from 2023 The first AI that can laugh : 2022 blog post backing up ElevenLab’s claim of laughter (it got better in v3 ) Darth Vader's voice in Fortnite : ElevenLabs used actual voice clips provided by James Earl Jones before he died Lex Fridman interviews Prime Minister Modi : ElevenLabs enabled Fridman to speak in Hindi and Modi to speak in English. Time Person of the Year 2024 : ElevenLabs-powered experiment with “conversational journalism” Iconic Voices : Richard Feynman, Deepak Chopra, Maya Angelou and more available in ElevenLabs reader app SIP trunking : a method of delivering voice, video, and other unified communications over the internet using the Session Initiation Protocol (SIP) Genesys : Leading enterprise CX platform for agentic AI Hitchhiker’s Guide to the Galaxy : Comedy/science-fiction series by Douglas Adams that contains the concept of the Babel Fish instantaneous translator, cited by Mati FYI : communication and productivity app for creatives that Mati uses, founded by will.i.am Lovable : prototyping app that Mati loves…

1
From DevOps ‘Heart Attacks’ to AI-Powered Diagnostics With Traversal’s AI Agents 40:32

לפני 5 weeks40:32

40:32

Anish Agarwal and Raj Agrawal, co-founders of Traversal, are transforming how enterprises handle critical system failures. Their AI agents can perform root cause analysis in 2-4 minutes instead of the hours typically spent by teams of engineers scrambling in Slack channels. Drawing from their academic research in causal inference and gene regulatory networks, they’ve built agents that systematically traverse complex dependency maps to identify the smoking gun logs and problematic code changes. As AI-generated code becomes more prevalent, Traversal addresses a growing challenge: debugging systems where humans didn’t write the original code, making AI-powered troubleshooting essential for maintaining reliable software at scale. Hosted by Sonya Huang and Bogomil Balkansky, Sequoia Capital Mentioned in this episode: SRE : Site reliability engineering. The function within engineering teams that monitors and improves the availability and performance of software systems and services. Golden signals : four key metrics used by Site Reliability Engineers (SREs) to monitor the health and performance of IT systems: latency, traffic, errors and saturation. MELT data : Metrics, events, log, and traces. A framework for observability. The Bitter Lesson : Another mention of Nobel Prize winner Rich Sutton’s influential post.…

1
The Breakthroughs Needed for AGI Have Already Been Made: OpenAI Former Research Head Bob McGrew 48:51

לפני 6 weeks48:51

48:51

As OpenAI's former Head of Research, Bob McGrew witnessed the company's evolution from GPT-3’s breakthrough to today's reasoning models. He argues that there are three legs of the stool for AGI—Transformers, scaled pre-training, and reasoning—and that the fundamentals that will shape the next decade-plus are already in place. He thinks 2025 will be defined by reasoning while pre-training hits diminishing returns. Bob discusses why the agent economy will price services at compute costs due to near-infinite supply, fundamentally disrupting industries like law and medicine, and how his children use ChatGPT to spark curiosity and agency. From robotics breakthroughs to managing brilliant researchers, Bob offers a unique perspective on AI’s trajectory and where startups can still find defensible opportunities. Hosted by: Stephanie Zhan and Sonya Huang, Sequoia Capital Mentioned in this episode : Solving Rubik’s Cube with a robot hand : OpenAI’s original robotics research Computer Use and Operator : Anthropic and OpenAI reasoning breakthroughs that originated with OpenAi researchers Skild and Physical Intelligence : Robotics-oriented companies Bob sees as well-positioned now Distyl : AI company founded by ex-Palintir alums to create enterprise workflows driven by proprietary data Member of the technical staff : Title at OpenAI designed to break down barriers between AI researchers and engineers Howie.ai : Scheduling app that Bob uses…

1
OpenAI Codex Team: From Coding Autocomplete to Asynchronous Autonomous Agents 37:44

לפני 7 weeks37:44

37:44

Hanson Wang and Alexander Embiricos from OpenAI's Codex team discuss their latest AI coding agent that works independently in its own environment for up to 30 minutes, generating full pull requests from simple task descriptions. They explain how they trained the model beyond competitive programming to match real-world software engineering needs, the shift from pairing with AI to delegating to autonomous agents, and their vision for a future where the majority of code is written by agents working on their own computers. The conversation covers the technical challenges of long-running inference, the importance of creating realistic training environments, and how developers are already using Codex to fix bugs and implement features at OpenAI. Hosted by Sonya Huang and Lauren Reeder, Sequoia Capital Mentioned in this episode : The Culture : Sci-Fi series by Iain Banks portraying an optimistic view of AI The Bitter Lesson : Influential paper by Rich Sutton on the importance of scale as a strategic unlock for AI.…

1
Google I/O Afterparty: The Future of Human-AI Collaboration, From Veo to Mariner 53:51

לפני 8 weeks53:51

53:51

Fresh off impressive releases at Google’s I/O event, three Google Labs leaders explain how they’re reimagining creative tools and productivity workflows. Thomas Iljic details how video generation is merging filmmaking with gaming through generative AI cameras and world-building interfaces in Whisk and Veo. Jaclyn Konzelmann demonstrates how Project Mariner evolved from a disruptive browser takeover to an intelligent background assistant that remembers context across multiple tasks. Simon Tokumine reveals NotebookLM’s expansion beyond viral audio overviews into a comprehensive platform for transforming information into personalized formats. The conversation explores the shift from prompting to showing and telling, the economics of AI-powered e-commerce, and why being “too early” has become Google Labs’ biggest challenge and advantage. Hosted by Sonya Huang, Sequoia Capital 00:00 Introduction 02:12 Google's AI models and public perception 04:18 Google's history in image and video generation 06:45 Where Whisk and Flow fit 10:30 How close are we to having the ideal tool for the craft? 13:05 Where do the movie and game worlds start to merge? 16:25 Introduction to Project Mariner 17:15 How Mariner works 22:34 Mariner user behaviors 27:07 Temporary tattoos and URL memory 27:53 Project Mariner's future 29:26 Agent capabilities and use cases 31:09 E-commerce and agent interaction 35:03 Notebook LM evolution 48:26 Predictions and future of AI Mentioned in this episode: Whisk : Image and video generation app for consumers Flow : AI-powered filmmaking with new Veo 3 model Project Mariner : research prototype exploring the future of human-agent interaction, starting with browsers NotebookLM : tool for understanding and engaging with complex information including Audio Overviews and now a mobile app Shop with AI Mode : Shopping app with a virtual try-on tool based on your own photos Stitch : New prompt-based interface to design UI for mobile and web applications. ControlNet paper : Outlined an architecture for adding conditional language to direct the outputs of image generation with diffusion models…

1
From Data Centers to Dyson Spheres: P-1 AI's Path to Hardware Engineering AGI 38:25

לפני 9 weeks38:25

38:25

Former Airbus CTO Paul Eremenko shares his vision for bringing AI to physical engineering, starting with Archie—an AI agent that works alongside human engineers. P-1 AI is tackling the challenge of generating synthetic training data to teach AI systems about complex physical systems, from data center cooling to aircraft design and beyond. Eremenko explains how Archie breaks down engineering tasks into primitive operations and uses a federated approach combining multiple AI models. The goal is to progress from entry-level engineering capabilities to eventually achieving engineering AGI that can design things humans cannot. Hosted by Sonya Huang and Pat Grady, Sequoia Capital…

1
Gong’s Amit Bendov: From Meeting Recordings to Revenue AI 42:05

לפני 10 weeks42:05

42:05

CEO Amit Bendov shares how Gong evolved from a meeting transcription tool to an AI-powered revenue platform that's increasing sales capacity by up to 60%. He explains why task-specific AI agents are the key to enterprise adoption, and why human accountability will remain crucial even as AI takes over routine sales tasks. Amit also reveals how Gong survived recent market headwinds by expanding their product suite while maintaining their customer-first approach. Hosted by Sonya Huang and Pat Grady, Sequoia Capital Mentioned in this episode : “ New paradigm of AI architectures ”: Yann LeCun’s talk at Davos where he talks about the world beyond transformers and LLMs in the next 3-5 years. The Beginning of Infinity : Book by David Deutsch that Amit says is “mind-changing.”…

1
LIVE: Google's Jeff Dean on the Coming Transformations in AI 30:59

לפני 10 weeks30:59

30:59

At AI Ascent 2025, Jeff Dean makes bold predictions. Discover how the pioneer behind Google's TPUs and foundational AI research sees the technology evolving, from specialized hardware to more organic systems, and future engineering capabilities.

1
LIVE: Ambient Agents and the New Agent Inbox ft. Harrison Chase of LangChain 8:28

לפני 10 weeks8:28

8:28

Recorded live at Sequoia’s AI Ascent 2025: LangChain CEO Harrison Chase introduces the concept of ambient agents, AI systems that operate continuously in the background responding to events rather than direct human prompts. Learn how these agents differ from traditional chatbots, why human oversight remains essential and how this approach could dramatically scale our ability to leverage AI.…

1
LIVE: How AI is Reinventing Software Business Models ft. Bret Taylor of Sierra 34:30

לפני 10 weeks34:30

34:30

Recorded live at Sequoia’s AI Ascent 2025: Sierra co-founder Bret Taylor discusses why AI is driving a fundamental shift from subscription-based pricing to outcomes-based models. Learn why this transition is harder for incumbents than startups, why applied AI and vertical specialization represent the biggest opportunities for entrepreneurs and how to position your AI company for success in this new paradigm.…

1
LIVE: Sam Altman of OpenAI on Building the ‘Core AI Subscription’ for Your Life 32:27

לפני 11 weeks32:27

32:27

Recorded live at Sequoia’s AI Ascent 2025: Sam reflects on OpenAI’s evolution from a 14-person research lab to a dominant AI platform. He envisions transforming ChatGPT into a deeply personal AI service that remembers your entire life's context—from conversations to emails—while working seamlessly across all services. Sam describes the generation gap in how users engage with ChatGPT, and makes surprisingly specific predictions for the next 2-3 years of AI evolution.…

1
Workday CEO Carl Eschenbach: Building the System of Record for the AI Era 47:49

לפני 12 weeks47:49

47:49

Workday CEO and Sequoia partner Carl Eschenbach explains how the company is evolving its platform to handle both human and AI workers. He shares Workday’s three-pronged approach to monetizing AI through seat-based pricing, role-based agents and consumption-based API access. Eschenbach discusses why domain-specific agents with curated data will be more valuable than general-purpose models in the enterprise, and how Workday is helping enterprises navigate the transition to an AI-powered workplace while maintaining human connection. Hosted by: Sonya Huang and Pat Grady, Sequoia Capital…

Training Data

1
The Quest to ‘Solve All Diseases’ with AI: Isomorphic Labs’ Max Jaderberg 55:40

לפני 13 weeks55:40

55:40

After pioneering reinforcement learning breakthroughs at DeepMind with Capture the Flag and AlphaStar, Max Jaderberg aims to revolutionize drug discovery with AI as Chief AI Officer of Isomorphic Labs, which was spun out of DeepMind. He discusses how AlphaFold 3's diffusion-based architecture enables unprecedented understanding of molecular interactions, and why we're approaching a "Move 37 moment" in AI-powered drug design where models will surpass human intuition. Max shares his vision for general AI models that can solve all diseases, and the importance of developing agents that can learn to search through the whole potential design space. Hosted by Stephanie Zhan, Sequoia capital Mentioned in this episode : Playing Atari with Deep Reinforcement Learning : Seminal 2013 paper on Reinforcement Learning Capture the Flag : 2019 DeepMind paper on the emergence of cooperative agents AlphaStar : 2019 DeepMind paper on attaining grandmaster level in StarCraft II using multi-agent RL AlphaFold Server : Web interface for AlphaFold 3 model for non-commercial academic use…

Training Data

1
Pricing in the AI Era: From Inputs to Outcomes, with Paid CEO Manny Medina 45:29

לפני 14 weeks45:29

45:29

Former Outreach CEO Manny Medina discusses his new company Paid, which provides billing, pricing and margin management tools for AI companies. He explains why traditional SaaS pricing models don’t work for AI businesses, and breaks down emerging approaches like outcome-based and agent-based pricing. Manny shares why he believes focused AI applications targeting specific workflows will win over broad platforms, while emphasizing that AI companies need better tools to understand their unit economics and capture more value. Hosted by Pat Grady and Lauren Reeder, Sequoia Capital Mentioned in this episode: CPQ : Configure, Price, Quote Invent and Wander : Book by Jeff Bezos and Walter Isaacson Foundations of Statistical Natural Language Processing : 1999 book by Chris Manning and Hinrich Schütze that Manny cites as a piece of AI content every AI founder should read. (still in print, companion site here ) The fox and the hedgehog: Quandri XBOW HappyRobot Owl Crosby…

Training Data

1
Arc Institute's Patrick Hsu on Building an App Store for Biology with AI 58:11

לפני 15 weeks58:11

58:11

Patrick Hsu, co-founder of Arc Institute, discusses the opportunities for AI in biology beyond just drug development, and how Evo 2, their new biology foundation model, is enabling a broad ecosystem of applications. Evo 2 was trained on a vast dataset of genomic data to learn evolutionary patterns that would have taken years to find; as a result, the model can be used for applications from identifying mutations that cause disease to designing new molecular and even genome scale biological systems. Hosted by Josephine Chen and Pat Grady, Sequoia Capital Mentioned in this episode: Sequence modeling and design from molecular to genome scale with Evo : Public pre-print of original Evo paper Genome modeling and design across all domains of life with Evo 2 : Public pre-print of Evo 2 paper ClinVar : NIH database of the genes that are known to cause disease, and mutations in those genes causally associated with disease state Sequence Read Archive : Massive NIH database of gene sequencing data Machines of Loving Grace : Daria Amodei essay that Patrick cites on how AI could transform the world for the better Arc Virtual Cell Atlas : Arc’s first step toward assembling, curating and generating large-scale cellular data from AI-driven biological discovery (among many other tools ) Protein Data Bank (PDB): a global archive of 3D structural information of biomolecules used by DeepMind to train AlphaFold OpenAI Deep Research : The one AI app Patrick uses daily…

Training Data

1
Replit CEO Amjad Masad on 1 Billion Developers: A Better End State than AGI? 1:26:18

לפני 16 weeks1:26:18

1:26:18

Amjad Masad set out more than a decade ago to pursue the dream of unleashing 1B software creators around the world. With millions of Replit users pre-ChatGPT, that vision was already becoming a reality. Turbocharged by LLMs, the vision of enabling anyone to code—from 12-year-olds in India to knowledge workers in the U.S.—seems less and less radical. In this episode, Amjad explains how an explosion in the developer population could change the economy, society and more. He also discusses his early days programming in Jordan, his unique management approach and what AI will mean for the global economy. Hosted by David Cahn and Sonya Huang, Sequoia Capital Mentioned in this episode: On the Naturalness of Software : 2012 paper on applying NLP to code Attention Is All You Need : Seminal 2017 paper on transformers I Am a Strange Loop : 2007 follow up to Douglas Hofstadter’s 1979 classic Gödel, Escher, Bach that explores how self-referential systems can describe minds On Lisp : Paul Graham’s 1993 book on the original programming language of AI…

Training Data

1
Why CRM Needs an AI Revolution, with Day.ai Founder Christopher O’Donnell 1:11:05

לפני 17 weeks1:11:05

1:11:05

Christopher O’Donnell believes the fundamental problems with CRM—incomplete data, complex workflows, siloed work products and the fear of leads falling through the cracks—can finally be solved through AI. Founder of Day.ai and former Chief Product Officer of HubSpot, Christopher explains how his team is building a system that automatically captures the full context of customer relationships while giving users transparency and control. He shares lessons from building HubSpot’s CRM and why he’s taking a deliberate approach to product development despite the pressure to scale quickly in the AI era. Hosted by Pat Grady, Sequoia Capital Mentioned in this episode: The Innovator's Dilemma : Classic book by Clay Christensen (referenced regarding HubSpot's second S-curve strategy) Hubspot CRM : The only product to successfully challenge Salesforce’s dominance in the CRM category From Super Mario Brothers to Elden Ring : Analogy to what an AI-powered CRM experience can be through comparison of video games launched in 1985 vs 2022 Punk’d : Hidden camera–practical joke reality television series that premiered on MTV in 2003, created by Ashton Kutcher and Jason Goldberg Slow is smooth and smooth is fast : SEALs-derived concept mentioned regarding product development) Aga stove (highlighted as extraordinary product design example)…

Training Data

1
From Software Engineers to AI Word Artisans: Filip Kozera of Wordware 43:04

לפני 18 weeks43:04

43:04

Filip Kozera sees parallels between Excel’s democratization of data analytics and Wordware’s mission to put AI development in the hands of knowledge workers. Drawing inspiration from Excel’s 750 million users (compared to 30 million software developers), Wordware is creating tools that balance the rigid structure of programming with the fuzziness of natural language. Filip explains why effective AI development requires working across multiple abstraction layers—from high-level concepts to detailed implementation—while preserving human creative control. He shares his vision for “word artisans” who will use AI to amplify their creative impact. Hosted by Sonya Huang, Sequoia Capital Mentioned in this episode: Lovable : Generative AI app that builds UIs and web apps Her : 2013 Spike Jonze film that Filip uses as an example of how voice will not be the best modality to express knowledge work. Descript : AI video editing app that Filip uses a lot. Granola : AI notetaking app Filip uses every day.. Gemini 2.0 Pro : Google’s newest long context model that can handle 6000 page pdfs. Limitless pendant : Wearable device for collecting personal conversational context to drive AI experiences that Filip can’t wait for to ship. DeepLearning.AI : Andrew Ng’s amazing resource for learning about AI 3Blue1Brown : Grant Sanderson’s incredible channel on YouTube that explains math and AI visually.…

Training Data

1
Josh Woodward: Google Labs is Rapidly Building AI Products from 0-to-1 51:16

לפני 19 weeks51:16

51:16

As VP of Google Labs, Josh Woodward leads teams exploring the frontiers of AI applications. He shares insights on their rapid development process, why today’s written prompts will become outdated and how AI is transforming everything from video generation to computer control. He reveals that 25% of Google’s code is now written by AI and explains why coding could see major leaps forward this year. He emphasizes the importance of taste, design and human values in building AI tools that will shape how future generations work and create. Mentioned in this episode: Notebook LM : Personal research product based on Gemini 2 (previously discussed on Training Data .) Veo 2 : Google DeepMind’s new video generation model. Paul Graham on X replying to Aaron Levie’s post that “One approach to take in building in AI is to do something that's too expensive to be reasonably practical right now, and just bet that the costs will drop by 10X or 100X over time. The cost curve is on your side.” Where Good Ideas Come From : Book on the history of innovation by Steven Johnson. Project Mariner : Google DeepMind’s research prototype exploring human-agent interaction starting with browser use. Replit Agent : Josh’s favorite new AI app The Lego Story : Book on the history of Lego. Hosted by: Ravi Gupta and Sonya Huang, Sequoia Capital…

Training Data

1
How AI Breakout Harvey is Transforming Legal Services, with CEO Winston Weinberg 54:09

לפני 20 weeks54:09

54:09

Harvey CEO Winston Weinberg explains why success in legal AI requires more than just model capabilities—it demands deep process expertise that doesn’t exist online. He shares how Harvey balances rapid product development with earning trust from law firms through hyper-personalized demos and deep industry expertise. The discussion covers Harvey’s approach to product development—expanding specialized capabilities then collapsing them into unified workflows—and why focusing on complex work like international mergers creates the most defensible position in legal AI. Hosted by: Sonya Huang and Pat Grady, Sequoia Capital…

Training Data

1
The AI Product Going Viral With Doctors: OpenEvidence, with CEO Daniel Nadler 1:04:52

לפני 21 weeks1:04:52

1:04:52

OpenEvidence is transforming how doctors access medical knowledge at the point of care, from the biggest medical establishments to small practices serving rural communities. Founder Daniel Nadler explains his team’s insight that training smaller, specialized AI models on peer-reviewed literature outperforms large general models for medical applications. He discusses how making the platform freely available to all physicians led to widespread organic adoption and strategic partnerships with publishers like the New England Journal of Medicine. In an industry where organizations move glacially, 10-20% of all U.S. doctors began using OpenEvidence overnight to find information buried deep in the long tail of new medical studies, to validate edge cases and improve diagnoses. Nadler emphasizes the importance of accuracy and transparency in AI healthcare applications. Hosted by: Pat Grady, Sequoia Capital Mentioned in this episode: Do We Still Need Clinical Language Models? : Paper from OpenEvidence founders showing that small, specialized models outperformed large models for healthcare diagnostics Chinchilla paper : Seminal 2022 paper about scaling laws in large language models Understand : Ted Chiang sci-fi novella published in 1991…

Training Data

1
OpenAI’s Deep Research Team on Why Reinforcement Learning is the Future for AI Agents 32:45

לפני 22 weeks32:45

32:45

OpenAI’s Isa Fulford and Josh Tobin discuss how the company’s newest agent, Deep Research, represents a breakthrough in AI research capabilities by training models end-to-end rather than using hand-coded operational graphs. The product leads explain how high-quality training data and the o3 model’s reasoning abilities enable adaptable research strategies, and why OpenAI thinks Deep Research will capture a meaningful percentage of knowledge work. Key product decisions that build transparency and trust include citations and clarification flows. By compressing hours of work into minutes, Deep Research transforms what’s possible for many business and consumer use cases. Hosted by: Sonya Huang and Lauren Reeder, Sequoia Capital Mentioned in this episode: Yann Lecun’s Cake : An analogy Meta AI’s leader shared in his 2016 NIPS keynote…

Training Data

1
Palo Alto Networks’ Nikesh Arora: AI, Security and the New World Order 1:00:08

לפני 23 weeks1:00:08

1:00:08

Palo Alto Networks’s CEO Nikesh Arora dispels DeepSeek hype by detailing all of the guardrails enterprises need to have in place to give AI agents “arms and legs.” No matter the model, deploying applications for precision-use cases means superimposing better controls. Arora emphasizes that the real challenge isn’t just blocking threats but matching the accelerated pace of AI-powered attacks, requiring a fundamental shift from prevention-focused to real-time detection and response systems. CISOs are risk managers, but legacy companies competing with more risk-tolerant startups need to move quickly and embrace change. Hosted by: Sonya Huang and Pat Grady, Sequoia Capital Mentioned in this episode: Cortex XSIAM : Security operations and incident remediation platform from Palo Alto Networks…

Training Data

1
MongoDB’s Sahir Azam: Vector Databases and the Data Structure of AI 44:26

לפני 23 weeks44:26

44:26

MongoDB product leader Sahir Azam explains how vector databases have evolved from semantic search to become the essential memory and state layer for AI applications. He describes his view of how AI is transforming software development generally, and how combining vectors, graphs and traditional data structures enables high-quality retrieval needed for mission-critical enterprise AI use cases. Drawing from MongoDB's successful cloud transformation, Azam shares his vision for democratizing AI development by making sophisticated capabilities accessible to mainstream developers through integrated tools and abstractions. Hosted by: Sonya Huang and Pat Grady, Sequoia Capital Mentioned in this episode: Introducing ambient agents : Blog post by Langchain on a new UX pattern where AI agents can listen to an event stream and act on it Google Gemini Deep Research : Sahir enjoys its amazing product experience Perplexity : AI search app that Sahir admires for its product craft Snipd : AI powered podcast app Sahir likes…

Training Data

1
Roblox Studio Head Stef Corazza: Using AI to Empower Creators 54:46

לפני 25 weeks54:46

54:46

Stef Corazza leads generative AI development at Roblox after previously building Adobe’s 3D and AR platforms. His technical expertise, combined with Roblox’s unique relationship with its users, has led to the infusion of AI into its creation tools. Roblox has assembled the world’s largest multimodal dataset. Stef previews the Roblox Assistant and the company’s new 3D foundation model, while emphasizing the importance of maintaining positive experiences and civility on the platform. Mentioned in this episode: Driving Empire : A Roblox car racing game Stef particularly enjoys RDC : Roblox Developer Conference Ego.live : Roblox app to create and share synthetic worlds populated with human-like generative agents and simulated communities| PINNs : Physics Informed Neural Networks ControlNet : A model for controlling image diffusion by conditioning on an additional input image that Stef says can be used as a 2.5D approach to 3D generation. Neural rendering : A combination of deep learning with computer graphics principles developed by Nvidia in its RTX platform Hosted by: Konstantine Buhler and Sonya Huang, Sequoia Capital…

Training Data

1
ReflectionAI Founder Ioannis Antonoglou: From AlphaGo to AGI 52:29

לפני 26 weeks52:29

52:29

Ioannis Antonoglou, founding engineer at DeepMind and co-founder of ReflectionAI, has seen the triumphs of reinforcement learning firsthand. From AlphaGo to AlphaZero and MuZero, Ioannis has built the most powerful agents in the world. Ioannis breaks down key moments in AlphaGo's game against Lee Sodol (Moves 37 and 78), the importance of self-play and the impact of scale, reliability, planning and in-context learning as core factors that will unlock the next level of progress in AI. Hosted by: Stephanie Zhan and Sonya Huang, Sequoia Capital Mentioned in this episode: PPO : Proximal Policy Optimization algorithm developed by DeepMind in game environments. Also used by OpenAI for RLHF in ChatGPT. MuJoCo : Open source physics engine used to develop PPO Monte Carlo Tree Search : Heuristic search algorithm used in AlphaGo as well as video compression for YouTube and the self-driving system at Tesla AlphaZero : The DeepMind model that taught itself from scratch how to master the games of chess, shogi and Go MuZero : The DeepMind follow up to AlphaZero that mastered games without knowing the rules and able to plan winning strategies in unknown environments AlphaChem : Chemical Synthesis Planning with Tree Search and Deep Neural Network Policies DQN : Deep Q-Network, Introduced in 2013 paper, Playing Atari with Deep Reinforcement Learning AlphaFold : DeepMind model for predicting protein structures for which Demis Hassabis, John Jumper and David Baker won the 2024 Nobel Prize in Chemistry…

Training Data

1
Kumo’s Hema Raghavan: Turning Graph AI into ROI 52:06

לפני 27 weeks52:06

52:06

Hema Raghavan is co-founder of Kumo, a company that makes graph neural networks accessible to enterprises by connecting to their relational data stored in Snowflake and Databricks. Hema talks about how running GNNs on GPUs has led to breakthroughs in performance as well as the query language Kumo developed to help companies predict future data points. Although approachable for non-technical users, the product provides full control for data scientists who use Kumo to automate time-consuming feature engineering pipelines. Mentioned in this episode: Graph Neural Networks : Learning mechanism for data in graph format, the basis of the Kumo product Graph RAG : Popular extension of retrieval-augmented generation using GNNs LiGNN : Graph Neural Networks at LinkedIn paper KDD : Knowledge Discovery and Data Mining Conference Hosted by: Konstantine Buhler and Sonya Huang, Sequoia Capital…

Training Data

1
Databricks Founder Ion Stoica: Turning Academic Open Source into Startup Success 1:00:05

לפני 28 weeks1:00:05

1:00:05

Berkeley professor Ion Stoica, co-founder of Databricks and Anyscale, transformed the open source projects Spark and Ray into successful AI infrastructure companies. He talks about what mattered most for Databricks' success -- the focus on making Spark win and making Databricks the best place to run Spark. He highlights the importance of striking key partnerships -- the Microsoft partnership in particular that accelerated Databricks' growth and contributed to Spark's dominance among data scientists and AI engineers. He also shares his perspective on finding new problems to work on, which holds lessons for aspiring founders and builders: 1) building systems in new areas that, if widely adopted, put you in the best position to understand the new problem space, and 2) focusing on a problem that is more important tomorrow than today. Hosted by: Stephanie Zhan and Sonya Huang, Sequoia Capital Mentioned in this episode: Spark : The open source platform for data engineering that Databricks was originally based on. Ray : Open source framework to manage, executes and optimizes compute needs across AI workloads, now productized through Anyscale MosaicML : Generative AI startups founded by Naveen Rao that Databricks acquired in 2023. Unity Catalog : Data and AI governance solution from Databricks. CIB Berkeley : Multi-strategy hedge fund at UC Berkeley that commercializes research in the UC system. Hadoop : A long-time leading platform for large scale distributed computing. VLLM and Chatbot Arena : Two of Ion’s students’ projects that he wanted to highlight.…

Training Data

1
XBOW CEO and GitHub Copilot Creator Oege de Moor: Cracking the Code on Offensive Security With AI 51:37

לפני 33 weeks51:37

51:37

Oege de Moor, the creator of GitHub Copilot, discusses how XBOW’s AI offensive security system matches and even outperforms top human penetration testers, completing security assessments in minutes instead of days. The team’s speed and focus is transforming the niche market of pen testing with an always-on service-as-a-software platform. Oege describes how he is building a large and sustainable business while also creating a product that will “protect all the software in the free world.” XBOW shows how AI is essential for protecting software systems as the amount of AI-generated code increases along with the scale and sophistication of cyber threats. Hosted by: Konstantine Buhler and Sonya Huang, Sequoia Capital Mentioned in this episode: Semmle : Oege’s previous startup, a code analysis tool to secure software, acquired in 2019 by GitHub Nico Waisman : Head of security at XBOW, previously a researcher at Semmle The Bitter Lesson : Highly influential post by Richard Sutton HackerOne : Cybersecurity company that runs one of the largest bug bounty programs Suno : AI songwriting app that Oege loves Machines of Loving Grace : Essay by Anthropic founder, Dario Amodei…

Training Data

1
Ramp CEO Eric Glyman: Using AI to Build “Self-Driving Money” 38:48

לפני 34 weeks38:48

38:48

When ChatGPT ushered in a new paradigm of AI in everyday use, many companies attempted to adapt to the new paradigm by rushing to add chat interfaces to their products. Eric has a different take—he doesn’t think chatbots are the right form factor for everything. He thinks “zero-touch” automation that works invisibly in the background can be more valuable in many cases. He cites self-driving cars as an analogy—or in this case, “self-driving money.” Ramp is a new kind of finance management company for businesses, offering AI-powered financial tools to help companies handle spending and expense processes. We’ll hear why Eric thinks AI that you never see is one of the most powerful instruments for reducing time spent on drudgery and unlocking more time for meaningful work. Hosted by: Ravi Gupta and Sonya Huang, Sequoia Capital Mentioned in this episode: Paribus : Glyman’s previous company, acquired by Capital One in 2016 Karim Atiyeh : Cofounder and CTO at Ramp and Glyman’s cofounder at Paribus Devin : AI agent product from Cognition Labs and Glyman’s favorite AI app Hit Refresh : Book by Satya Nadella…

Training Data

1
Dust’s Gabriel Hubert and Stanislas Polu: Getting the Most From AI With Multiple Custom Agents 1:03:07

לפני 35 weeks1:03:07

1:03:07

Founded in early 2023 after spending years at Stripe and OpenAI, Gabriel Hubert and Stanislas Polu started Dust with the view that one model will not rule them all, and that multi-model integration will be key to getting the most value out of AI assistants. In this episode we’ll hear why they believe the proprietary data you have in silos will be key to unlocking the full power of AI, get their perspective on the evolving model landscape, and how AI can augment rather than replace human capabilities. Hosted by: Konstantine Buhler and Pat Grady, Sequoia Capital 00:00 - Introduction 02:16 - One model will not rule them all 07:15 - Reasoning breakthroughs 11:15 - Trends in AI models 13:32 - The future of the open source ecosystem 16:16 - Model quality and performance 21:44 - “No GPUs before PMF” 27:24 - Dust in action 37:40 - How do you find “the makers” 42:36 - The beliefs Dust lives by 50:03 - Keeping the human in the loop 52:33 - Second time founders 56:15 - Lightning round…

Training Data

1
Clay’s Kareem Amin on Building the Sales ‘System of Action’ with AI 51:38

לפני 36 weeks51:38

51:38

Clay is leveraging AI to help go-to-market teams unleash creativity and be more effective in their work, powering custom workflows for everything from targeted outreach to personalized landing pages. It’s one of the fastest growing AI-native applications, with over 4,500 customers and 100,000 users. Founder and CEO Kareem Amin describes Clay’s technology, and its approach to balancing imagination and automation in order to help its customers achieve new levels of go-to-market success. Hosted by: Alfred Lin, Sequoia Capital…

Training Data

1
Decart’s Dean Leitersdorf on AI-Generated Video Games and Worlds 46:34

לפני 37 weeks46:34

46:34

Can GenAI allow us to connect our imagination to what we see on our screens? Decart’s Dean Leitersdorf believes it can. In this episode, Dean Leitersdorf breaks down how Decart is pushing the boundaries of compute in order to create AI-generated consumer experiences, from fully playable video games to immersive worlds. From achieving real-time video inference on existing hardware to building a fully vertically integrated stack, Dean explains why solving fundamental limitations rather than specific problems could lead to the next trillion-dollar company. Hosted by: Sonya Huang and Shaun Maguire, Sequoia Capital 00:00 Introduction 03:22 About Oasis 05:25 Solving a problem vs overcoming a limitation 08:42 The role of game engines 11:15 How video real-time inference works 14:10 World model vs pixel representation 17:17 Vertical integration 34:20 Building a moat 41:35 The future of consumer entertainment 43:17 Rapid fire questions…

Training Data

1
How Glean CEO Arvind Jain Solved the Enterprise Search Problem – and What It Means for AI at Work 44:48

לפני 39 weeks44:48

44:48

Years before co-founding Glean, Arvind was an early Google employee who helped design the search algorithm. Today, Glean is building search and work assistants inside the enterprise, which is arguably an even harder problem. One of the reasons enterprise search is so difficult is that each individual at the company has different permissions and access to different documents and information, meaning that every search needs to be fully personalized. Solving this difficult ingestion and ranking problem also unlocks a key problem for AI: feeding the right context into LLMs to make them useful for your enterprise context. Arvind and his team are harnessing generative AI to synthesize, make connections, and turbo-change knowledge work. Hear Arvind’s vision for what kind of work we’ll do when work AI assistants reach their potential. Hosted by: Sonya Huang and Pat Grady, Sequoia Capital 00:00 - Introduction 08:35 - Search rankings 11:30 - Retrieval-Augmented Generation 15:52 - Where enterprise search meets RAG 19:13 - How is Glean changing work? 26:08 - Agentic reasoning 31:18 - Act 2: application platform 33:36 - Developers building on Glean 35:54 - 5 years into the future 38:48 - Advice for founders…

Training Data

1
OpenAI Researcher Dan Roberts on What Physics Can Teach Us About AI 41:42

לפני 40 weeks41:42

41:42

In recent years there’s been an influx of theoretical physicists into the leading AI labs. Do they have unique capabilities suited to studying large models or is it just herd behavior? To find out, we talked to our former AI Fellow (and now OpenAI researcher) Dan Roberts. Roberts, co-author of The Principles of Deep Learning Theory , is at the forefront of research that applies the tools of theoretical physics to another type of large complex system, deep neural networks. Dan believes that DLLs, and eventually LLMs, are interpretable in the same way a large collection of atoms is—at the system level. He also thinks that emphasis on scaling laws will balance with new ideas and architectures over time as scaling asymptotes economically. Hosted by: Sonya Huang and Pat Grady, Sequoia Capital Mentioned in this episode: The Principles of Deep Learning Theory : An Effective Theory Approach to Understanding Neural Networks , by Daniel A. Roberts, Sho Yaida, Boris Hanin Black Holes and the Intelligence Explosion : Extreme scenarios of AI focus on what is logically possible rather than what is physically possible. What does physics have to say about AI risk? Yang-Mills & The Mass Gap : An unsolved Millennium Prize problem AI Math Olympiad : Dan is on the prize committee…

Training Data

1
Google NotebookLM’s Raiza Martin and Jason Spielman on Creating Delightful AI Podcast Hosts and the Potential for Source-Grounded AI 32:07

לפני 41 weeks32:07

32:07

NotebookLM from Google Labs has become the breakout viral AI product of the year. The feature that catapulted it to viral fame is Audio Overview, which generates eerily realistic two-host podcast audio from any input you upload—written doc, audio or video file, or even a PDF. But to describe NotebookLM as a “podcast generator” is to vastly undersell it. The real magic of the product is in offering multi-modal dimensions to explore your own content in new ways—with context that’s surprisingly additive. 200-page training manuals become synthesized into digestible chapters, turned into a 10-minute podcast—or both—and shared with the sales team, just to cite one example. Raiza Martin and Jason Speilman join us to discuss how the magic happens, and what’s next for source-grounded AI. Hosted by: Sonya Huang and Pat Grady, Sequoia Capital…

Training Data

1
Snowflake CEO Sridhar Ramaswamy on Using Data to Create Simple, Reliable AI for Businesses 59:29

לפני 42 weeks59:29

59:29

All of us as consumers have felt the magic of ChatGPT—but also the occasional errors and hallucinations that make off-the-shelf language models problematic for business use cases with no tolerance for errors. Case in point: A model deployed to help create a summary for this episode stated that Sridhar Ramaswamy previously led PyTorch at Meta. He did not. He spent years running Google’s ads business and now serves as CEO of Snowflake, which he describes as the data cloud for the AI era. Ramaswamy discusses how smart systems design helped Snowflake create reliable "talk-to-your-data" applications with over 90% accuracy, compared to around 45% for out-of-the-box solutions using off the shelf LLMs. He describes Snowflake's commitment to making reliable AI simple for their customers, turning complex software engineering projects into straightforward tasks. Finally, he stresses that even as frontier models progress, there is significant value to be unlocked from current models by applying them more effectively across various domains. Hosted by: Sonya Huang and Pat Grady, Sequoia Capital Mentioned in this episode: Cortex Analyst : Snowflake’s talk-to-your-data API Document AI : Snowflake feature that extracts in structured information from documents…

Training Data

1
OpenAI's Noam Brown, Ilge Akkaya and Hunter Lightman on o1 and Teaching LLMs to Reason Better 45:22

לפני 43 weeks45:22

45:22

Combining LLMs with AlphaGo-style deep reinforcement learning has been a holy grail for many leading AI labs, and with o1 (aka Strawberry) we are seeing the most general merging of the two modes to date. o1 is admittedly better at math than essay writing, but it has already achieved SOTA on a number of math, coding and reasoning benchmarks. Deep RL legend and now OpenAI researcher Noam Brown and teammates Ilge Akkaya and Hunter Lightman discuss the ah-ha moments on the way to the release of o1, how it uses chains of thought and backtracking to think through problems, the discovery of strong test-time compute scaling laws and what to expect as the model gets better. Hosted by: Sonya Huang and Pat Grady, Sequoia Capital Mentioned in this episode: Learning to Reason with LLMs : Technical report accompanying the launch of OpenAI o1. Generator verifier gap : Concept Noam explains in terms of what kinds of problems benefit from more inference-time compute. Agent57: Outperforming the human Atari benchmark , 2020 paper where DeepMind demonstrated “the first deep reinforcement learning agent to obtain a score that is above the human baseline on all 57 Atari 2600 games.” Move 37 : Pivotal move in AlphaGo’s second game against Lee Sedol where it made a move so surprising that Sedol thought it must be a mistake, and only later discovered he had lost the game to a superhuman move. IOI competition : OpenAI entered o1 into the International Olympiad in Informatics and received a Silver Medal. System 1, System 2: The thesis if Danial Khaneman’s pivotal book of behavioral economics, Thinking, Fast and Slow , that positied two distinct modes of thought, with System 1 being fast and instinctive and System 2 being slow and rational. AlphaZero : The predecessor to AlphaGo which learned a variety of games completely from scratch through self-play. Interestingly, self-play doesn’t seem to have a role in o1. Solving Rubik’s Cube with a robot hand : Early OpenAI robotics paper that Ilge Akkaya worked on. The Last Question : Science fiction story by Isaac Asimov with interesting parallels to scaling inference-time compute. Strawberry: Why? O1-mini : A smaller, more efficient version of 1 for applications that require reasoning without broad world knowledge. 00:00 - Introduction 01:33 - Conviction in o1 04:24 - How o1 works 05:04 - What is reasoning? 07:02 - Lessons from gameplay 09:14 - Generation vs verification 10:31 - What is surprising about o1 so far 11:37 - The trough of disillusionment 14:03 - Applying deep RL 14:45 - o1’s AlphaGo moment? 17:38 - A-ha moments 21:10 - Why is o1 good at STEM? 24:10 - Capabilities vs usefulness 25:29 - Defining AGI 26:13 - The importance of reasoning 28:39 - Chain of thought 30:41 - Implication of inference-time scaling laws 35:10 - Bottlenecks to scaling test-time compute 38:46 - Biggest misunderstanding about o1? 41:13 - o1-mini 42:15 - How should founders think about o1?…

Training Data

1
Why Vlad Tenev and Tudor Achim of Harmonic Think AI Is About to Change Math—and Why It Matters 39:45

לפני 44 weeks39:45

39:45

Adding code to LLM training data is a known method of improving a model’s reasoning skills. But wouldn’t math, the basis of all reasoning, be even better? Up until recently, there just wasn’t enough usable data that describes mathematics to make this feasible. A few years ago, Vlad Tenev (also founder of Robinhood) and Tudor Achim noticed the rise of the community around an esoteric programming language called Lean that was gaining traction among mathematicians. The combination of that and the past decade’s rise of autoregressive models capable of fast, flexible learning made them think the time was now and they founded Harmonic. Their mission is both lofty—mathematical superintelligence—and imminently practical, verifying all safety-critical software. Hosted by: Sonya Huang and Pat Grady, Sequoia Capital Mentioned in this episode: IMO and the Millennium Prize : Two significant global competitions Harmonic hopes to win (soon) Riemann hypothesis : One of the most difficult unsolved math conjectures (and a Millenium Prize problem) most recently in the sights of MIT mathematician Larry Guth Terry Tao : perhaps the greatest living mathematician and Vlad’s professor at UCLA Lean : an open source functional language for code verification launched by Leonardo de Moura when at Microsoft Research in 2013 that powers the Lean Theorem Prover mathlib : the largest math textbook in the world, all written in Lean Metaculus : online prediction platform that tracks and scores thousands of forecasters Minecraft Beaten in 20 Seconds : The video Vlad references as an analogy to AI math Navier-Stokes equations : another important Millenium Prize math problem. Vlad considers this more tractable that Riemann John von Neumann : Hungarian mathematician and polymath that made foundational contributions to computing, the Manhattan Project and game theory Gottfried Wilhelm Leibniz : co-inventor of calculus and (remarkably) creator of the “universal characteristic,” a system for reasoning through a language of symbols and calculations—anticipating Lean and Harmonic by 350 years! 00:00 - Introduction 01:42 - Math is reasoning 06:16 - Studying with the world's greatest living mathematician 10:18 - What does the math community think of AI math? 15:11 - Recursive self-improvement 18:31 - What is Lean? 21:05 - Why now? 22:46 - Synthetic data is the fuel for the model 27:29 - How fast will your model get better? 29:45 - Exploring the frontiers of human knowledge 34:11 - Lightning round…

Training Data

1
Jim Fan on Nvidia’s Embodied AI Lab and Jensen Huang’s Prediction that All Robots will be Autonomous 49:13

לפני 45 weeks49:13

49:13

AI researcher Jim Fan has had a charmed career. He was OpenAI’s first intern before he did his PhD at Stanford with “godmother of AI,” Fei-Fei Li. He graduated into a research scientist position at Nvidia and now leads its Embodied AI “GEAR” group. The lab’s current work spans foundation models for humanoid robots to agents for virtual worlds. Jim describes a three-pronged data strategy for robotics, combining internet-scale data, simulation data and real world robot data. He believes that in the next few years it will be possible to create a “foundation agent” that can generalize across skills, embodiments and realities—both physical and virtual. He also supports Jensen Huang’s idea that “Everything that moves will eventually be autonomous.” Hosted by: Stephanie Zhan and Sonya Huang, Sequoia Capital Mentioned in this episode: World of Bits : Early OpenAI project Jim worked on as an intern with Andrej Karpathy. Part of a bigger initiative called Universe Fei-Fei Li : Jim’s PhD advisor at Stanford who founded the ImageNet project in 2010 that revolutionized the field of visual recognition, led the Stanford Vision Lab and just launched her own AI startup, World Labs Project GR00T : Nvidia’s “moonshot effort” at a robotic foundation model, premiered at this year’s GTC Thinking Fast and Slow : Influential book by Daniel Kahneman that popularized some of his teaching from behavioral economics Jetson Orin chip : The dedicated series of edge computing chips Nvidia is developing to power Project GR00T Eureka : Project by Jim’s team that trained a five finger robot hand to do pen spinning MineDojo: A project Jim did when he first got to Nvidia that developed a platform for general purpose agents in the game of Minecraft. Won NeurIPS 2022 Outstanding Paper Award ADI: artificial dog intelligence Mamba : Selective State Space Models, an alternative architecture to Transformers that Jim is interested in (original paper here ) 00:00 Introduction 01:35 Jim’s journey to embodied intelligence 04:53 The GEAR Group 07:32 Three kinds of data for robotics 10:32 A GPT-3 moment for robotics 16:05 Choosing the humanoid robot form factor 19:37 Specialized generalists 21:59 GR00T gets its own chip 23:35 Eureka and Issac Sim 25:23 Why now for robotics? 28:53 Exploring virtual worlds 36:28 Implications for games 39:13 Is the virtual world in service of the physical world? 42:10 Alternative architectures to Transformers 44:15 Lightning round…

Training Data

1
Founder Eric Steinberger on Magic’s Counterintuitive Approach to Pursuing AGI 51:15

לפני 46 weeks51:15

51:15

There’s a new archetype in Silicon Valley, the AI researcher turned founder. Instead of tinkering in a garage they write papers that earn them the right to collaborate with cutting-edge labs until they break out and start their own. This is the story of wunderkind Eric Steinberger, the founder and CEO of Magic.dev. Eric came to programming through his obsession with AI and caught the attention of DeepMind researchers as a high school student. In 2022 he realized that AGI was closer than he had previously thought and started Magic to automate the software engineering necessary to get there. Among his counterintuitive ideas are the need to train proprietary large models, that value will not accrue in the application layer and that the best agents will manage themselves. Eric also talks about Magic’s recent 100M token context window model and the HashHop eval they’re open sourcing. Hosted by: Sonya Huang, Sequoia Capital Mentioned in this episode: David Silver : DeepMind researcher that led the AlphaGo team Johannes Heinrich : a PhD student of Silver’s and DeepMind researcher who mentored Eric as a highschooler Reinforcement Learning from Self-Play in Imperfect-Information Games : Johannes’s dissertation that inspired Eric Noam Brown : DeepMind, Meta and now OpenAI reinforcement learning researcher who eventually collaborated with Eric and brought him to FAIR ClimateScience : NGO that Eric co-founded in 2019 while a university student Noam Shazeer : One of the original Transformers researchers at Google and founder of Charater.ai DeepStack: Expert-Level Artificial Intelligence in Heads-Up No-Limit Poker : the first AI paper Eric ever tried to deeply understand LTM-2-mini : Magic’s first 100M token context model, build using the HashHop eval (now available open source) 00:00 - Introduction 01:39 - Vienna-born wunderkind 04:56 - Working with Noam Brown 8:00 - “I can do two things. I cannot do three.” 10:37 - AGI to-do list 13:27 - Advice for young researchers 20:35 - Reading every paper voraciously 23:06 - The army of Noams 26:46 - The leaps still needed in research 29:59 - What is Magic? 36:12 - Competing against the 800-pound gorillas 38:21 - Ideal team size for researchers 40:10 - AI that feels like a colleague 44:30 - Lightning round 47:50 - Bonus round: 200M token context announcement…

Training Data

1
Crucible Moments Returns for S2: The ServiceNow Story ft. CEO Frank Slootman & Founder Fred Luddy 42:53

לפני 47 weeks42:53

42:53

On Training Data, we learn from innovators pushing forward the frontier of AI’s capabilities. Today we’re bringing you something different. It’s the story of a company currently implementing AI at scale in the enterprise, and how it was built from a bootstrapped idea in the pre-AI era to a 150 billion dollar market cap giant. It’s the Season 2 premiere of Sequoia’s other podcast, Crucible Moments, where we hear from the founders and leaders of some legendary companies about the crossroads and inflection points that shaped their journeys. In this episode, you’ll hear from Fred Luddy and Frank Slootman about building and scaling ServiceNow. Listen to Crucible Moments wherever you get your podcasts or go to: Spotify: https://open.spotify.com/show/40bWCUSan0boCn0GZJNpPn Apple: https://podcasts.apple.com/us/podcast/crucible-moments/id1705282398 Hosted by: Roelof Botha, Sequoia Capital Transcript: https://www.sequoiacap.com/podcast/crucible-moments-servicenow/…

ברוכים הבאים אל Player FM!

Player FM סורק את האינטרנט עבור פודקאסטים באיכות גבוהה בשבילכם כדי שתהנו מהם כרגע. זה יישום הפודקאסט הטוב ביותר והוא עובד על אנדרואיד, iPhone ואינטרנט. הירשמו לסנכרון מנויים במכשירים שונים.

תקשיבו ל-500+ נושאים

16 subscribers

דומה לTraining Data

Amazon eGift Card - Bright Balloons (Animated)

Amazon Basics Dog and Puppy Pee Pads, 5-Layer Leak-Proof Super Absorbent, Quick-Dry Surface, Potty Training, Regular (22x22"), 100 Count, Blue & White

Amazon Basics Multipurpose Copy Printer Paper, 8.5 x 11 Inches, 20 lb, 1 Ream, (500 Sheets), 92 Bright, White

פודקאסטים ששווה להאזין

Training Data « » MongoDB’s Sahir Azam: Vector Databases and the Data Structure of AI

MongoDB’s Sahir Azam: Vector Databases and the Data Structure of AI

פודקאסטים ששווה להאזין

ברוכים הבאים אל Player FM!

The Let Them Theory: A Life-Changing Tool That Millions of People Can't Stop Talking About

The Let Them Theory: A Life-Changing Tool That Millions of People Can't Stop Talking About

Bounty Paper Towels Quick Size, White, 16 Family Rolls = 40 Regular Rolls (Packaging May Vary)

Tubi: Watch Free Movies & TV Shows

The Let Them Theory: A Life-Changing Tool That Millions of People Can't Stop Talking About

דומה לTraining Data

מדריך עזר מהיר

Training Data « »
MongoDB’s Sahir Azam: Vector Databases and the Data Structure of AI