התחל במצב לא מקוון עם האפליקציה Player FM !
פודקאסטים ששווה להאזין
בחסות


How Do Models Get Smarter? Pre-training, Fine-tuning, Long Context, Real-time Reasoning
Manage episode 453390459 series 3541344
In this illuminating discussion, hosts JC Bonilla and Ardis Kadiu break down the four fundamental ways AI models become smarter: pre-training (long-term memory), context/prompting (short-term memory), real-time reasoning (inference-time processing), and fine-tuning (specialized learning). Using real-world examples from Bloomberg GPT and Apple's strategy, they explain why bigger models aren't always better and how companies can achieve remarkable results by intelligently combining these different approaches to model intelligence. Kadiu provides a masterclass in understanding AI model development, challenging common assumptions about specialized models while explaining why current AI capabilities are sufficient for most applications over the next 4-5 years.
Post-Thanksgiving Welcome and Updates (00:00:07)
- Warm opening with hosts sharing Thanksgiving experiences
- Discussion of family gatherings and cooking adventures
- Setting the stage for a technical but accessible conversation
Understanding Model Intelligence: The Four Paths (00:29:06)
- Pre-training explained as "long-term memory" for models
- Context/prompting described as "short-term memory"
- Real-time reasoning capabilities during inference
- Fine-tuning as a specialized learning approach
- How these methods combine in practical applications
Pre-training Deep Dive (00:31:07)
- Explanation of the "P" in GPT (Generative Pre-trained Transformer)
- How pre-training works as foundational knowledge
- Cost implications of extensive pre-training
- Trade-offs between model size and performance
Context and Prompting Insights (00:32:44)
- Role of context in model performance
- How prompting provides short-term guidance
- Examples of effective context usage
- Impact on model accuracy and results
Real-time Reasoning Capabilities (00:34:06)
- How models perform inference-time reasoning
- Internal processing and decision-making
- Benefits of self-guided problem-solving
- Examples of reasoning in action
Fine-tuning and Specialization (00:36:16)
- When and why to use fine-tuning
- Cost benefits of specialized training
- Real-world examples of successful fine-tuning
- Limitations and considerations
Practical Applications and Cost Considerations (00:42:26)
- Analysis of decreasing model costs
- Speed vs accuracy trade-offs
- When to use which approach
- Future trends in model development
Industry Examples and Case Studies (00:47:20)
- Bloomberg GPT's lessons learned
- Apple's strategic approach to AI
- OpenAI's revenue model
- Success factors in model deployment
Looking Forward: The Next 4-5 Years (00:49:13)
- Current capabilities vs future needs
- Role of evaluation and testing
- Importance of proper tooling
- Balance between innovation and practical application
- - - -
Connect With Our Co-Hosts:
Ardis Kadiu
https://www.linkedin.com/in/ardis/
https://twitter.com/ardis
Dr. JC Bonilla
https://www.linkedin.com/in/jcbonilla/
https://twitter.com/jbonillx
About The Enrollify Podcast Network:
Generation AI is a part of the Enrollify Podcast Network. If you like this podcast, chances are you’ll like other Enrollify shows too!
Enrollify is made possible by Element451 — The AI Workforce Platform for Higher Ed. Learn more at element451.com.
98 פרקים
Manage episode 453390459 series 3541344
In this illuminating discussion, hosts JC Bonilla and Ardis Kadiu break down the four fundamental ways AI models become smarter: pre-training (long-term memory), context/prompting (short-term memory), real-time reasoning (inference-time processing), and fine-tuning (specialized learning). Using real-world examples from Bloomberg GPT and Apple's strategy, they explain why bigger models aren't always better and how companies can achieve remarkable results by intelligently combining these different approaches to model intelligence. Kadiu provides a masterclass in understanding AI model development, challenging common assumptions about specialized models while explaining why current AI capabilities are sufficient for most applications over the next 4-5 years.
Post-Thanksgiving Welcome and Updates (00:00:07)
- Warm opening with hosts sharing Thanksgiving experiences
- Discussion of family gatherings and cooking adventures
- Setting the stage for a technical but accessible conversation
Understanding Model Intelligence: The Four Paths (00:29:06)
- Pre-training explained as "long-term memory" for models
- Context/prompting described as "short-term memory"
- Real-time reasoning capabilities during inference
- Fine-tuning as a specialized learning approach
- How these methods combine in practical applications
Pre-training Deep Dive (00:31:07)
- Explanation of the "P" in GPT (Generative Pre-trained Transformer)
- How pre-training works as foundational knowledge
- Cost implications of extensive pre-training
- Trade-offs between model size and performance
Context and Prompting Insights (00:32:44)
- Role of context in model performance
- How prompting provides short-term guidance
- Examples of effective context usage
- Impact on model accuracy and results
Real-time Reasoning Capabilities (00:34:06)
- How models perform inference-time reasoning
- Internal processing and decision-making
- Benefits of self-guided problem-solving
- Examples of reasoning in action
Fine-tuning and Specialization (00:36:16)
- When and why to use fine-tuning
- Cost benefits of specialized training
- Real-world examples of successful fine-tuning
- Limitations and considerations
Practical Applications and Cost Considerations (00:42:26)
- Analysis of decreasing model costs
- Speed vs accuracy trade-offs
- When to use which approach
- Future trends in model development
Industry Examples and Case Studies (00:47:20)
- Bloomberg GPT's lessons learned
- Apple's strategic approach to AI
- OpenAI's revenue model
- Success factors in model deployment
Looking Forward: The Next 4-5 Years (00:49:13)
- Current capabilities vs future needs
- Role of evaluation and testing
- Importance of proper tooling
- Balance between innovation and practical application
- - - -
Connect With Our Co-Hosts:
Ardis Kadiu
https://www.linkedin.com/in/ardis/
https://twitter.com/ardis
Dr. JC Bonilla
https://www.linkedin.com/in/jcbonilla/
https://twitter.com/jbonillx
About The Enrollify Podcast Network:
Generation AI is a part of the Enrollify Podcast Network. If you like this podcast, chances are you’ll like other Enrollify shows too!
Enrollify is made possible by Element451 — The AI Workforce Platform for Higher Ed. Learn more at element451.com.
98 פרקים
כל הפרקים
×
1 State of AI Worldwide — consumer vs enterprise, pilots vs scale, culture and policy set the pace 45:55

1 GPT-5 review: what is it good for, reasoning on by default, hallucinations down, prompt rules change 46:53

1 ChatGPT Study Mode & Google's $1B Education Play: How AI Just Became Your Personal Tutor 50:08

1 Prompt engineering is dead, long live context engineering 1:01:29

1 America's AI Action Plan, AI wins gold at Math Olympiad, GPT-5 coming soon 43:15

1 The Great Content Collapse: How AI Agents Are Killing Clicks and Rewriting Marketing 56:38

1 Software 3.0 and the Future of Software Development 59:10

1 Ardis Closes an Era at Element451, Returns to Builder Mode as AI Reshapes Tech 44:47

1 From AI features to AI teammates, exponential vs incremental change, student satisfaction metrics, culture transformation 23:57

1 Engage Summit 2025 Preview Special 41:50

1 CEO insights on AI transformation, Speed as the new moat, OpenAI hits $10B, Meta buys Scale AI, Apple's AI struggles 41:32

1 AGI Timeline 2029: What Happens When AI Surpasses Human Intelligence? 1:08:06

1 Dear Class of 2025: You're Competing with AI for Your First Job 44:56

1 Windows goes agent-first, Google AI Mode lands, Veo 3 kills Hollywood, Ive + OpenAI $6.5B deal, Claude 4 hits 47:56

1 AI Didn't Break Education - It Exposed the Cracks 52:51

1 Model Context Protocol (MCP): Making AI Agents Talk to Your Data 31:07


1 EO on AI Education, Hollywood's AI Validation, and OpenAI's O3 Visual Reasoning Power 43:59

1 ASU+GSV Summit 2025 Recap - Learning at the Speed of Light 45:56

1 ASU's AI Strategy, Digital Workforce, and the End of Frankenstein Tech 28:54

1 ChatGPT image magic changes design forever, Gemini 2.5 raises the bar, MCP connects everything, Claude for Education brings AI to classrooms 44:54

1 AI & Admissions: Making Fast, Fair Decisions in Higher Ed 35:08

1 The State of Personal AI Agents: Manus, Deep Research, Perplexity, Grok, Lovable, Cursor, Lindy.ai, and the Future of Autonomous Workflows 37:10

1 Goodbye Chatbots, Hello Voice AI Agents 42:20

1 AI Trust, Eval Frameworks, and Why Data Quality Matters 41:30

1 FERPA & AI: What Higher Ed Needs to Know 31:46

1 AI Agents are Here: What this Means for Higher Education (Engage AI Summit Keynote) 1:11:51

1 Grok-3, Reasoning Models, and the Path to AI Agents 42:13

1 AGI Timeline: OpenAI's Bold Claims Meet Anthropic's Economic Index 47:33

1 AI's Sputnik moment + Deepseek, CSU partners with OpenAI, ChatGP Operator + Deep Research 43:26

1 Moving from Assistants to an AI Workforce 42:22


1 Building the Agentic AI Framework 1:02:48

1 AI Mastery in 2025: Skills, Tools & Practical Steps to Adoption for Higher Ed 45:51

1 12 Days of OpenAI - o1/o3 models, $200 Pro plan, Sora, Apple Intelligence, Projects + Canvas in ChatGPT, Search, Voice w/Video mode, 1-800-ChatGPT 48:26

1 AI Predictions and Trends for 2025 55:24

1 Element451 lands $175M, signals AI's key role in higher ed 34:03

1 How Do Models Get Smarter? Pre-training, Fine-tuning, Long Context, Real-time Reasoning 52:49

1 AI vs. Application Fraud: Protecting Admissions from Digital Deception 49:29

1 Beyond the Limits: How AI Models Are Redefining Capabilities 40:49

1 Post election AI agenda, Chegg's ChatGPT nightmare, Tech consolidation ahead 43:41

1 Gen AI Powered Search - How will ChatGPT Search and Perplexity change SEO? 40:30

1 JC Returns to Element451 + Higher Ed's AI Reality Check as 91% of CTOs Feel Unprepared 46:00

1 Catching Rockets, Self-Driving Taxis, and Supercomputers: Elon Musk's Vision 57:19

1 OpenAI's voice API, Notebook LM, Meta's AR glasses wow, Google's $120M AI education push ALT: From Voice Interfaces to Virtual Reality 38:32

1 Why AI Agents Are the Future of Software 51:05

1 State of AI Adoption in HigherEd: Insights from USG AI Summit 41:34

1 Synthetic Data: The Secret Weapon for Smarter AI 25:48

1 AI Reasoning: How Machines Think and Learn 40:04

1 OpenAI's o1 Model: A Leap Forward in AI Reasoning 53:46

1 Top 100 GenAI Apps: ChatGPT dominates, Perplexity rises, ByteDance pushes AI, dating apps get smarter 37:02

1 How Universities are Integrating AI Today | Why Personalization is the Key to Student Retention | Ethical Challenges in Higher Ed AI | How AI is Reshaping the Job Market for Graduate | Preparing for… 48:36

1 AI Agents in Action: Building Proactive AI Systems for Higher Ed (part 2) 41:30

1 AI Agents 101: The Building Blocks of Intelligent Student Engagement (part 1) 32:53

1 AI in Cybersecurity: Inside the CrowdStrike Crash 31:22

1 Navigating AI Vendor Selection in Higher Ed 34:50

1 Llama 3.1 Release, AI Avatars with AI Studio, Meta's Open Source Vision, Segment Anything 2 Model (SAM) Release 28:03

1 Creating Interactive Apps for Better Teaching with Claude 3.5 23:18

1 The 5 Levels of AI: From Chatbots to Organizational Intelligence 37:32

1 AI That Listens and Speaks: A Look at New Voice Models 23:08



1 Rethinking Ads and Monetization in the Age of AI Search 43:54

1 The Science Behind Viral Content: Cracking the Social Media Code 43:38

1 Copilot for Your PC, Gemini Everywhere, and the Death of SEO 41:09

1 OpenAI Launches GPT-4o: Faster, Cheaper, and Free for All 29:39



1 Custom GPTs are the New Productivity Hack for Enterprises 28:25

1 Rethinking University Websites: The Future of AI-Powered Knowledge Discovery 20:44

1 Meta's AI Strategy: From Open Source Models to Integrated Experiences 30:14

1 How ASU is Embracing AI to Lead the Way 20:58

1 Business Models for AI: Who Pays for the Revolution? 45:50

1 Leading with AI: Strategic Insights for Higher Ed CMOs 16:24

1 Embracing AI Without Fear: Preparing Students for a Tech-Driven Future 14:14


1 AI Assistants: Your New Team Members in Higher Ed 43:49

1 Microsoft's New Acquisition, GPT-5's Summer Debut, and NVIDIA's Game-Changer Blackwell 41:36

1 From SQL to AI: Transforming Your Technical Team for the Future 40:13

1 The Evolution of Dashboards: From Static to AI-Powered Insights 41:20

1 Beyond the Hype: A Deep Dive into the Inner Workings of Conversational AI Chatbots 54:48

1 Google's Image AI Bias, Stable Diffusion 3, New video models, LLMs (Llama3, Gemma, Mistral) 40:56

1 Predictive AI: The Art and Science of Student Engagement 50:28

1 OpenAI’s SORA Transforms Video Production and Google introduces Gemini 1.5 26:07

1 AI on Your Wrist, in Your Eyes, and On Your Mind: The Future of Wearables 32:21

1 AI Touchdown: The Future Showcased in Super Bowl LVIII Commercials 22:19

1 Deepfakes Unmasked: The Art, Science, and Ethics of Synthetic Media 40:55

1 From Bard to Gemini: Google's Next-Gen AI Leap Forward 11:31

1 The Future of Online Search: From Keywords to Conversations 34:49

1 How AI Rethinks Grades and Bias in Admissions: From GPA to GPT (Grade Point Trajectory) 29:28

1 The Battle for LLM Supremacy: Open Source vs. Closed Source 25:01

1 The App Store of AI: Rise of Custom Bots 31:14

1 AI and Journalism Clash: The New York Times vs. OpenAI Lawsuit and Its Implications 21:45

1 AI Toolkit Essentials: Key Tools and Skills for 2024 Success 56:12

1 AI in the Eyes of Gen Z: Digital Natives Reshaping the Future 39:28

1 AI in 2024: Bold Predictions and Emerging Trends 34:04

1 AI in 2023: A Game-Changing Year in Review 34:55

ברוכים הבאים אל Player FM!
Player FM סורק את האינטרנט עבור פודקאסטים באיכות גבוהה בשבילכם כדי שתהנו מהם כרגע. זה יישום הפודקאסט הטוב ביותר והוא עובד על אנדרואיד, iPhone ואינטרנט. הירשמו לסנכרון מנויים במכשירים שונים.