74 subscribers
התחל במצב לא מקוון עם האפליקציה Player FM !
Beyond Preference Alignment: Teaching AIs to Play Roles & Respect Norms, with Tan Zhi Xuan
Manage episode 452849963 series 3452589
In this episode of The Cognitive Revolution, Nathan explores groundbreaking perspectives on AI alignment with MIT PhD student Tan Zhi Xuan. We dive deep into Xuan's critique of preference-based AI alignment and their innovative proposal for role-based AI systems guided by social consensus. The conversation extends into their fascinating work on how AI agents can learn social norms through Bayesian rule induction. Join us for an intellectually stimulating discussion that bridges philosophical theory with practical implementation in AI development.
Check out:
"Beyond Preferences in AI Alignment" paper: https://arxiv.org/pdf/2408.16984
"Learning and Sustaining Shared Normative Systems via Bayesian Rule Induction in Markov Games" paper: https://arxiv.org/pdf/2402.13399
Help shape our show by taking our quick listener survey at https://bit.ly/TurpentinePulse
SPONSORS:
Notion: Notion offers powerful workflow and automation templates, perfect for streamlining processes and laying the groundwork for AI-driven automation. With Notion AI, you can search across thousands of documents from various platforms, generating highly relevant analysis and content tailored just for you - try it for free at https://notion.com/cognitiverevolution
Weights & Biases RAG++: Advanced training for building production-ready RAG applications. Learn from experts to overcome LLM challenges, evaluate systematically, and integrate advanced features. Includes free Cohere credits. Visit https://wandb.me/cr to start the RAG++ course today.
Oracle Cloud Infrastructure (OCI): Oracle's next-generation cloud platform delivers blazing-fast AI and ML performance with 50% less for compute and 80% less for outbound networking compared to other cloud providers13. OCI powers industry leaders with secure infrastructure and application development capabilities. New U.S. customers can get their cloud bill cut in half by switching to OCI before December 31, 2024 at https://oracle.com/cognitive
RECOMMENDED PODCAST:
Unpack Pricing - Dive into the dark arts of SaaS pricing with Metronome CEO Scott Woody and tech leaders. Learn how strategic pricing drives explosive revenue growth in today's biggest companies like Snowflake, Cockroach Labs, Dropbox and more.
Apple: https://podcasts.apple.com/us/podcast/id1765716600
Spotify: https://open.spotify.com/show/38DK3W1Fq1xxQalhDSueFg
CHAPTERS:
(00:00:00) Teaser
(00:01:09) About the Episode
(00:04:25) Guest Intro
(00:06:25) Xuan's Background
(00:12:03) AI Near-Term Outlook
(00:17:32) Sponsors: Notion | Weights & Biases RAG++
(00:20:18) Alignment Approaches
(00:26:11) Critiques of RLHF
(00:34:40) Sponsors: Oracle Cloud Infrastructure (OCI)
(00:35:50) Beyond Preferences
(00:40:27) Roles and AI Systems
(00:45:19) What AI Owes Us
(00:51:52) Drexler's AI Services
(01:01:08) Constitutional AI
(01:09:43) Technical Approach
(01:22:01) Norms and Deviations
(01:32:31) Norm Decay
(01:38:06) Self-Other Overlap
(01:44:05) Closing Thoughts
(01:54:23) Outro
SOCIAL LINKS:
Website: https://www.cognitiverevolution.ai
Twitter (Podcast): https://x.com/cogrev_podcast
Twitter (Nathan): https://x.com/labenz
LinkedIn: https://www.linkedin.com/in/nathanlabenz/
Youtube: https://www.youtube.com/@CognitiveRevolutionPodcast
Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk
221 פרקים
Beyond Preference Alignment: Teaching AIs to Play Roles & Respect Norms, with Tan Zhi Xuan
"The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis
Manage episode 452849963 series 3452589
In this episode of The Cognitive Revolution, Nathan explores groundbreaking perspectives on AI alignment with MIT PhD student Tan Zhi Xuan. We dive deep into Xuan's critique of preference-based AI alignment and their innovative proposal for role-based AI systems guided by social consensus. The conversation extends into their fascinating work on how AI agents can learn social norms through Bayesian rule induction. Join us for an intellectually stimulating discussion that bridges philosophical theory with practical implementation in AI development.
Check out:
"Beyond Preferences in AI Alignment" paper: https://arxiv.org/pdf/2408.16984
"Learning and Sustaining Shared Normative Systems via Bayesian Rule Induction in Markov Games" paper: https://arxiv.org/pdf/2402.13399
Help shape our show by taking our quick listener survey at https://bit.ly/TurpentinePulse
SPONSORS:
Notion: Notion offers powerful workflow and automation templates, perfect for streamlining processes and laying the groundwork for AI-driven automation. With Notion AI, you can search across thousands of documents from various platforms, generating highly relevant analysis and content tailored just for you - try it for free at https://notion.com/cognitiverevolution
Weights & Biases RAG++: Advanced training for building production-ready RAG applications. Learn from experts to overcome LLM challenges, evaluate systematically, and integrate advanced features. Includes free Cohere credits. Visit https://wandb.me/cr to start the RAG++ course today.
Oracle Cloud Infrastructure (OCI): Oracle's next-generation cloud platform delivers blazing-fast AI and ML performance with 50% less for compute and 80% less for outbound networking compared to other cloud providers13. OCI powers industry leaders with secure infrastructure and application development capabilities. New U.S. customers can get their cloud bill cut in half by switching to OCI before December 31, 2024 at https://oracle.com/cognitive
RECOMMENDED PODCAST:
Unpack Pricing - Dive into the dark arts of SaaS pricing with Metronome CEO Scott Woody and tech leaders. Learn how strategic pricing drives explosive revenue growth in today's biggest companies like Snowflake, Cockroach Labs, Dropbox and more.
Apple: https://podcasts.apple.com/us/podcast/id1765716600
Spotify: https://open.spotify.com/show/38DK3W1Fq1xxQalhDSueFg
CHAPTERS:
(00:00:00) Teaser
(00:01:09) About the Episode
(00:04:25) Guest Intro
(00:06:25) Xuan's Background
(00:12:03) AI Near-Term Outlook
(00:17:32) Sponsors: Notion | Weights & Biases RAG++
(00:20:18) Alignment Approaches
(00:26:11) Critiques of RLHF
(00:34:40) Sponsors: Oracle Cloud Infrastructure (OCI)
(00:35:50) Beyond Preferences
(00:40:27) Roles and AI Systems
(00:45:19) What AI Owes Us
(00:51:52) Drexler's AI Services
(01:01:08) Constitutional AI
(01:09:43) Technical Approach
(01:22:01) Norms and Deviations
(01:32:31) Norm Decay
(01:38:06) Self-Other Overlap
(01:44:05) Closing Thoughts
(01:54:23) Outro
SOCIAL LINKS:
Website: https://www.cognitiverevolution.ai
Twitter (Podcast): https://x.com/cogrev_podcast
Twitter (Nathan): https://x.com/labenz
LinkedIn: https://www.linkedin.com/in/nathanlabenz/
Youtube: https://www.youtube.com/@CognitiveRevolutionPodcast
Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk
221 פרקים
כל הפרקים
×
1 New Jersey’s AI Moonshot: Governor Phil Murphy on Partnerships, Progress, and Preparedness 55:54

1 Inference Scaling, Alignment Faking, Deal Making? Frontier Research with Ryan Greenblatt of Redwood Research 3:21:07

1 An Application-Free Future? Speaking Directly to Data with illumex CEO Inna Tokarev Sela 1:31:26

1 Claude Cooperates! Exploring Cultural Evolution in LLM Societies, with Aron Vallinder & Edward Hughes 1:32:52

1 Software Supernova: Lovable's "Superhuman Full Stack Engineer" to Transform Idea to App in Seconds 1:34:53

1 Software Supernova: Bolt.new - The AI Web App Developer In Your Browser 1:26:53

1 Gemini's Next Frontier: 2.0 Flash, Flash Lite Strategy & Real-Time APIs with Logan K from Google Deepmind 59:45

1 Distributed Training, Decentralized AI: Prime Intellect's Master Plan to Make AI Too Cheap to Meter 2:18:41

1 Aaron Levie, CEO of Box, on Box AI, Enterprise Enthusiasm, and the Evolution of SaaS 56:35

1 Emergency Pod: Reinforcement Learning Works! Reflecting on Chinese Reasoning Models DeepSeek-R1 and Kimi k1.5 1:47:53

1 Material Progress: Developing AI's Scientific Intuition, with Orbital Materials' Jonathan Godwin & Tim Duignan 1:40:50

1 Dodging Latent Space Detectors: Obfuscated Activation Attacks with Luke, Erik, and Scott. 2:10:23

1 Gene Hunting with o1-pro: Reasoning about Rare Diseases with ChatGPT Pro Grantee Dr. Catherine Brownstein 1:33:29

1 AI AMA – Part 2: AI Utopia, Consciousness, and the Future of Work 2:01:36

1 AI AMA – Part 1: OpenAI’s o3, Deliberative Alignment, and AI Surprises of 2024 2:06:57
ברוכים הבאים אל Player FM!
Player FM סורק את האינטרנט עבור פודקאסטים באיכות גבוהה בשבילכם כדי שתהנו מהם כרגע. זה יישום הפודקאסט הטוב ביותר והוא עובד על אנדרואיד, iPhone ואינטרנט. הירשמו לסנכרון מנויים במכשירים שונים.