))
Deceptively Aligned Mesa-Optimizers: It’s Not Funny if I Have to Explain It
סדרה בארכיון ("עדכון לא פעיל" status)
When?
This feed was archived on February 21, 2025 21:08 (
Why? עדכון לא פעיל status. השרתים שלנו לא הצליחו לאחזר פודקאסט חוקי לזמן ממושך.
What now? You might be able to find a more up-to-date version using the search function. This series will no longer be checked for updates. If you believe this to be in error, please check if the publisher's feed link below is valid and contact support to request the feed be restored or if you have any other concerns about this.
Manage episode 424744812 series 3498845
תוכן מסופק על ידי BlueDot Impact. כל תוכן הפודקאסטים כולל פרקים, גרפיקה ותיאורי פודקאסטים מועלים ומסופקים ישירות על ידי BlueDot Impact או שותף פלטפורמת הפודקאסט שלהם. אם אתה מאמין שמישהו משתמש ביצירה שלך המוגנת בזכויות יוצרים ללא רשותך, אתה יכול לעקוב אחר התהליך המתואר כאן https://he.player.fm/legal.
Our goal here is to popularize obscure and hard-to-understand areas of AI alignment.
So let’s try to understand the incomprehensible meme!
Our main source will be Hubinger et al 2019, Risks From Learned Optimization In Advanced Machine Learning Systems.
Mesa- is a Greek prefix which means the opposite of meta-. To “go meta” is to go one level up; to “go mesa” is to go one level down (nobody has ever actually used this expression, sorry). So a mesa-optimizer is an optimizer one level down from you.
Consider evolution, optimizing the fitness of animals. For a long time, it did so very mechanically, inserting behaviors like “use this cell to detect light, then grow toward the light” or “if something has a red dot on its back, it might be a female of your species, you should mate with it”. As animals became more complicated, they started to do some of the work themselves. Evolution gave them drives, like hunger and lust, and the animals figured out ways to achieve those drives in their current situation. Evolution didn’t mechanically instill the behavior of opening my fridge and eating a Swiss Cheese slice. It instilled the hunger drive, and I figured out that the best way to satisfy it was to open my fridge and eat cheese.
Source:
https://astralcodexten.substack.com/p/deceptively-aligned-mesa-optimizers
Crossposted from the Astral Codex Ten podcast.
---
A podcast by BlueDot Impact.
Learn more on the AI Safety Fundamentals website.
85 פרקים
שתפו
