"Where I agree and disagree with Eliezer" by Paul Christiano

42:47
 
שתפו
 

Manage episode 332422118 series 3364760
על ידי LessWrong התגלה על ידי Player FM והקהילה שלנו - זכויות היוצרים שמורות למפרסם, לא ל-Player FM, והשמע מוזרם ישירות מהשרתים שלכם. הירשמו כדי לעקוב אחר עדכונים ב-Player FM, או הדביקו את כתובת העדכונים באפליקציות פודקאסט אחרות.

https://www.lesswrong.com/posts/CoZhXrhpQxpy9xw9y/where-i-agree-and-disagree-with-eliezer#fnh5ezxhd0an

by paulfchristiano, 20th Jun 2022.

Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.

(Partially in response to AGI Ruin: A list of Lethalities. Written in the same rambling style. Not exhaustive.)

Agreements
  1. Powerful AI systems have a good chance of deliberately and irreversibly disempowering humanity. This is a much easier failure mode than killing everyone with destructive physical technologies.
  2. Catastrophically risky AI systems could plausibly exist soon, and there likely won’t be a strong consensus about this fact until such systems pose a meaningful existential risk per year. There is not necessarily any “fire alarm.”
  3. Even if there were consensus about a risk from powerful AI systems, there is a good chance that the world would respond in a totally unproductive way. It’s wishful thinking to look at possible stories of doom and say “we wouldn’t let that happen;” humanity is fully capable of messing up even very basic challenges, especially if they are novel.

15 פרקים