Chuyển sang chế độ ngoại tuyến với ứng dụng Player FM !
#76 – Joe Carlsmith on Scheming AI
Manage episode 406789446 series 2607952
Joe Carlsmith is a writer, researcher, and philosopher. He works as a senior research analyst at Open Philanthropy, where he focuses on existential risk from advanced artificial intelligence. He also writes independently about various topics in philosophy and futurism, and holds a doctorate in philosophy from the University of Oxford.
You can find links and a transcript at www.hearthisidea.com/episodes/carlsmith
In this episode we talked about a report Joe recently authored, titled ‘Scheming AIs: Will AIs fake alignment during training in order to get power?’. The report “examines whether advanced AIs that perform well in training will be doing so in order to gain power later”; a behaviour Carlsmith calls scheming.
We talk about:
- Distinguishing ways AI systems can be deceptive and misaligned
- Why powerful AI systems might acquire goals that go beyond what they’re trained to do, and how those goals could lead to scheming
- Why scheming goals might perform better (or worse) in training than less worrying goals
- The ‘counting argument’ for scheming AI
- Why goals that lead to scheming might be simpler than the goals we intend
- Things Joe is still confused about, and research project ideas
You can get in touch through our website or on Twitter. Consider leaving us an honest review wherever you're listening to this — it's the best free way to support the show. Thanks for listening!
86 tập
Manage episode 406789446 series 2607952
Joe Carlsmith is a writer, researcher, and philosopher. He works as a senior research analyst at Open Philanthropy, where he focuses on existential risk from advanced artificial intelligence. He also writes independently about various topics in philosophy and futurism, and holds a doctorate in philosophy from the University of Oxford.
You can find links and a transcript at www.hearthisidea.com/episodes/carlsmith
In this episode we talked about a report Joe recently authored, titled ‘Scheming AIs: Will AIs fake alignment during training in order to get power?’. The report “examines whether advanced AIs that perform well in training will be doing so in order to gain power later”; a behaviour Carlsmith calls scheming.
We talk about:
- Distinguishing ways AI systems can be deceptive and misaligned
- Why powerful AI systems might acquire goals that go beyond what they’re trained to do, and how those goals could lead to scheming
- Why scheming goals might perform better (or worse) in training than less worrying goals
- The ‘counting argument’ for scheming AI
- Why goals that lead to scheming might be simpler than the goals we intend
- Things Joe is still confused about, and research project ideas
You can get in touch through our website or on Twitter. Consider leaving us an honest review wherever you're listening to this — it's the best free way to support the show. Thanks for listening!
86 tập
Tüm bölümler
×Chào mừng bạn đến với Player FM!
Player FM đang quét trang web để tìm các podcast chất lượng cao cho bạn thưởng thức ngay bây giờ. Đây là ứng dụng podcast tốt nhất và hoạt động trên Android, iPhone và web. Đăng ký để đồng bộ các theo dõi trên tất cả thiết bị.