David Thorstad: Bounded Rationality and the Case Against Longtermism

The Gradient: Perspectives on AI

Episode 122

I spoke with Professor David Thorstad about:

* The practical difficulties of doing interdisciplinary work

* Why theories of human rationality should account for boundedness, heuristics, and other cognitive limitations

* why EA epistemics suck (ok, it’s a little more nuanced than that)

Professor Thorstad is an Assistant Professor of Philosophy at Vanderbilt University, a Senior Research Affiliate at the Global Priorities Institute at Oxford, and a Research Affiliate at the MINT Lab at Australian National University. One strand of his research asks how cognitively limited agents should decide what to do and believe. A second strand asks how altruists should use limited funds to do good effectively.

Reach me at [e-mail] for feedback, ideas, guest suggestions.

Subscribe to The Gradient Podcast: Apple Podcasts | Spotify | Pocket Casts | RSSFollow The Gradient on Twitter

Outline:

* (00:00) Intro

* (01:15) David’s interest in rationality

* (02:45) David’s crisis of confidence, models abstracted from psychology

* (05:00) Blending formal models with studies of the mind

* (06:25) Interaction between academic communities

* (08:24) Recognition of and incentives for interdisciplinary work

* (09:40) Movement towards interdisciplinary work

* (12:10) The Standard Picture of rationality

* (14:11) Why the Standard Picture was attractive

* (16:30) Violations of and rebellion against the Standard Picture

* (19:32) Mistakes made by critics of the Standard Picture

* (22:35) Other competing programs vs Standard Picture

* (26:27) Characterizing Bounded Rationality

* (27:00) A worry: faculties criticizing themselves

* (29:28) Self-improving critique and longtermism

* (30:25) Central claims in bounded rationality and controversies

* (32:33) Heuristics and formal theorizing

* (35:02) Violations of Standard Picture, vindicatory epistemology

* (37:03) The Reason Responsive Consequentialist View (RRCV)

* (38:30) Objective and subjective pictures

* (41:35) Reason responsiveness

* (43:37) There are no epistemic norms for inquiry

* (44:00) Norms vs reasons

* (45:15) Arguments against epistemic nihilism for belief

* (47:30) Norms and self-delusion

* (49:55) Difficulty of holding beliefs for pragmatic reasons

* (50:50) The Gibbardian picture, inquiry as an action

* (52:15) Thinking how to act and thinking how to live — the power of inquiry

* (53:55) Overthinking and conducting inquiry

* (56:30) Is thinking how to inquire as an all-things-considered matter?

* (58:00) Arguments for the RRCV

* (1:00:40) Deciding on minimal criteria for the view, stereotyping

* (1:02:15) Eliminating stereotypes from the theory

* (1:04:20) Theory construction in epistemology and moral intuition

* (1:08:20) Refusing theories for moral reasons and disciplinary boundaries

* (1:10:30) The argument from minimal criteria, evaluating against competing views

* (1:13:45) Comparing to other theories

* (1:15:00) The explanatory argument

* (1:17:53) Parfit and Railton, norms of friendship vs utility

* (1:20:00) Should you call out your friend for being a womanizer

* (1:22:00) Vindicatory Epistemology

* (1:23:05) Panglossianism and meliorative epistemology

* (1:24:42) Heuristics and recognition-driven investigation

* (1:26:33) Rational inquiry leading to irrational beliefs — metacognitive processing

* (1:29:08) Stakes of inquiry and costs of metacognitive processing

* (1:30:00) When agents are incoherent, focuses on inquiry

* (1:32:05) Indirect normative assessment and its consequences

* (1:37:47) Against the Singularity Hypothesis

* (1:39:00) Superintelligence and the ontological argument

* (1:41:50) Hardware growth and general intelligence growth, AGI definitions

* (1:43:55) Difficulties in arguing for hyperbolic growth

* (1:46:07) Chalmers and the proportionality argument

* (1:47:53) Arguments for/against diminishing growth, research productivity, Moore’s Law

* (1:50:08) On progress studies

* (1:52:40) Improving research productivity and technology growth

* (1:54:00) Mistakes in the moral mathematics of existential risk, longtermist epistemics

* (1:55:30) Cumulative and per-unit risk

* (1:57:37) Back and forth with longtermists, time of perils

* (1:59:05) Background risk — risks we can and can’t intervene on, total existential risk

* (2:00:56) The case for longtermism is inflated

* (2:01:40) Epistemic humility and longtermism

* (2:03:15) Knowledge production — reliable sources, blog posts vs peer review

* (2:04:50) Compounding potential errors in knowledge

* (2:06:38) Group deliberation dynamics, academic consensus

* (2:08:30) The scope of longtermism

* (2:08:30) Money in effective altruism and processes of inquiry

* (2:10:15) Swamping longtermist options

* (2:12:00) Washing out arguments and justified belief

* (2:13:50) The difficulty of long-term forecasting and interventions

* (2:15:50) Theory of change in the bounded rationality program

* (2:18:45) Outro

Links:

* David’s homepage and Twitter and blog

* Papers mentioned/read

* Bounded rationality and inquiry

* Why bounded rationality (in epistemology)?

* Against the newer evidentialists

* The accuracy-coherence tradeoff in cognition

* There are no epistemic norms of inquiry

* Permissive metaepistemology

* Global priorities and effective altruism

* What David likes about EA

* Against the singularity hypothesis (+ blog posts)

* Three mistakes in the moral mathematics of existential risk (+ blog posts)

* The scope of longtermism

* Epistemics

Get full access to The Gradient at thegradientpub.substack.com/subscribe