works
Scott Alexander Heuristics That Almost Always Work online Simple heuristics that are correct a vast majority of the time, such as ‘a noise is just the wind’ or ‘a new technology won’t change everything,’ present a subtle challenge. Individuals, particularly those in expert roles, may come to rely on these highly successful shortcuts to the point where their judgment offers no additional value beyond what the heuristic itself provides. This reliance can be problematic because the rare instances where these heuristics fail are often of critical importance. When experts, tasked with identifying these rare exceptions, instead default to the simple heuristic, they become effectively replaceable by an inanimate object stating the heuristic. This can lead to a build-up of false confidence, as multiple experts citing the same underlying heuristic can create an illusion of corroborated, deeper evidence, rather than a shared reliance on a basic rule of thumb. Consequently, while these heuristics save effort and are often correct, their uncritical adoption by experts can undermine the very purpose of expertise and lead to significant failures when the low-probability, high-impact event occurs. – AI-generated abstract.

Heuristics That Almost Always Work

Scott Alexander

Astral Codex Ten, February 28, 2022

Abstract

Simple heuristics that are correct a vast majority of the time, such as ‘a noise is just the wind’ or ‘a new technology won’t change everything,’ present a subtle challenge. Individuals, particularly those in expert roles, may come to rely on these highly successful shortcuts to the point where their judgment offers no additional value beyond what the heuristic itself provides. This reliance can be problematic because the rare instances where these heuristics fail are often of critical importance. When experts, tasked with identifying these rare exceptions, instead default to the simple heuristic, they become effectively replaceable by an inanimate object stating the heuristic. This can lead to a build-up of false confidence, as multiple experts citing the same underlying heuristic can create an illusion of corroborated, deeper evidence, rather than a shared reliance on a basic rule of thumb. Consequently, while these heuristics save effort and are often correct, their uncritical adoption by experts can undermine the very purpose of expertise and lead to significant failures when the low-probability, high-impact event occurs. – AI-generated abstract.

PDF

First page of PDF