The train to crazy town

Abstract

Sincere moral reasoning can lead to highly counterintuitive or extreme conclusions, raising concerns about the neglect of immediate human interests in favor of speculative causes and the potential justification of fanatical behaviors. These dual risks of moral swamping and extremist action can be mitigated by distinguishing ideal theory from imperfect practice. To address philosophical uncertainty and avoid swamping, individuals and institutions should adopt a strategy of worldview diversification rather than maximizing expected choiceworthiness. This approach distributes resources among subagents representing distinct, credible moral perspectives, thereby treating moral theories as guides for specific subagents rather than direct instructions for the primary agent. To prevent extremist behavior, agents must reject naive instrumentalism. Because human calculation is highly fallible, superficially plausible arguments for norm-breaking or harmful actions are likely counterproductive; thus, adhering to commonsense deontic constraints functions as a necessary, pragmatic heuristic to limit downside risk. Combined, these strategies provide asymmetric guardrails that preserve the benefits of abstract rational inquiry while enforcing the moderation required for safe, non-ideal practice. – AI-generated abstract.