Quotes

If a catastrophe should really dominate our thinking, it will not be because of the people it kills. There will be other harms, of course. But the effect that seems the most potentially harmful is the huge number of people whose existence might be prevented by a catastrophe. If we become extinct within the next few thousand years, that will prevent the existence of tens of trillions of people, as a very conservative estimate. If those nonexistences are bad, then this is a consideration that might dominate our calculations of expected utility.

John Broome, A small chance of disaster, European Review, vol. 21, no. S1, 2013, pp. S27—S31, p. 830

The main reason to focus on existential risk generally, and human extinction in particular, is that anything else about posthuman society can be modified by the posthumans (who will be far smarter and more knowledgeable than us) if desired, while extinction can obviously never be undone.

Alyssa Vance, Levels of action, LessWrong, April 14, 2011

One thing that greatly matters is the failure of we rich people to prevent, as we so easily could, much of the suffering and many of the early deaths of the poorest people in the world. The money that we spend on an evening’s entertainment might instead save some poor person from death, blindness, or chronic and severe pain. If we believe that, in our treatment of these poorest people, we are not acting wrongly, we are like those who believed that they were justified in having slaves.

Some of us ask how much of our wealth we rich people ought to give to these poorest people. But that question wrongly assumes that our wealth is ours to give. This wealth is legally ours. But these poorest people have much stronger moral claims to some of this wealth. We ought to transfer to these people […] at least ten per cent of what we inherit or earn.

What now matters most is how we respond to various risks to the survival of humanity. We are creating some of these risks, and we are discovering how we could respond to these and other risks. If we reduce these risks, and humanity survives the next few centuries, our descendants or successors could end these risks by spreading through this galaxy.

Life can be wonderful as well as terrible, and we shall increasingly have the power to make life good. Since human history may be only just beginning, we can expect that future humans, or supra-humans, may achieve some great goods that we cannot now even imagine. In Nietzsche’s words, there has never been such a new dawn and clear horizon, and such an open sea.

If we are the only rational beings in the Universe, as some recent evidence suggests, it matters even more whether we shall have descendants or successors during the billions of years in which that would be possible. Some of our successors might live lives and create worlds that, though failing to justify past suffering, would have given us all, including those who suffered most, reasons to be glad that the Universe exists.

Derek Parfit, On What Matters: Volume 3, Oxford, 2017, pp. 436-437

There have been great advances in the human condition due to science: recollect the horrors of childbirth, surgical operations, even of having a tooth out, a hundred years ago. If the human race is not extinguished there may be cures of cancer, senility, and other evils, so that happiness may outweigh unhappiness in the case of more and more individuals. Perhaps our far superior descendants of a million years hence (if they exist) will be possessed of a felicity unimaginable to us.

J. J. C. Smart, Ethics, persuasion and truth, London, 1984, p. 141

Is there anything we can do about animal suffering in wildlife? There was a time when many said that nothing should be done to obviate human suffering, since attempts to establish a welfare state would either be in vain, jeopardise what kind of welfare there happens to exist, or produce perverse (even worse) results. We rarely meet with that reaction any more. However, many seem to be ready to argue that wildlife constitutes such a complex system of ecological balances that any attempt to interfere must produce no good results, put into jeopardy whatever ecological ‘balances’ there happen to exist, or perversely make the situation even worse. This is not the place to settle whether they are right or not, but, certainly, there must exist some measures we could take, if we bothered to do so, rendering wildlife at least slightly less terrible. If this were so, we should do so, according to utilitarianism.

Torbjörn Tännsjö, Taking life: three theories on the ethics of killing, Oxford, 2015, pp. 260-261

Not every method of creating human-level intelligence is an extendible method. For example, the currently standard method of creating human-level intelligence is biological reproduction. But biological reproduction is not obviously extendible. If we have better sex, for example, it does not follow that our babies will be geniuses.

David J. Chalmers, The singularity: A philosophical analysis, Journal of consciousness studies, vol. 17, no. 9, 2010, pp. 7–65, p. 18

It is surprising and somewhat disappointing that movements aiming at the prevention of nuclear war are regarded throughout the West as Left-Wing movements or as inspired by some -ism which is repugnant to a majority of ordinary people. It is not in this way that opposition to nuclear warfare should be conceived. It should be conceived rather on the analogy of sanitary measures against epidemic.

Bertrand Russell, Common sense and nuclear warfare, London, 2010

Clearly, Eliezer should seriously consider devoting himself more to writing fiction. But it is not clear to me how this helps us overcome biases any more than any fictional moral dilemma. Since people are inconsistent but reluctant to admit that fact, their moral beliefs can be influenced by which moral dilemmas they consider in what order, especially when written by a good writer. I expect Eliezer chose his dilemmas in order to move readers toward his preferred moral beliefs, but why should I expect those are better moral beliefs than those of all the other authors of fictional moral dilemmas? If I’m going to read a literature that might influence my moral beliefs, I’d rather read professional philosophers and other academics making more explicit arguments. In general, I better trust explicit academic argument over implicit fictional “argument.”

Robin Hanson, Comment on 'The ethic of hand-washing and community epistemic practice', LessWrong, March 5, 2009

The threshold between a machine which was the intellectual inferior or superior of a man would probably be reached if the machine could do its own programming.

Irving John Good, Review of D. R. Hartree, Calculating Instruments and Machines, Journal of the Royal Statistical Society. Series A (General), vol. 114, no. 1, 1951, pp. 106–7, p. 107

Once a machine is designed that is good enough, […] it can be put to work designing an even better machine. At this point an “explosion” will clearly occur; all the problems of science and technology will be handed over to machines and it will no longer be necessary for people to work. Whether this will lead to a Utopia or to the extermination of the human race will depend on how the problem is handled by the machines. The important thing will be to give them the aim of serving human beings.

Irving John Good, Speculations on perceptrons and other automata, 1959, p. 17

Throughout human history, you see that the worst problems for people almost always come from other people[.]

Dario Maestripieri, quoted in 'Political Animals (Yes, Animals)' Natalie Angier, Political Animals (Yes, Animals), The New York Times, 2008

The evidence is clear and overwhelming that both the detection of deception and often its propagation have been major forces favoring the evolution of intelligence. It is perhaps ironic that dishonesty has often been the file against which intellectual tool for truth have been sharpened.

Robert Trivers, The folly of fools: The logic of deceit and self-deception in human life, New York, 2011, p. 5

Total and average utilitarianism are very different theories, and where they differ most is over extinction. If global warming extinguishes humanity, according to total utilitarianism, that would be an inconceivably bad disaster. The loss would be all the future wellbeing of all the people who would otherwise have lived. On the other hand, according to at least some versions of average utilitarianism, extinction might not be a very bad thing at all; it might not much affect the average wellbeing of the people who do live. So the difference between these theories makes a vast difference to the attitude we should take to global warming. According to total utilitarianism, although the chance of extinction is slight, the harm extinction would do is so enormous that it may well be the dominant consideration when we think about global warming. According to average utilitarianism, the chance of extinction may well be negligible.

John Broome, Counting the cost of global warming: A report to the Economic and Social Research Council on research by John Broome and David Ulph, London, 1992, p. 121

Good design resembles nature. It’s not so much that resembling nature is intrinsically good as that nature has had a long time to work on the problem. It’s a good sign when your answer resembles nature’s.

Paul Graham, Taste for Makers, February, 2002

There is good pain and bad pain. You want the kind of pain you get from going running, not the kind you get from stepping on a nail.

Paul Graham, Taste for Makers, February, 2002

I don’t think it works to cultivate strangeness. The best you can do is not squash it if it starts to appear. Einstein didn’t try to make relativity strange. He tried to make it true, and the truth turned out to be strange.

At an art school where I once studied, the students wanted most of all to develop a personal style. But if you just try to make good things, you’ll inevitably do it in a distinctive way, just as each person walks in a distinctive way. Michelangelo was not trying to paint like Michelangelo. He was just trying to paint well; he couldn’t help painting like Michelangelo.

The only style worth having is the one you can’t help. And this is especially true for strangeness. There is no shortcut to it. The Northwest Passage that the Mannerists, the Romantics, and two generations of American high school students have searched for does not seem to exist. The only way to get there is to go through good and come out the other side.

Paul Graham, Taste for Makers, February, 2002

Our policy of fixing bugs on the fly changed the relationship between customer support people and hackers. At most software companies, support people are underpaid human shields, and hackers are little copies of God the Father, creators of the world. Whatever the procedure for reporting bugs, it is likely to be one-directional: support people who hear about bugs fill out some form that eventually gets passed on (possibly via QA) to programmers, who put it on their list of things to do. It was very different at Viaweb. Within a minute of hearing about a bug from a customer, the support people could be standing next to a programmer hearing him say “Shit, you’re right, it’s a bug.” It delighted the support people to hear that “you’re right” from the hackers. They used to bring us bugs with the same expectant air as a cat bringing you a mouse it has just killed.

Paul Graham, The Other Road Ahead, September, 2001

Neoreactionaries are obsessed with taking down what Moldbug refers to as “the Cathedral”: a complex of Ivy League universities, the New York Times and other elite media institutions, Hollywood, and more that function to craft and mold public opinion so as to silence opposing viewpoints.

Park MacDougald, in an excellent piece on Nick Land’s brand of neoreaction, describes the Cathedral as a “media-academic mind-control apparatus.” I actually think the best analogy is to the role the patriarchy plays in radical feminist epistemology, or the role of “ideology” in Marxism. Neo-reaction demands a total rethinking of the way the world works, and such attempts generally only succeed if they can attack the sources of knowledge in society and offer a theory for why they’re systematically fallible.

That’s how feminist scholars have (I think correctly) undermined pseudoscientific attempts to paint female servility as natural, or male aggression and violence as inevitable and ultimately acceptable. Yes, the argument goes, these ideas have had elite supporters in the past, but those elites were tainted by institutional sexism. Similarly, Marxists are always alert to how media produced by big corporations can be tilted to serve those corporations’ class interests. The philosopher Paul Ricoeur once helpfully dubbed this kind of argument the “hermeneutics of suspicion.”

Neoreaction takes this approach and flips it on its head. No, it’s not institutional sexism or bourgeois class interest that’s perverting our knowledge base. It’s institutional progressivism, and fear of the revival of monarchism, tribalism, and prejudice.

That makes it a lot easier for neoreactionaries to defend their narrative of Western decline and democratic failure. If you look at the numbers, the Whig theory of history — with some faults and starts, everything’s getting better — appears to be basically right. Extreme poverty is at historic lows, hunger and infant mortality are plummeting, life expectancy is going up, war is on the decline, education is more available, homicide rates are down, etc.

But what if those numbers are all lies /produced by biased Cathedral sources in academia and propagated by Cathedral tools in the media like Vox? /What then?

Even if we think the prior existence view is more plausible than the total view, we should recognize that we could be mistaken about this and therefore give some value to the life of a possible future—let’s say, for example, 10 per cent of the value we give to the similar life of a presently existing being. The number of human beings who will come into existence only if we can avoid extinction is so huge that even with that relatively low value, reducing the risk of human extinction will often be a highly cost-effective strategy for maximizing utility, as long as we have some understanding of what will reduce that risk.

Katarzyna de Lazari-Radek and Peter Singer, The point of view of the universe: Sidgwick and contemporary ethics, Oxford, 2014, pp. 376-377

There may be a temptation to regard one life as trivial when compared with seven million. What difference will a choice of life or death for Smith make when compared with the millions who will surely die whatever you choose? Or perhaps we could say that it is not so much that one more life is trivial compared with several million, but rather than morality should not have anything to say about such a difference. Bernard Williams could be taken to be describing such a view when he talks of a moral agent for whom ’there are certain situations so monstruous that the idea that the processes of moral rationality could yield an answer in them is insane: they are situations which so transcend in enormity the human business of moral deliberation that from a moral point of view it cannot matter any more what happens’. Williams constrats such a view with consequentialism, which ‘will have something to say even on the difference between massacring seven million, and massacring seven million and one’. One can certainly sypmathize with the agent who is so horrified at the scale of a massacre that she fhinds it difficult to deliberate rationally in the circumstances. This does not, however, support the view that from a moral point of view it cannot matter anymore what happens. If there really is no moral difference between massacring seven million and massacring seven million and one, the allied soldier arriving at Auschwitz can have no moral reason for preventing the murder of one last Jew before the Nazi surrender. The Nazi himself can have no moral reason for refraining from one last murder. While Williams’s moral agent is berating the universe for transcending the bounds of rationality, the consequentialist is saving a life. It is not hard to guess which of these agents I would rather have on my side.

Alastair Norcross, Consequentialism and the unforeseeable future, Analysis (United Kingdom), vol. 50, no. 4, 1990, pp. 253–256, p. 255