Deep Democracy as a promising target for positive AGI futures

Abstract

If you want the long-term future to go well by the lights of a certain value function, you might be tempted to try to align AGI(s) to your own values (broadly construed, including your deliberative values and intellectual temperaments).[1]Suppose that you’re not going to do that, for one of three reasons:You can’t. People more powerful than you are going to build AGIs and you don’t have a say over that.You object to aligning AGI(s) to your own values for principled reasons. It would be highly uncooperative, undemocratic, coercive, and basically cartoon supervillain evil.You recognize that this behaviour would, when pursued by lots of people, lead to a race to the bottom where everyone fights to build AGI aligned to their values as fast as possible and destroys a ton of value in the process, so you want to strongly reject this kind of norm.Then a good next-best option is Deep Democracy. What I mean by this is aligning AGI(s) to a process that is arbitrarily sensitive to every person’s entire value function. Not democracy in the sense of the current Western electoral system, but in the idealistic theoretical sense of deeply capturing and being responsive to every single person’s values. (Think about the ideal that democratic mechanisms like quadratic voting and bargaining theory are trying to capture, where democracy is basically equivalent to enlightened preference utilitarianism.)This is basically just the first class of Political Philosophy 101: it sure would be nice if you could install your favorite benevolent dictator, wouldn’t it? Well it turns out you can’t, and even if you could that’s evil and a very dangerous policy — what if someone else does this and you get a dictator you don’t like? As civilized people, let’s agree to give everyone a seat at the table to decide what happens.Deep Democracy has a lot of nice properties:It avoids the ascendence of an arbitrary dictator who decides the future.Suitably deep kinds of democracy avoid the tyranny of the majority, where if 51% of people say they want something, it happens. Instead decisions are sensitive to everyone’s values. This means that if you personally value something really weird, that doesn’t get stamped out by majority values, it still gets a place in the future.As a corollary, it makes outcomes sensitive to the number of people who care about something and how much they care about something.And it means that what you specifically care about will have some place in the long-term future, no matter what it is.It facilitates “moral hedging” — if everyone has a say, then everyone’s moral theories get a seat at the table in a real life moral parliament, hedging against both moral uncertainty and the possibility that a wrong moral theory wins and controls everything, destroying all value in the process.If value is a power law or similarly distributed, then you have a high chance of at least capturing some of the stuff that is astronomically more valuable than everything else, rather than losing out on this