Philosopher Nick Bostrom recently posted a paper, where he postulated that a small chance of AI annihilating all humans might be worth the risk, because advanced AI might relieve humanity of “its universal death sentence.” That upbeat gamble is quite a leap from his previous dark musings on AI, which made him a doomer godfather. His 2014 book Superintelligence was an early examination of AI’s existential risk. One memorable thought experiment: An AI tasked with making paper clips winds up destroying humanity because all those resource-needy people are an impediment to paper clip production. His more recent book, Deep Utopia, reflects a shift in his focus. Bostrom, who leads Oxford’s Future of Humanity Institute, dwells on the “solved world” that comes if we get AI right.
STEVEN LEVY: Deep Utopia is more optimistic than your previous book. What changed for you?
NICK BOSTROM: I call myself a fretful optimist. I am very excited about the potential for radically improving human life and unlocking possibilities for our civilization. That’s consistent with the real possibility of things going wrong.
You wrote a paper with a striking argument: Since we’re all going to die anyway, the worst that can happen with AI is that we die sooner. But if AI works out, it might extend our lives, maybe indefinitely.
That paper explicitly looks at only one aspect of this. In any given academic paper, you can’t address life, the universe, and the meaning of everything. So let’s just look at this little issue and try to nail that down.
That isn’t a little issue.
I guess I’ve been irked by some of the arguments made by doomers who say that if you build AI, you’re going to kill me and my children and how dare you. Like the recent book If Anyone Builds It, Everyone Dies. Even more probable is that if nobody builds it, everyone dies! That’s been the experience for the last several 100,000 years.
But in the doomer scenario everybody dies and there’s no more people being born. Big difference.
I have obviously been very concerned with that. But in this paper, I’m looking at a different question, which is, what would be best for the currently existing human population like you and me and our families and the people in Bangladesh? It does seem like our life expectancy would go up if we develop AI, even if it is quite risky.
In Deep Utopia you speculate that AI could create incredible abundance, so much that humanity might have a huge problem with finding purpose. I live in the United States. We’re a very rich country, but our government, ostensibly with support of the people, has policies that deny services to the poor and distribute rewards to the rich. I think that even if AI was able to provide abundance for everyone, we would not supply it to everyone.
You might be right. Deep Utopia takes as its starting point the postulation that everything goes extremely well. If we do a reasonably good job on governance, everybody gets a share. There is quite a deep philosophical question of what a good human life would look like under these ideal circumstances.
The meaning of life is something you hear a lot about in Woody Allen movies and maybe in the philosophers community. I’m worried more about the wherewithal to support oneself and get a stake in this abundance.
The book is not only about meaning. That’s one out of a bunch of different values that it considers. This could be a wonderful emancipation from the drudgery that humans have been subjected to. If you have to give up, say, half of your waking hours as an adult just to make ends meet, doing some work you don’t enjoy and that you don’t believe in, that’s a sad condition. Society is so used to it that we’ve invented all kinds of rationalizations around it. It’s like a partial form of slavery.
