The primary error, as Russell sees it, is to persist in building objectives into a machine and to designing algorithms to keep it progressing measurably toward the objective. ' To this Russell replied that 'Improving decision quality, irrespective of the utility function chosen, has been the goal of AI research – the mainstream goal on which we now spend billions per year,' and that 'A highly capable decision maker can have an irreversible impact on humanity.' In other words, the errors in AI design can be highly consequential, even catastrophic. The last 1/3 explains Russell's proposed solution and why he thinks it could work.Įrrors : Jaron Lanier wrote in 2014 that talking about such catastrophe scenarios ' is a way of avoiding the profoundly uncomfortable political problem, which is that if there's some actuator that can do harm, we have to figure out some way that people don't do harm with it. The first 2/3 of the book are devoted to presenting the problem, often in the apocalyptic imagery that has become familiar in the AI literature. Instead they should be designed to optimize our objectives, but even we have a hard time figuring what these are fortunately, Russell has a theory, one that will require us “to tear out and replace part of the foundations' of AI, but that could lead to a healthy relation between humans and machines. The problem with existing models of AI is that they are based on programming to optimize the machine's objectives. This will have profound implications for human life. We have to understand where we went wrong and then fix it.Īnd here is a more leisurely version: AI is coming that will soon - sooner than most of us think - be making decisions, more quickly than we can fathom. …it seems that the march towards superhuman intelligence is unstoppable, but success might be the undoing of the human race. Russell does roughly that himself, in his first chapter: Its thrust, though, can be summarized in four sentences. The book is dense in parts, and it contains as useful an introduction to the state of the art in AI, as of 2019, as any I've seen. Stuart Russell's contribution to the vast and growing popular literature on AI was widely reviewed when it came out, just before the pandemic its significance is such that it has its own Wikipedia page, like The Divine Comedy - which can also be profitably read as a theory of humans. This must be doubly true of a book with the word 'human' in the title. We will need to become good at being human.Īny book that aims to explain or to make predictions about artificial intelligence is necessarily in thrall to a theory of intelligence, and thus nearly always to a theory of humans as well. 1 Perhaps future generations will wonder why we ever worried about such a futile thing as “work.” … if we can no longer supply routine physical labor and routine mental labor, we can still supply our humanity.

Human compatible stuart russell