reactors to release nuclear energy from atoms. Or it might embark on
space exploration to acquire more mineral resources.
4. A superintelligence will be creative; it will explore innovative methods
to meet its goals as fully as possible. From a human standpoint,
the machine might come up with some unexpected and disturbing
consequences. James Barrat gives a hypothetical example in which a
superintelligence programmed to protect humans decides to imprison
them in their homes for their own safety.
Of course, such a scenario might never happen. Nevertheless, the
example is a warning that superintelligence could lead to bizarre, totally
unanticipated results.
MIT physics professor Max Tegmark is worried about the potential
dangers of superintelligence. In April 2014, he hosted a gathering of thirty-
three scientists and others to talk about the hazards of the new, rapidly
progressing technologies. Their discussions led to the establishment of the
The United States and its allies use the MQ-9 Reaper, a military drone, to spy
and drop bombs on enemy targets. Human operators control the Reaper, but
someday drones equipped with advanced artificial intelligence might make their
own decisions about bombing operations, a scenario that frightens many.
80
Artificial Intelligence
