Red Ice News

The Future is the Past

Chess-playing computers may cause the robot apocalypse
New to Red Ice? Start Here!

Chess-playing computers may cause the robot apocalypse

Source: ca.news.yahoo.com


Sore-loser chess programs might be the end of us all...

In the movie The Terminator, we heard the human side of the story of Judgment Day, with the machines getting smart and seeing us as a threat. It wasn’t until the sequel that we heard the tale from the machine’s point of view, as it chose to start World War III to avoid having its plug pulled. This is all science fiction of course, but according to Steve Omohundro, a researcher in artificial intelligence, the vision of those sci-fi movies may not be so far off the mark.

In a recent paper, he pointed out that unless we’re very careful, some of the autonomous systems that we’re designing now could turn against us, and it’s the simplest systems that are potentially the most dangerous. It isn’t so much out of some kind of malice, but simply because if the system is designed to achieve maximum effectiveness, it might completely bypass any safety measures (like having a functional ’off’ switch) we put into place in order to reach that goal.

As Omohundro writes: "When roboticists are asked by nervous onlookers about safety, a common answer is ’We can always unplug it!’ But imagine this outcome from the chess robot’s point of view. A future in which it is unplugged is a future in which it cannot play or win any games of chess. This has very low utility and so expected utility maximization will cause the creation of the instrumental subgoal of preventing itself from being unplugged. If the system believes the roboticist will persist in trying to unplug it, it will be motivated to develop the subgoal of permanently stopping the roboticist. Because nothing in the simple chess utility function gives a negative weight to murder, the seemingly harmless chess robot will become a killer out of the drive for self-protection."

Of course, there’s Asimov’s Laws (although there’s four of them, not just three, and failures in some science fiction seem to be based on forgetting the ’zeroth’ law), but again, what if the program can simply re-write itself to bypass those limitations if they aren’t convenient for its purpose?

It looks like we need to be very careful going into the future, and perhaps take a tip from our science fiction ... if we’re going to design learning computers, the first thing to teach it should be the value of human life.

[...]

Read the full article at: news.yahoo.com



READ: Has mankind created a technical world that the mind cannot master or control?

Comments

We're Hiring

We are looking for a professional video editor, animator and graphics expert that can join us full time to work on our video productions.

Apply

Help Out

Sign up for a membership to support Red Ice. If you want to help advance our efforts further, please:

Donate

Tips

Send us a news tip or a
Guest suggestion

Send Tip

Related News

The Robot Apocalypse Is Nigh: Boston Dynamics Releases New Videos
The Robot Apocalypse Is Nigh: Boston Dynamics Releases New Videos
Google buys military robotics company Boston Dynamics, eighth robotics firm bought in past 6 months
Google buys military robotics company Boston Dynamics, eighth robotics firm bought in past 6 months

Archives Pick

Red Ice T-Shirts

Red Ice Radio

3Fourteen

Con Inc., J6 Political Prisoners & The Pedophile Problem
Kim Coulter - Con Inc., J6 Political Prisoners & The Pedophile Problem
Why European Culture, Art and Beauty Matter
Gifts - Why European Culture, Art and Beauty Matter

TV

Laughter: The New Weapon Of The Far-Right
Laughter: The New Weapon Of The Far-Right
London’s Fourth Plinth To Get Fat Black “Everywoman” Statue
London’s Fourth Plinth To Get Fat Black “Everywoman” Statue

RSSYoutubeGoogle+iTunesSoundCloudStitcherTuneIn

Design by Henrik Palmgren © Red Ice Privacy Policy