The world must act quickly to avert a future in which autonomous robots with artificial intelligence roam the battlefields killing humans, scientists and arms experts warned at an elite gathering in the Swiss Alps.
Rules must be agreed to prevent the development of such weapons, they said at a January 19-23 meeting of billionaires, scientists and political leaders in the snow-covered ski resort of Davos.
Angela Kane, the German UN High Representative for Disarmament Affairs from 2012-2015, said the world had been slow to take pre-emptive measures to protect humanity from the lethal technology.
"It may be too late," she told a debate in Davos.
"There are many countries and many representatives in the international community that really do not understand what is involved. This development is something that is limited to a certain number of advanced countries," Kane said.
The deployment of autonomous weapons would represent a dangerous new era in warfare, scientists said.
"We are not talking about drones, where a human pilot is controlling the drone," said Stuart Russell, professor of computer science at University of California, Berkeley.
"We are talking about autonomous weapons, which means that there is no one behind it. AI: artificial intelligence weapons," he told a forum in Davos. "Very precisely, weapons that can locate and attack targets without human intervention."
Robot chaos on battlefield
Russell said he did not foresee a day in which robots fight the wars for humans and at the end of the day one side says: "OK you won, so you can have all our women."
But some 1,000 science and technology chiefs including British physicist Stephen Hawking, said in an open letter last July that the development of weapons with a degree of autonomous decision-making capacity could be feasible within years, not decades.
They called for a ban on offensive autonomous weapons that are beyond meaningful human control, warning that the world risked sliding into an artificial intelligence arms race and raising alarm over the risks of such weapons falling into the hands of violent extremists.
"The question is can these machines follow the rules of war?" Russell said.
'Beyond comprehension'
How, for an example, could an autonomous weapon differentiate between civilians, soldiers, resistance fighters and rebels? How could it know that it should not kill a pilot who has ejected from a plane and is parachuting to the ground?
"I am against robots for ethical reasons but I do not believe ethical arguments will win the day. I believe strategic arguments will win the day," Russell said.
The United States had renounced biological weapons because of the risk that one day they could deployed by "almost anybody", he said. "I hope this will happen with robots."
Alan Winfield, professor of electronic engineering at the University of the West of England, warned that removing humans from battlefield decision-making would have grave consequences.
"It means that humans are deprived from moral responsibility," Winfield said.
Moreover, the reaction of the robots may be hard to predict, he said: "When you put a robot in a chaotic environment, it behaves chaotically."
Roger Carr, chairman of the British aerospace and defense group BAE, agreed.
"If you remove ethics and judgement and morality from human endeavor whether it is in peace or war, you will take humanity to another level which is beyond our comprehension," Carr warned.
"You equally cannot put something into the field that, if it malfunctions, can be very destructive with no control mechanism from a human. That is why the umbilical link, man to machine, is not only to decide when to deploy the weapon but it is also the ability to stop the process. Both are equally important."
