Paul Scharre, director of the Technology and National Security Program at the Center for a New American Security.
On Sept. 26, 1983, Lieutenant Colonel Stanislav Petrov was on duty in bunker Serpukhov-15 outside Moscow when sirens began blaring and a red backlit screen flashed a warning. The Soviet Union’s new Oko satellite-early warning system had detected what appeared to be an intercontinental ballistic missile launch from the United States. Then another. Then three more. It appeared that the Soviet Union was under nuclear attack. But Petrov was uncertain. A surprise attack by only five ICBMs made no sense. He called ground-based radar operators for confirmation. The ground radars detected nothing. Going with his gut, Petrov concluded that the new system had malfunctioned. He now didn’t launch a counter strike.
As it turned out, Petrov was right. Sunlight reflecting off cloud tops had triggered a false alarm. Thanks to one man’s intuition, nuclear war was averted, said Paul Scharre, author of “Army of None: Autonomous Weapons and the Future of War,” in a speech at Saint Stephen’s Episcopal church last night.
What would a machine have done in his place, Scharre asked. It would have done whatever it was programmed to do — even if it meant the destruction of the world.
Robots enhanced by artificial intelligence are coming on fast. We can see the technology in driverless cars, in the algorithms that drive stock market trading, and in weapons deployed by armed forces around the world. Some 16 countries now have armed drones. The Islamic State has weaponized machines that cost $300 retail. On a grander scale, the United States recently launched its first robotic submarine killer, which at $30 million a pop are a lot cheaper to build and operate than $2 billion destroyers.
There doesn’t seem to be any halting the rush toward using robots and AI to enhance our military, said Scharre. In the long run, they’re cheaper. Robots don’t need health care, they don’t ask for pensions, and their morale doesn’t suffer from continual overseas deployments. Also in war, as in finance, speed is crucial. Machines working on a faster decision-making curve will beat slower machines — and humans.
Computers can pound grand masters at chess and even beat humans at the infinitely more complex game of Go. Designers know how to program robots to abide by strict rules so they always obey the law. That’s great when situations are clear and predictable. But no one has figured out how to imbue technology with the kind of intuition that Stanislav Petrov displayed when he questioned the Oko system, Scharre said. No one has figured out how to imbue robots with the ability to make moral decisions in situations of conflict and ambiguity.
These moral issues are of more than remote interest to Virginians. Elected representatives such as U.S. Senator Mark Warner and former Governor Terry McAuliffe are enthusiastic proponents of developing autonomous vehicles and drones as an economic development strategy. The Old Dominion also is home to armed forces that will deploy AI-enhanced technologies — and home to some of the tech companies that write the code for Artificial Intelligence.
Despite misgivings, the U.S. has little choice but to pursue the technologies behind autonomous vehicles and weapons systems. Other countries are working on them, Scharre warns. China aspires to becoming the world leader in AI. If we fall behind in the race, we risk losing our military primacy.
Preserving the global balance of power, averting nuclear catastrophe, and probing the existential angst of robots are fine and dandy, but what about us? I mean, what’s the impact on Virginia? I asked Scharre, who resides in Northern Virginia, what an “Army of None” would mean for the military presence in the Old Dominion.
It turns out that “Army of None” makes a clever title for a book but is a bit of a misnomer. If there’s one thing that the U.S. Army and the Marines have learned from their long campaigns in Afghanistan and Iraq, it’s that they need manpower to fight insurgencies. While technology can make American soldiers more effective, they can’t substitute for boots on the ground.
Conversely, Scharre said, the Navy and Air Force are the branches of the armed services that have the most to gain from automation. The Navy doesn’t care how many men and women it takes to run a warship. Actually, the fewer the better — highly automated vessels mean less payroll, less pension cost, less deployment fatigue and more money for new weapons systems. By this logic, if the Navy aggressively adopts autonomous systems, the long-run outlook could be a much smaller (though higher tech) Navy payroll in Hampton Roads.
How about the tooth-to-tail ratio? The U.S. Defense Department famously has as many civilian employees today as combatants, the consequence of massive bureaucratization. Is there any hope, I asked, of replacing administrative drones with AI-enhanced drones? Scharre was noncommittal. It might be possible to use AI to improve the efficiency of business processes, just as it is in the private sector, he suggested. But don’t look for a massive displacement of excess admirals and generals, much less a wholesale riffing of mid-level functionaries.
Robots, it seems, do not yet constitute a constituency that can lobby for the advancement and preservation of their interests. Career employees in Northern Virginia’s defense bureaucracy are safe for now.
