Artificial intelligence learns teamwork in a deadly game of capture the flag

first_img By Edd GentMay. 30, 2019 , 2:00 PM DeepMind’s bots work in pairs to capture the opposing team’s flag on indoor and outdoor maps in Quake III Arena. Artificial intelligence learns teamwork in a deadly game of capture the flag Click to view the privacy policy. Required fields are indicated by an asterisk (*) Country * Afghanistan Aland Islands Albania Algeria Andorra Angola Anguilla Antarctica Antigua and Barbuda Argentina Armenia Aruba Australia Austria Azerbaijan Bahamas Bahrain Bangladesh Barbados Belarus Belgium Belize Benin Bermuda Bhutan Bolivia, Plurinational State of Bonaire, Sint Eustatius and Saba Bosnia and Herzegovina Botswana Bouvet Island Brazil British Indian Ocean Territory Brunei Darussalam Bulgaria Burkina Faso Burundi Cambodia Cameroon Canada Cape Verde Cayman Islands Central African Republic Chad Chile China Christmas Island Cocos (Keeling) Islands Colombia Comoros Congo Congo, the Democratic Republic of the Cook Islands Costa Rica Cote d’Ivoire Croatia Cuba Curaçao Cyprus Czech Republic Denmark Djibouti Dominica Dominican Republic Ecuador Egypt El Salvador Equatorial Guinea Eritrea Estonia Ethiopia Falkland Islands (Malvinas) Faroe Islands Fiji Finland France French Guiana French Polynesia French Southern Territories Gabon Gambia Georgia Germany Ghana Gibraltar Greece Greenland Grenada Guadeloupe Guatemala Guernsey Guinea Guinea-Bissau Guyana Haiti Heard Island and McDonald Islands Holy See (Vatican City State) Honduras Hungary Iceland India Indonesia Iran, Islamic Republic of Iraq Ireland Isle of Man Israel Italy Jamaica Japan Jersey Jordan Kazakhstan Kenya Kiribati Korea, Democratic People’s Republic of Korea, Republic of Kuwait Kyrgyzstan Lao People’s Democratic Republic Latvia Lebanon Lesotho Liberia Libyan Arab Jamahiriya Liechtenstein Lithuania Luxembourg Macao Macedonia, the former Yugoslav Republic of Madagascar Malawi Malaysia Maldives Mali Malta Martinique Mauritania Mauritius Mayotte Mexico Moldova, Republic of Monaco Mongolia Montenegro Montserrat Morocco Mozambique Myanmar Namibia Nauru Nepal Netherlands New Caledonia New Zealand Nicaragua Niger Nigeria Niue Norfolk Island Norway Oman Pakistan Palestine Panama Papua New Guinea Paraguay Peru Philippines Pitcairn Poland Portugal Qatar Reunion Romania Russian Federation Rwanda Saint Barthélemy Saint Helena, Ascension and Tristan da Cunha Saint Kitts and Nevis Saint Lucia Saint Martin (French part) Saint Pierre and Miquelon Saint Vincent and the Grenadines Samoa San Marino Sao Tome and Principe Saudi Arabia Senegal Serbia Seychelles Sierra Leone Singapore Sint Maarten (Dutch part) Slovakia Slovenia Solomon Islands Somalia South Africa South Georgia and the South Sandwich Islands South Sudan Spain Sri Lanka Sudan Suriname Svalbard and Jan Mayen Swaziland Sweden Switzerland Syrian Arab Republic Taiwan Tajikistan Tanzania, United Republic of Thailand Timor-Leste Togo Tokelau Tonga Trinidad and Tobago Tunisia Turkey Turkmenistan Turks and Caicos Islands Tuvalu Uganda Ukraine United Arab Emirates United Kingdom United States Uruguay Uzbekistan Vanuatu Venezuela, Bolivarian Republic of Vietnam Virgin Islands, British Wallis and Futuna Western Sahara Yemen Zambia Zimbabwe DeepMind Human gamers know just how hard it is to win a new spin on the classic computer game Quake: In a mazelike arena, they must work with other players to capture floating flags—all while dodging deadly gunfire. Now, for the first time, artificial intelligence (AI) has mastered teamwork in a complex first-person video game, coordinating its actions with both human and computer teammates to consistently beat opponents.“The scale of the experiments is remarkable,” says Michael Littman, an AI expert at Brown University. Getting AI agents to work together is incredibly tough, he says.Although AI can drive cars and easily defeat the world’s greatest chess and Go players one on one, researchers have struggled to get it to master teamwork. The practice may seem intuitive to us, but predicting how others will behave—a crucial component of working on a team—adds a whole new level of complexity and uncertainty for AI to deal with. Email Sign up for our daily newsletter Get more great content like this delivered right to you! Country In the new study, researchers got AI bots to teach each other to work as a team. Their classroom was a simplified version of 1999 first-person shooter, Quake III Arena. The game involves two teams that navigate around a 3D map to retrieve a flag from their opponent’s base and return it to theirs. The team with the most captures after 5 minutes wins. Players also fire a laser to tag enemies, sending them back to their home base.To train the AI to work as a team, the scientists created 30 different bots and pitted them against each other in a series of matches on randomly generated maps. The bots trained using brain-inspired algorithms called neural networks, which learn from data by altering the strength of connections between artificial neurons. The only data the bots had to learn from was the first-person visual perspective of their character and game points, awarded for things like picking up flags or tagging opponents. Initially the bots acted randomly. But when their actions scored points, the connections that led to the behavior were strengthened through a process called reinforcement learning. The training program also culled the bots that tended to lose and replaced them with mutated copies of top performers inspired by the way genetic variation and natural selection help animals evolve.After 450,000 games, the researchers arrived at the best bot, which they named For The Win (FTW). They then tested it in various matches with a mirror FTW, an FTW bot missing a crucial learning element, the game’s in-built bots, and humans. Teams of FTW bots consistently outperformed all other groups, though humans paired with FTW bots were able to beat them 5% of the time, they report today in Science.The FTW bots learned to play seamlessly with humans and machines, and they even developed classic cooperative strategies, says study co-leader Max Jaderberg, an AI researcher at Google-owned DeepMind in London. Those strategies included following teammates in order to outnumber opponents in later firefights and loitering near the enemy base when their teammate has the flag to immediately grab it when it reappears. In one test, the bots invented a completely novel strategy, exploiting a bug that let teammates give each other a speed boost by shooting them in the back.“What was amazing during the development of this project was seeing the emergence of some of these high-level behaviors,” Jaderberg says. “These are things we can relate to as human players.”The approach is still a long way from working in the real world, Jaderberg adds. But the advance is good for more than computer games. If AI can learn to work in teams, it could make everything from self-driving cars that avoid crashes by coordinating with each other to robotic surgical assistants that help out doctors during procedures.Still, Littman warns against extrapolating too much from a relatively simple computer simulation. “It could be that the details of this particular game require only a very narrow slice of what we think of as teamwork,” he says. And that, he says, means there’s no guarantee the same approach would teach AI to work as a team on other tasks.last_img

Recommended Reading

Discuss

Your email address will not be published. Required fields are marked *