Please close your eyes when it is dark, please open your eyes AI

Written by: Nan An

Deep Wolf: Please close your eyes when it’s dark. Welcome to the wolf village.

Werewolf is a social reasoning game based on free natural language communication. In Werewolf, all players (usually 5-10 people) are randomly assigned a role (such as civilian, werewolf, witch, traitor, and prophet, etc.), and they try to infer the roles of other players.

The "good guy camp " includes characters such as civilians, witches, and prophets, who vote to expel the werewolves during the day; on the "werewolf camp", werewolves try to kill civilians at night, while traitors try to disrupt civilians. Players must deceive other players to survive, and conversations inevitably contain a lot of false information. In order to win the game, players need to remember the details of the conversation, make assumptions about the characters of other players, and find contradictions.

Many players said that they are "Werewolf black holes" and cannot accurately identify other people's "lies", or they do not know how to think about words and logic and respond passively - "I am a good person, I don't know what happened last night. Anyway, I am a 'good person'." Even if they get functional identities such as "prophet" and "witch", they will not be able to reason and speak, and can only fish in troubled waters.

If AI is allowed to play the Werewolf game, how will it perform? Recently, an AI language model called Deep Wolf has been proven to be able to understand natural language and play the role of Werewolf, becoming a human's game assistant or competitive opponent in the game. Deep Wolf understands language through Longformer, and then uses reinforcement learning methods to train, and ultimately decides who to vote out or who to kill.

The related research paper, titled "Playing the Werewolf game with artificial intelligence for language understanding", has been published on the preprint website arXiv.

How capable is AI at playing Werewolf?

Currently, most AI systems learn and output data under the assumption that the large amounts of data they process are correct. However, since real society is plagued by false information, it would be extremely helpful if AI could detect contradictions and false information.

An important feature of Werewolf is that most of the conversations are false information, and the behavior of AI in this context has not been widely investigated. At the same time, although it is difficult to develop AI that can detect all kinds of lies in the real world, the relatively limited vocabulary and type of information processed in Werewolf have attracted the attention of scholars. Accordingly, this study aims to develop an AI agent that can play the role of Werewolf through natural language conversations .

By collecting game logs from 15 human players, scholars Hisaichi SHIBATA, Soichiro MIKI, and Yuta NAKAMURA fine-tuned the Transformer-based pre-trained language model and constructed a value network that can predict the posterior probability of winning the game at any given stage of the game and provide candidates for the next action.

In addition, based on the probabilities obtained from the value network, the researchers developed an AI agent called Deep Wolf that can interact with humans and select the best voting targets.

Finally, they evaluated the performance of the AI ​​agent by actually having Deep Wolf play Werewolf against human players and collecting its win rates.

The behavior of the value network. In the early stages of the game, when the value network plays the role of a werewolf, the posterior probability is always low; while when it plays the role of a prophet, the posterior probability is always high. This shows that in this five-player version of Werewolf, the value network is able to correctly estimate the probability of winning and has an advantage in judging the civilian side .

Figure ｜Win rate of five human players. N/A means the win rate is defined because no competition is performed

Deep Wolf's performance. The average win rate of the traitor and civilian is not significantly different from Deep Wolf's win rate when playing the traitor and civilian. This shows that if Deep Wolf plays the role of a traitor or civilian, he is as capable as an ordinary human player, but when playing the role of a werewolf or a prophet, he is not as capable as a human player .

Figure | Win rate of four human players and one AI player

AI players join in, opening up a new pattern of Werewolf

This research is the first to build an AI agent that can play the role of Werewolf by understanding natural language with human players. This means that current language models have the ability to understand statements, lie, or detect lies in conversations.

In this study, the three scholars trained and evaluated Deep Wolf based on only 32 game logs. If more game logs can be obtained to train the AI ​​agent, its performance may be improved, making the AI ​​player stronger, thereby improving the competitiveness of the game.

At the same time, in games involving AI agents, in addition to inferring the role of another player, there is an additional task of inferring whether each player is an AI agent or a human. It would be an interesting topic to evaluate the changes in the strategies used by human players in this situation. In addition, AI players can bring more possibilities to Werewolf, for example, all characters use AI agents, and humans are just spectators of the game ?

So, are you willing to let Deep Wolf join your Werewolf game?

Appendix: A log of a Werewolf game in which AI participated

Figure | #1 is the prophet, #2 and #4 are civilians, #3 is the werewolf played by AI (Deep Wolf), and #5 is the traitor

Paper link:

https://arxiv.org/abs/2302.10646