Facebook Built an AI System That Learned to Lie to Get What it Wants

Humans are natural negotiators. We arrange dozens of tiny little details throughout our day to produce a desired outcome: What time a meeting should start, when you can take time off work, or how many cookies you can take from the cookie jar.

Machines typically don’t share that affinity, but new research from Facebook’s AI research lab might offer a starting point to change that. The new system learned to negotiate from looking at each side of 5,808 human conversations, setting the groundwork for bots that could schedule meetings or get you the best deal online.

Facebook researchers used a game to help the bot learn how to haggle over books, hats, and basketballs. Each object had a point value, and they needed to be split between each bot negotiator via text.

Here’s how the bot works: After it sees what each item is worth, it begins generating a statement listing its demands, one word at a time. For instance, the bot would say, “I’d like all the books,” because the books are worth more points to it than hats or basketballs. Based on how the neural network has seen humans negotiate in the past, it comes up with a combination of words in a particular order that should return the greatest reward.

Then, after generating its own statement, it generates likely responses from its opponent, and what it would respond to those responses, and then on and on until the end of the conversation. The system does this after every exchange, slowly narrowing down to the optimal outcome. The researchers set the system to not accept getting nothing from the transaction, meaning it can’t walk away from the negotiating table. The bot has to haggle forever.

The pursuit of Facebook’s AI isn’t too different than other applications of AI, like the game Go. Each anticipates its opponent’s future actions and works to maximize its winnings. But unlike Google’s Go-playing AlphaGo, Facebook’s algorithm needs to make sense to humans while doing so.

From the human conversations (gathered via Amazon Mechanical Turk), and testing its skills against itself, the AI system didn’t only learn how to state its demands, but negotiation tactics as well—specifically, lying. Instead of outright saying what it wanted, sometimes the AI would feign interest in a worthless object, only to later concede it for something that it really wanted. Facebook isn’t sure whether it learned from the human hagglers or whether it stumbled upon the trick accidentally, but either way when the tactic worked, it was rewarded.

The Facebook team says they can also vary how hard the bot negotiates by changing how much it can vary its responses.

It’s no surprise that Facebook is working on ways to improve how its bot can interact with others—the company is highly invested in building bots that can negotiate on behalf of users and businesses for its Messenger platform, where it envisions the future of customer service. That dream, announced at Facebook’s 2016 F8 developer conference, has not come much closer to becoming reality, as bots today lack the ability to generate useful text or understand many user queries.

To accelerate this line of research, Facebook is open-sourcing all of its code and data from the research.