DARPA Researchers Develop Sarcasm Detection Capability (Yeah, Right)

If you’ve ever been fooled by an Onion headline on Twitter, you’re not alone: Computational models aren’t great at detecting sarcasm either. 

That’s a problem in a world where the information ecosystem is becoming a contested space. But Defense Advanced Research Projects Agency researchers are getting closer to being able to understand the sentiment—including sarcasm—expressed in online messages, according to a recent announcement.

Sarcasm is hard for models to detect because it’s usually a performance dependent on facial expressions, gestures and tone of voice. But as part of DARPA’s Computational Simulation of Online Social Behavior, or SocialSim, program, a University of Central Florida team developed an artificial intelligence-enabled, deep-learning model that looks at online messages like Twitter posts for words that may be signaling satire. The model then creates a classification score indicating whether the message is sarcastic.

“Essentially, the researchers’ approach is focused on discovering patterns in the text that indicate sarcasm,” Brian Kettler, a program manager in DARPA’s Information Innovation Office, said in the press release. “It identifies cue-words and their relationship to other words that are representative of sarcastic expressions or statements.”

The model scored achieved a “nearly perfect sarcasm detection score” on a Twitter benchmark dataset, according to the press release, and also scored well on four other major datasets. One important characteristic of the model is that it’s easy to interpret, meaning it’s easier to explain where outputs come from. The capability is also language agnostic, according to the press release. 

DARPA is working on computational simulations of online behavior through the SocialSim program to foster a better understanding of how adversaries are using the global information environment. At a recent hearing before the House Armed Services Committee’s cyber and information systems subcommittee, Nina Jankowicz, a disinformation fellow at the Wilson Center, told lawmakers adversaries are engaging in perpetual information competition. 

“Adversaries understand information competition is the new normal and they are constantly probing for societal fissures to exploit,” Jankowicz said.