Your Phone Knows What You Watch on TV

The TV is on in the background, and you’re replying to a quick email on your phone nearby. You don’t know it, but the devices are communicating. During a commercial, the TV emits an inaudible tone and your phone, which was listening for it, picks it up. Somewhere far away, a server makes a note: Both devices probably belong to you.

This information about which devices belong to whom is immensely valuable to advertisers hoping to target ads specifically to you. In a simpler time, targeted marketing was easy. Most people had a computer at work and maybe another at home. If you sent an email about your new cat, ads for cat food started cropping up. If you searched for Thanksgiving recipes, Safeway coupons for turkeys appeared in your Facebook newsfeed.

Those were good days for advertisers tracking Internet users. It wasn’t so hard to find what people were up to online, because most routinely used just one or two connected devices.

But now, between laptops, phones, tablets, wearables, and Internet-enabled cars and TVs, advertisers have access to more information than ever before for ad targeting. They just need to figure out which devices live under the same roof.

That’s harder than it sounds. Unless you’re logged into a service on all your devices—for example, by using Google’s various services everywhere—advertisers need to get creative to stitch together a portrait of you.

Verizon’s “supercookies”—a snippet of code injected into mobile users’ web requests—silently identify and track its customers, sharing the information with AOL’s wide-reaching ad network. Vizio Smart TVs tie customers’ viewing habits to a home Internet address and sell the information to advertisers. And both programs require customers, who are often unaware of the programs, to opt out of them if they don’t wish to be tracked.

But a newer method of cross-device tracking wanders into the realm of science fiction. According to a filing from the Center from Democracy and Technology, a digital human rights and privacy advocacy organization, companies have figured out how to use inaudible sounds to establish links between devices.

Here’s how software from SilverPush, a leading provider of “audio beacons,” works: When you visit a website that uses SilverPush tracking technology, the site causes your device to emit an inaudible ultrasonic sound. If any other devices you’ve got lying around—a laptop, a phone, a tablet—has an app installed that includes SilverPush code, it’s listening for that sound. If it hears it, SilverPush knows that the two devices are close to one another and, presumably, belong to the same person.

More recently, SilverPush expanded into television advertising: Certain TV commercials include an ultrasonic audio beacon. Any nearby devices running SilverPush software will be listening for the beacon—if a device hears it, it records the match, allowing the company to figure out what ads users watch and for how long, and add that information to the user’s profile.

SilverPush product manager Piyush Bhatt says the audio tracking only operates in India, but the company, which is based near New Delhi, has established offices in San Francisco and the Philippines as well.

The creativity displayed by SilverPush and its peers has raised privacy concerns. The company’s audio-based tracking method is far more accurate than most ways advertising software quietly follow Internet users, because it takes the guesswork out of the device-matching process, says Joseph Lorenzo Hall, the chief technologist at the Center for Democracy and Technology.

The methods marketing companies use to discover links between users’ devices—whether it’s audio beacons or examining Internet traffic for markers that distinguish a certain user’s computer and browser—are often silent and secret by design.

(Bhatt said that the smartphone apps that listen for SilverPush audio beacons present the user with a screen asking for detailed permission before using the device’s microphone for the first time.)

When it comes to audio beacons, Hall says he’s not just worried about marketing companies tracking average consumers—he thinks their techniques could be used by governments for surveillance. For example, if a group of dissidents in a country like China were to meet in secret, their phones might pick up a government-planted ultrasonic audio signal from a nearby TV. If hidden code on their phones relayed to the government that they all heard the tone, agents could easily tell that the dissidents were associated with one another, and meeting surreptitiously.

To address these privacy questions, SilverPush is in contact with the Center for Democracy and Technology, and Bhatt says his company is “actively engaging with all the stakeholders.”

“We are concerned about privacy and would not do something that intrudes ethical data practices,” Bhatt said.

Cross-device tracking has caught the eye of the Federal Trade Commission, which recently convened a workshop to discuss the technologies and associated risks. Hall said he hopes the FTC will issue guidelines for building transparency and control into cross-device tracking technologies. Next-generation tracking technologies like SilverPush, he says, should be required to meet a “higher bar” of warning users they’re being followed.

As new tracking technologies take hold, so too will technologies to block them. Experts are already hard at work searching for audio beacons and trying to identify their characteristics, laying the groundwork for ways to limit the reach of the ultrasonic signals that may one day populate our airwaves.

(Image via Myvector/Shutterstock.com)