How Ad Targeting Works: Your Phone Isn't Listening to You, Your Metadata Is

Manos Tsagkias

11 January 2026
Keywords: outreach

A growing number of people believe their phones listen to their conversations and share what they say or do with advertisers. This belief is usually used to explain why they see highly personalized ads shortly after an activity or an encounter—often without having searched for it online.

Fortunately, this is technologically implausible at scale. Continuous speech recognition and transcription on billions of active devices would require enormous processing power. That would either drain a phone’s battery within hours or result in massive, constant uploads of audio data to remote servers. Both are easy to detect and would have been flagged long ago.

Note: This argument addresses only the technical side; I’ll set the legal aspect aside for now—but it’s worth noting that audio is classified as private data and any such processing would have to be explicitly disclosed and regulated under GDPR (in Europe) or similar laws (e.g., in California).

So what’s really happening?

The most likely explanation is far less dramatic and far more effective: data correlation. Apps like Facebook and others have access to location data and can send information in the background. When two people are physically near each other, both devices report their location. The server can then infer co-presence and correlate interests, profiles, and local events (for example, a concert or festival). Ads are then targeted based on shared context and overlapping social graphs—not on recorded conversations. This gets even more powerful through trackers and beacons embedded in websites and accepted via cookies. These allow advertisers such as Facebook and Google to follow users across the web, continuously updating interest profiles and refining ad targeting throughout the day.

What can you do?

  1. Limit location access and background activity to apps that genuinely need it.
  2. Reject cookies whenever possible.
  3. Regularly clear cookies and site data.
  4. Use privacy-focused browsers; for example, Safari includes built-in protections that block many trackers by default.

As chatGPT likes to say: “Your phone isn’t listening. Your digital exhaust is.”

Related Posts

  1. I took part in SIREN 2008, a research event in the Netherlands, presenting our work with Martha Larson and Maarten de Rijke, on Term Clouds as Surrogates for User Generated Speech [1]; see post

    References

    [1] Manos Tsagkias, Martha Larson, and Maarten de Rijke. 2008. Term clouds as surrogates for user generated speech. In Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval (SIGIR ‘08). Association for Computing Machinery, New York, NY, USA, 773–774. ACM Link PDF

    13.71% similar — Talk at SIREN 2008 on Speech Term Clouds

  2. Abstract

    Streamwatchr (2013–2016), developed at the University of Amsterdam, leveraged machine learning and named entity extraction to track global music listening behavior in real time through tweets. It offered features like Top-100 charts, interactive maps, and a “radio” stream mode driven by a dynamic recommender system. Using Python and MongoDB, it processed over 438 million tweets, identified 660,941 artists, and linked them to MusicBrainz and YouTube videos. This innovative blend of analytics and entity linking earned it a spot in the Dutch delegation at South by Southwest (SXSW).

    Streamwatchr (2013–2016) was a web application developed at the University of Amsterdam, by Wouter Weerkamp, Maarten de Rijke, and myself. Streamwatchr monitored the Twitter feed for #nowplaying, and tracked how people interacted with songs, artists, albums, in real-time. In its latest version it also identified which parts of the lyrics were the most sung along!

    12.50% similar — Streamwatchr — Inside the world's playlist

  3. Abstract

    User generated spoken audio remains a challenge for Automatic Speech Recognition (ASR) technology and content-based audio surrogates derived from ASR-transcripts must be error robust. An investigation of the use of term clouds as surrogates for podcasts demonstrates that ASR term clouds closely approximate term clouds derived from human-generated transcripts across a range of cloud sizes. A user study confirms the conclusion that ASR-clouds are viable surrogates for depicting the content of podcasts.

    References

    [1] Manos Tsagkias, Martha Larson, and Maarten de Rijke. 2008. Term clouds as surrogates for user generated speech. In Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval (SIGIR ‘08). Association for Computing Machinery, New York, NY, USA, 773–774. ACM Link PDF

    11.53% similar — Paper at SIGIR 2008 — Term Clouds as Surrogates for User Generated Speech

  4. Abstract

    We focus on improving the effectiveness of a Virtual Assistant (VA) in recognizing emerging entities in spoken queries. We introduce a method that uses historical user interactions to forecast which entities will gain in popularity and become trending, and it subse- quently integrates the predictions within the Automated Speech Recognition (ASR) component of the VA. Experiments show that our proposed approach results in a 20% relative reduction in errors on emerging entity name utterances without degrading the overall recognition quality of the system.

    Happy to share the news about my first joint pubication with the Siri Speech team at Apple. Our short paper Predicting Entity Popularity to Improve Spoken Entity Recognition by Virtual Assistants with Christophe van Gysel, myself, Ernie Pusateri, and Ilya Oparin, is accepted at SIGIR 2020.

    10.78% similar — Paper at SIGIR 2020 — Predicting Entity Popularity to Improve Spoken Entity Recognition by Virtual Assistants

  5. Abstract

    Modeling user browsing behavior is an active research area with tangible real-world applications, e.g., organizations can adapt their online presence to their visitors browsing behavior with positive effects in user engagement, and revenue. We concentrate on online news agents, and present a semi-supervised method for predicting news articles that a user will visit after reading an initial article. Our method tackles the problem using language intent models trained on historical data which can cope with unseen articles. We evaluate our method on a large set of articles and in several experimental settings. Our results demonstrate the utility of language intent models for predicting user browsing behavior within online news sites.

    10.67% similar — Paper at SIGIR 2012 — Language Intent Models for Inferring User Browsing Behavior

  6. Spoken audio, like any time-continuous medium, is notoriously difficult to browse or skim without support of an interface providing semantically annotated jump points to signal the user where to listen in. Creation of time-aligned metadata by human annotators is prohibitively expensive, motivating the investigation of representations of segment-level semantic content based on transcripts generated by automatic speech recognition (ASR). This paper examines the feasibility of using term clouds to provide users with a structured representation of the semantic content of podcast episodes. Podcast episodes are visualized as a series of sub-episode segments, each represented by a term cloud derived from a transcript generated by automatic speech recognition (ASR). Quality of segment-level term clouds is measured quantitatively and their utility is investigated using a small-scale user study based on human labeled segment boundaries. Since the segment-level clouds generated from

    10.21% similar — Paper at SSCS 2008 — Using Term Clouds to Represent Segment-Level Semantic Content of Podcasts

  7. It was an intense two days, with lots of people dropping by 904Labs’ booth and asking about what we do and how 904Labs search technology can help them in their settings. Altough the average visitor profile was not looking for onsite search, we had good chats with a bunch of Dutch and international and small, medium, and big companies on how onsite search can help their endeavors. We talked on how search technology is underneath many applications, including chatbots, and self-service applications. No matter the application, good search can help increase revenue in the setting of selling products, or reduce costs in the setting of self-service as visitors are better able to find the information they’re looking for without calling the company’s call centre.

    10.11% similar — Exhibiting at A.I. Expo Europe 2018