ROI: A method for identifying organizations receiving personal data

Many studies have exposed the massive collection of personal data in the digital ecosystem through, for instance, websites, mobile apps, or smart devices. This fact goes unnoticed by most users, who are also unaware that the collectors are sharing their personal data with many different organizations around the globe. This paper assesses techniques available in the state of the art to identify the organizations receiving this personal data. Based on our findings, we propose ROI (Receiver Organization Identifier), a fully automated method that combines different techniques to achieve a 95.71% precision score in identifying an organization receiving personal data. We demonstrate our method in the wild by evaluating 10,000 Android apps and exposing the organizations that receive users’ personal data.

💡 Research Summary

The paper addresses the problem of uncovering which organizations receive personal data collected by digital services such as websites, mobile apps, and smart devices. While many studies have documented the sheer volume of data harvested, users remain largely unaware of the downstream recipients. Existing techniques for identifying these recipients rely on WHOIS look‑ups, SSL certificate inspection, or manual analysis of privacy policies, each of which suffers from significant reliability issues. WHOIS records are often incomplete, outdated, or deliberately obfuscated; SSL certificates may not contain clear organizational identifiers and can be shared across multiple services via CDNs or load balancers; privacy policies are unstructured natural‑language documents that are difficult to parse automatically. Consequently, prior work (e.g., PolicyXray, WebXray, WHOIS‑only approaches) achieves modest precision—sometimes as low as 23 %—and provides limited coverage.

To overcome these limitations, the authors propose ROI (Receiver Organization Identifier), a fully automated pipeline that fuses three complementary techniques:

1. WHOIS Consultation – Queries the global WHOIS database for each destination domain observed in network traffic, extracts registrant name, organization, and contact fields, and flags records that appear incomplete or suspicious.
2. SSL Certificate Inspection – Retrieves the TLS certificate presented during the HTTPS handshake, parses the Subject and Subject Alternative Name fields for organization names, and cross‑checks them against WHOIS data.
3. Privacy‑Policy Analysis – Crawls the privacy policy associated with each app or website, then applies a Natural Language Processing (NLP) stack that includes tokenisation, part‑of‑speech tagging, and a custom Named‑Entity Recognition (NER) model trained on annotated policy corpora. The model extracts entities such as “data controller”, “third‑party recipient”, and contact details.

The pipeline operates sequentially: WHOIS provides an initial candidate set, SSL certificates refine or confirm the candidate, and policy analysis supplies the final verification. Only when at least two sources agree does ROI emit a definitive organization identifier, thereby reducing false positives.

Evaluation
The authors evaluated ROI on a corpus of 10,000 Android applications collected from the Google Play Store. They instrumented each app to capture outbound network connections, extracted the destination domains, and fed them through the ROI pipeline. Manual verification by domain experts served as ground truth. ROI achieved a precision of 95.71 %, meaning that only 4.29 % of the reported organizations were incorrect. The paper does not report recall or F1‑score, leaving open the question of how many true recipient organizations were missed.

Datasets
Two public datasets accompany the study: (a) a set of 300 domains whose WHOIS, SSL, and policy information were manually validated, and (b) a set of 1,112 unique domains observed in the Android traffic, each annotated with the types of personal data (e.g., location, contacts, device identifiers) they receive. Both datasets will be released on an open‑access repository, providing a valuable benchmark for future research.

Related Work Comparison
Compared with WHOIS‑only methods, which the authors cite as achieving only ~23 % accuracy due to stale or missing records, ROI’s multi‑source approach dramatically improves precision. PolicyXray and WebXray rely on keyword matching and external search engines to locate policies; ROI augments these with a trained NER model, achieving higher recall of relevant entities while maintaining low false‑positive rates. The authors also discuss NER‑based approaches (e.g., Hossain et al.) that identify third‑party entities but still require extensive post‑processing; ROI streamlines this by integrating SSL data to disambiguate entities that share similar names.

Limitations and Future Work
The authors acknowledge several constraints: (1) the absence of recall measurement limits understanding of coverage; (2) dynamic network behaviours (e.g., CDN redirection, load‑balancing) may cause domain‑to‑organization mappings to change over time; (3) SSL certificates sometimes omit organization names or use generic corporate certificates, leading to potential mis‑attribution; (4) privacy policies are frequently outdated, vague, or written in legalese, which can degrade NER performance; (5) the method focuses exclusively on Android apps, so applicability to iOS, web browsers, or IoT devices remains untested. They suggest future work on continuous monitoring, incorporation of dynamic analysis (e.g., runtime instrumentation), and expansion to other platforms.

Conclusion
ROI demonstrates that combining WHOIS, SSL certificate inspection, and NLP‑driven privacy‑policy analysis can yield a highly precise automated system for identifying organizations that receive personal data. By releasing both the methodology and supporting datasets, the authors provide a foundation for further research in privacy auditing, regulatory compliance automation, and transparency tools for end‑users and developers alike.