I wasn't the only one disappointed. A New York Times reporter wrote that the iOS labels were, "often illuminating," but complained that they "created more confusion."2 A Washington Post reporter wrote about apps that deceptively claim on their labels that they don't collect data, when in fact they do.8 In February 2021, members of the U.S. Congress wrote to Apple CEO Tim Cook with concerns that app labels were misleading consumers.3
The Idea Behind Privacy Nutrition Labels
Food nutrition labels were mandated in the U.S. by the Nutrition Labeling and Education Act of 1990 (NLEA) and are largely considered to be a success. Even so, research studies have found that nutrition labels have had only a modest impact on consumers' food in-take decisions, with consumers who are most interested in nutrition receiving the greatest benefit.10 The standardized label approach has also been adopted for drug labels, energy labels on appliances, and financial institution privacy notices.5 These labels provide information customized for each product that can help consumers compare products and make informed decisions. These informational labels serve a different purpose than warning labels, which alert consumers to hazards, such as the dangers of smoking cigarettes.
While food nutrition labels may not have had an enormous impact on improving eating habits, they have made it easier for those interested in seeking healthier food choices or avoiding certain ingredients to do so. They have also resulted in greater transparency about the food we consume. Even individuals who rarely consult nutrition labels on their own may benefit when journalists review nutrition labels and write articles about particularly unhealthy foods.
As a privacy researcher, I have been enthusiastic about the privacy nutrition label concept because it offers an opportunity to present privacy information in simple, standardized language for easy comparison. With students and colleagues, I have designed and evaluated privacy nutrition labels for websites,11 mobile apps,12 and IoT devices.7 Privacy is complicated and making privacy labels understandable takes some effort. However, our studies have consistently found that well-designed labels improve understanding about data practices and make privacy choices easier for consumers.
As with food nutrition labels, only a minority of consumers may consult privacy labels often, but labels have the potential to benefit consumers who are interested in considering privacy in their decision making. In addition, standardized labels allow for the development of search engines that take privacy into account5,6 as well as for easy collection of large-scale datasets describing company data practices.16 However, if privacy labels are confusing and designed in such a way that comparisons are difficult, and if the information on labels is not trustworthy, then they fail to meet their goals.
Figure. App privacy labels for the Just Dance Now app: a) compact version of Android label; b) compact version of iOS label; c) full Android label (which can be expanded further for additional details); d) full iOS label.
User Studies with Developers and End Users
After seeing concerns about the accuracy of iOS privacy labels, I worked with colleagues and students to investigate whether iOS app developers were able to create privacy labels accurately. While some developers may intentionally misrepresent their data practices, our study suggests that even those who attempt accurate representations have trouble doing so. We interviewed 12 iOS app developers and observed them creating labels for an app they had developed, thinking aloud as they worked. Developers were confused about the terminology used in privacy labels and often found the explanations in Apple's documentation to be ambiguous. We also found several reoccurring errors and misunderstandings about privacy labels. For example, many developers underreported data elements in the iOS "data linked to you" category, failing to realize that data elements that are not inherently identifiable can nonetheless be linked to individuals because they are stored with identifiable data. Many developers also failed to report data elements collected through third-party libraries because they were unaware of libraries' data collection practices.15 Indeed, a study of traffic sent to known advertising and tracking domains from apps in the German iOS app store found that at least 16% of the apps studied transmitted data to third parties without mention of this practice in their privacy labels.13
Having determined that iOS privacy label vocabulary was confusing to app developers, we next interviewed 24 lay iPhone users about their experience with the labels. Although the labels were introduced in the iOS app store over a year before our study, most participants were unaware they existed and few had ever seen one, probably because they require some scrolling to discover. Approximately half of our participants expected incorrectly that all data collected by an app would be disclosed in its label; in fact, the label focuses on data used for tracking or advertising. Many participants were confused about the structure of the label, which displays up to three tiles representing "data used to track you," "data linked to you," and "data not linked to you," and lists specific data types within each tile. Clicking on a tile provides access to a detailed view showing purposes for which data is used, with data types and sometimes specific data elements under each purpose.
Many data types are used in multiple ways and thus listed under more than one purpose, leading some participants to assume the label included redundant information. Users were just as confused about terminology as developers, and few participants noticed or followed the link to Apple's definitions of terms. Both developers and users were especially confused by the term "tracking," which Apple uses to convey that collected data can be linked with third-party data for targeted advertising or shared with data brokers. People often expected the term referred to tracking their physical location or website visits, as that is how they had heard it most commonly used. Finally, some users were frustrated that the iOS label is completely disconnected from iOS privacy controls.
We are currently continuing our research with Android app developers and users. Our preliminary results suggest that Android labels, which use different definitions for some terms, suffer from many of the same comprehension problems as iOS labels. Furthermore, Android label are structured differently and require additional clicks to understand how data is used. On the other hand, Android labels focus on data safety, and some users seem to appreciate the information they provide about app security practices.
A Recipe for Success
The current app privacy labels are disappointing. Here is what Apple and Google can do to improve them:
Improve label accuracy
- App developer tools should automatically compile lists of data collected by the app and aid developers in annotating it with data uses.9,14 This would allow most of the app privacy label to be generated automatically.
- App stores should use automated verification techniques to verify the accuracy of privacy labels to the extent possible and employ auditing techniques to at least spot check components that can't be verified automatically.
Make labels easier to understand
- App platforms should all develop the same unambiguous, standardized privacy terminology (consistent with common usage of terms) and should have tooltips or convenient links to definitions in both user and developer interfaces.
- Privacy labels should include a summary view that emphasizes what users find most important.
- As privacy labels represent multiple data types that each may be used for multiple purposes, the full label representation should employ a tabular or matrix format that makes this clear.
- Privacy labels should be refined through iterative design and evaluation with users. Testing is needed to determine what is most important to emphasize, how to explain privacy concepts clearly to both users and developers, and how to represent a complex label structure on a small screen.
Make labels easier to find and use
- Privacy labels (or links) should be placed prominently in app stores where they are visible without scrolling.
- Privacy label elements should be integrated into app privacy interfaces to help inform users' privacy decisions within the app (for example, making decisions about runtime permissions).
- App stores should develop (or support third-party development of) tools that make use of label data, for example to allow searching for apps that match a user's privacy criteria, comparing similar apps on the basis of privacy, or customizing labels based on each user's preferences.
2. Chen, B.X. What we learned from Apple's new privacy labels. The New York Times, (Jan. 27, 2021); https://nyti.ms/3LsOCc9
3. Congress of the United States. House of Representatives, Committee on Energy and Commerce, letter to Tim Cook. (Feb. 9, 2021); https://bit.ly/3xAaWv1
4. Cranor, L.F. Necessary but not sufficient: Standardized mechanisms for privacy notice and choice. J. on Telecomm. and High Tech. L. 10, 273 (2012); https://bit.ly/3RZTjgc
5. Cranor, L.F. et al. A large-scale evaluation of U.S. financial institutions' standardized privacy notices. ACM Trans. Web 10, 3, Article 17 (Aug. 2016); https://bit.ly/3RYMvQ3
6. Egelman, S. et al. Timing is everything? The effects of timing and placement of online privacy indicators. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (2009), 319–328; https://bit.ly/3QWE3PN
7. Emami-Naeini, P. et al. Ask the experts: What should be on an IoT privacy and security label? In Proceedings of the 2020 IEEE Symposium on Security and Privacy (SP) (2020), 447–)464; doi: 10.1109/SP40000.2020.00043.
8. Fowler, G.A. I checked apple's new privacy 'nutrition labels.' Many were false. The Washington Post (Jan. 29, 2021); https://wapo.st/3dx2rK9
9. Gardner, J. et al. Helping mobile application developers create accurate privacy labels. (IWPE'22, May 2022); https://bit.ly/3qOgs9g
10. Kelley, P.G. et al. A "nutrition label" for privacy. In Proceedings of the 5th Symposium on Usable Privacy and Security, pages 1–12, 2009. https://bit.ly/3dqMTrr
11. Kelley, P.G. et al. Standardizing privacy notices: An online study of the nutrition label approach. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '10 (2010), 1573–1582; https://bit.ly/3BSH0Nd
12. Kelley, P.G. et al. Privacy as part of the app decision-making process. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, (2013), 3393–3402; https://bit.ly/3LuiRji
14. Li, T. et al. Honeysuckle: Annotation-guided code generation of in-app privacy notices. In Proceedings of ACM Interact. Mob. Wearable Ubiquitous Technol. 5, 3 Article 112 (Sept 2021); https://bit.ly/3dsj2Pq
15. Li, T. et al. Understanding challenges for developers to create accurate privacy nutrition labels. In CHI Conference on Human Factors in Computing Systems, CHI '22, New York, NY, USA, 2022; https://bit.ly/3BUsu7K
16. Li, Y. et al. Understanding iOS privacy nutrition labels: An exploratory large-scale analysis of app store data. In CHI Conference on Human Factors in Computing Systems Extended Abstracts, CHI EA '22, New York, NY, USA, 2022; https://bit.ly/3eWR0vL
The Digital Library is published by the Association for Computing Machinery. Copyright © 2022 ACM, Inc.