How Topic Novelty Impacts the Effectiveness of News Veracity Interventions

As others have, we define fake news as "intentionally" and "verifiably" false news articles that mislead readers.¹ As such, the characterization, detection, and prevention of fake news has become a top priority for preventing the spread of misinformation (false or inaccurate information) and disinformation (false information that is intended to mislead). Significant efforts in algorithmic fake-news detection have led to the development of artificial intelligence (AI) tools that provide signals and advice to news consumers to assist with fake news detection, albeit with varying effectiveness. Research surrounding this topic has predominantly focused on the design of algorithms (for instance Baly et al.;³ Cruz et al.;⁷ Hosseinimotlagh and Papalexakis;¹¹ and Bozarth, Saraf, and Budak⁵), with other work examining surrounding issues such as the impact of advice (for example, Moravec et al.¹⁶) and potential negative implications (for instance, Pennycook et al.²¹).

Back to Top

Key Insights

In this work, we seek to provide insight on the effectiveness of AI advice in terms of reader acceptance. We specifically focus on news interventions in the form of statements pertaining to the accuracy and reliability of an article, which we term news veracity statements. Twitter, for example, began using new labels and warning messages on some "Tweets containing disputed or misleading information related to COVID-19."²³ We further examine news interventions through a specific lens: that of the novelty of the news topic. Novelty refers to the extent to which incoming information is similar to prior knowledge.^12,28 In this article, it refers to the extent to which news readers encounter unfamiliar news. Specifically, we ask: when a novel situation arises, will interventions in the form of statements on the veracity of articles be more effective than when those same interventions are used on news articles about more familiar situations? The theoretical reasoning for this research question comes from the concept of confirmation bias, which states that people tend to process information in a way that favors their previously held beliefs. We are, therefore, interested in testing a novel news scenario for which prior beliefs are weak. We find that interventions are significantly more effective in novel news situations, implying that their use in AI tools should be focused on novel situations. An important implication for future research is that work is needed to better understand the contingencies underlying the acceptance and effectiveness of AI news interventions.