The independent source for health policy research, polling, and news.
The Health Misinformation Monitor – Methodology
Media Monitoring Methods
Data Sources: Data in the Health Misinformation Monitor come, in part, from media monitoring and social listening research conducted by the Public Good Projects (PGP). PGP’s media monitoring systems operate continuously, monitoring discourse around the clock, year-round. Analysts at PGP engage in real-time and longitudinal media monitoring to gauge public knowledge, attitudes, and behaviors related to health issues. Analysts use various software platforms to gather publicly available media data, including Meta’s CrowdTangle platform and link checker, Quid, Talkwalker, Google Trends, and Google Alerts, among others. PGP’s media monitoring systems collect data in real-time across multiple media sources, including social and digital media platforms (e.g., Facebook, Instagram, Twitter, Reddit), video-sharing sites (e.g., TikTok, YouTube, Bitchute, Vimeo), online forums, and traditional media outlets like newspapers, magazines, and television.
Data Collection: Data are collected through keyword searches created by PGP analysts, which capture discussions on various health topics including vaccines, COVID-19, substance use, harm reduction, HIV/AIDS, abortion/contraception, health policy, firearms, LGBTQ+ issues, among others. For each of these subjects, analysts construct tailored Boolean search queries that include sometimes hundreds of standalone and co-occurring keywords relevant to the specific topic. These search queries are updated on an as-needed basis to align with evolutions in discourse and to incorporate the latest insights in monitoring practices. Further details about the data collection process have been documented in scientific journals, both by PGP (see Bonnevie, 2020; Bonnevie, 2020) and others in the field (see Mollema, 2015; Chen, 2018; Mooney, 2018; Jamison, 2020; Karafillakis, 2021).
Analysis: PGP adapts its methods for analysis to depend on the topic and types of conversation PGP is seeing. Their focus is primarily around 1) identifying spikes in conversation, and 2) understanding broader conversation themes.
- Conversation spikes: In some instances, PGP concentrates on pinpointing conversations revolving around specific stories that have spiked conversation. These conversation spikes can become pivotal moments, revealing how new misinformation is spread. Analyzing these spikes helps us understand the catalysts for the spread of misinformation, and the underlying dynamics of conversation that give rise to this type of sudden spread.
- Conversation themes: PGP also focuses on analyzing recurring or persistent conversation themes. These broad themes allow us to capture the evolving nature of online discussions and the continuous dialogue surrounding certain topics. Identifying and tracking the way themes evolve over time enables us to gauge long-term trends and changing narratives in the digital landscape.
Nuance in Monitoring: Complexities of Misinformation: While PGP’s media monitoring systems are designed to track and analyze a vast array of data related to conversations about health misinformation, it’s important to recognize that misinformation isn’t always straightforward; there is a lot of nuances surrounding cultural values and their influence on public health discourse. Misinformation exists on a spectrum and isn’t black and white. Cultural values play an important role in shaping people’s perceptions and behaviors, including their attitudes toward health-related issues. Some ideas deemed as cultural values may not inherently be misinformation but can contribute to misinformation by fostering stigmatizing perspectives on health issues which can ultimately result in the spread of inaccurate or harmful information. Our monitoring efforts extend beyond the straightforward identification of blatant misinformation and try to understand the interplay between cultural values and their potential to support or lead to misinformation. This more holistic approach allows us to better understand the multifaceted nature of the health discourse landscape.