COVID-19 Real-World Evidence Primer

Executive Summary

Executive Summary: COVID-19 Real-World Evidence Primer

In response to the COVID-19 pandemic, the use of real-world data (RWD) for research accelerated, supporting the global response to the COVID-19 pandemic via conduct of rigorous studies on the safety and effectiveness of diagnostics, drug repurposing, and novel therapies and vaccines in near real time. The use of RWD has many advantages, most notably that data typically are collected as a part of routine health care and are available expeditiously and often for large populations. However, there are special considerations for the use of these data for the secondary purpose of research. To learn from the evolving lessons of addressing the COVID-19 pandemic and to bolster the future use of real-world evidence (RWE), we created this primer to describe the RWD ecosystem. We included key principles of study design and potential sources of bias, examples of COVID-19 studies that use RWD, and mechanisms for disseminating RWE gleaned from the conduct of RWD research. We also introduce the COVID-19 Evidence Accelerator initiative that was established during the pandemic to support RWD study conduct.1

Chapter 1: Overview of Real-World Data

RWD include information recorded in electronic health records (EHRs); administrative claims; patient-reported outcomes (generally responses surveyed directly from patients on feeling and function); patient-generated health data (from apps, smartwatches, pedometers, etc.); product- and disease-specific registries; and information about environmental factors and social determinants of health. These RWD sources can both describe the patient experience and depict an overview of population health. Although much work had been done to lay the groundwork for the use of RWD, including the FDA RWE framework2 as part of the 21st Century Cures Act,3 many challenges for the use of RWD to support causal inferences of treatment effects exist — challenges that were further compounded by the need for accelerated research and evidence during the pandemic. To generate reliable RWE from RWD, the data should meet acceptable thresholds for validity and provenance in addition to being fit for the intended purpose. Data governance, privacy, and security are also important factors for RWD studies.

Chapter 2: Methods in Real-World Evidence Generation – Study Design

There are various observational study designs that are typically used to generate RWE, including between-person designs (cohort and case-control studies) and within-person designs, which compare different time windows (i.e., lengths of time) within the same person. The concept of a target trial, an observational study that aims to emulate the key features of a randomized control trial, provides a useful framework to think about potential biases in RWE studies. Descriptive designs examine disease or exposure patterns in the population, focusing on characteristics related to person, place, and time. This chapter describes the key features of each design, the types of questions that can be answered, and advantages and disadvantages of each type of design.

Chapter 3: Methods in Real-World Evidence Generation – Sources of Error

Because RWD are not collected for research purposes, there are potential sources of error that are typically tied to the context in which the data were originally collected. These potential sources of error, as well as the failure to consider these issues in study design and / or analysis, can cause a variety of biases, such as confounding or misclassification bias. In addition, investigators can compound such problems, resulting in problems such as selection bias or immortal time bias. This chapter describes common mechanisms for these biases in observational research and how to mitigate them.

Chapter 4: Examples of COVID-19 Real-World Evidence Studies

In response to the unprecedented impact and evolution of COVID-19, collaboration was critical for the launch of rigorous studies to address the evolving needs across the globe. This chapter describes selected example studies:

  • Cohort study presented in the context of target trial emulation: BNT162b2 mRNA COVID-19 Vaccine in a Nationwide Mass Vaccination Setting
  • Cohort study: Effect of pre-exposure use of hydroxychloroquine on COVID-19 mortality: a population-based cohort study in patients with rheumatoid arthritis or systemic lupus erythematosus using the OpenSAFELY platform
  • Case-control study: Effectiveness of Covid-19 Vaccines in Ambulatory and Inpatient Care Settings
  • Self-control case series: Risk of acute myocardial infarction and ischaemic stroke following COVID-19 in Sweden: a self-controlled case series and matched cohort study
  • Drug utilization study: Use of repurposed and adjuvant drugs in hospital patients with COVID-19: multinational network cohort study
  • Diagnostic test evaluation study: Optimizing SARS-CoV-2 Surveillance in the United States: Insights From the National Football League Occupational Health Program

Chapter 5: Major Multi-stakeholder Initiatives - Defining the Future of COVID-19 Observational Research

The pandemic of COVID-19 laid bare the limited interoperability capacity of the existing health care data infrastructure for assembling data to quickly address critical questions about a novel disease. Several initiatives have emerged to address this shortfall using RWD for COVID-19 observational research. This chapter features a selection of multi-stakeholder COVID-19 RWD initiatives that have contributed to our understanding of the COVID-19 pandemic and/or are structured to continue to provide opportunities for observational research about emerging issues related to the disease.

Six COVID-19 RWD multi-stakeholder initiatives are also covered in depth in this chapter:

  • Observational Health Data Science and Informatics (OHDSI): Initiative leveraging existing international distributed health care data in an interdisciplinary collaborative that facilitates open-source analyses to conduct observational studies on COVID-19 disease characterization, treatment, and care
  • FDA Sentinel: Food and Drug Administration (FDA) system leveraging a distributed data network with a common data model as well as other standalone data sources to conduct COVID-19-related studies
  • OpenSAFELY: Initiative that enables the access of multiple United Kingdom (UK) government data sources in a reliable and protected platform to address COVID-19 research needs
  • Vaccine Monitoring Collaboration for Europe (VAC4EU): International non-profit association set up with the aim of conducting collaborative real-world analysis on vaccines. The entity was founded as a result of the Innovative Medicines Initiative-funded ADVANCE project that was initiated after the H1N1 pandemic
  • COVID-19 Research Database: Cross-industry, cross-sector initiative composed of institutions that donate technology services, health care expertise, and de-identified data in the United States (US) for COVID-19 observational research. The data repository contains integrated, linked data sets from multiple sources, from the more traditional RWD (claims, EHR) to life insurance claims, consumer data, and mortality records
  • COVIDRIVE: Public-private partnership leveraging the existing vaccine effectiveness platform in Europe to comprehensively examine COVID-19 vaccine effectiveness across a range of products and assist vaccine companies in fulfilling their regulatory obligations

Chapter 6: The COVID-19 Evidence Accelerator

The COVID-19 Evidence Accelerator initiative1 was launched by the Reagan-Udall Foundation for the FDA (FDA Foundation),4 in collaboration with Friends of Cancer Research (Friends)5 and on behalf of the FDA, to provide a unique open venue for major data organizations, government and academic researchers, and health systems to share information about COVID-19 efforts, and to convene a community to urgently address questions about COVID-19.

Chapter 7: Communicating about Real-World Evidence

The pandemic illustrated the need for rapid, reliable dissemination of information — not only among researchers, but to other stakeholders including regulators, clinicians, and other health care workers, as well as to the general population. This demand for evidence that can guide regulatory and clinical decision-making must be weighed against the need for adequate vetting of RWE. The growing skepticism regarding scientific evidence is a critical problem that highlights the need for timely and trustworthy communication. This chapter provides practical guidance about dissemination, including the publication processes for developing a communications plan, creating messaging and communication channels, and transmitting internal communications.

Conclusion

Although RWD was being used for research prior to COVID-19, the pandemic pushed researchers to be more collaborative, be more efficient, react faster, and disseminate information more frequently, all while ensuring adequate study quality and high scientific standards. By learning from our experiences, we can build on the momentum of these collaborations to continue to address COVID-19 as well as other diseases and important public health issues.

References

  1. Evidence Accelerator Home | Evidence Accelerator. Accessed January 12, 2022. https://evidenceaccelerator.org/
  2. U.S. Food and Drug Administration. Framework for FDA’s Real-World Evidence Program. Published online January 4, 2022. Accessed January 4, 2022. https://www.fda.gov/media/120060/download
  3. Federal Register :: 21st Century Cures Act: Interoperability, Information Blocking, and the ONC Health IT Certification Program. Accessed August 20, 2020. https://www.federalregister.gov/documents/2020/05/01/2020-07419/21st-century-cures-act-interoperabi…
  4. Real-World Data Webinar Series: Data Standards | Reagan-Udall Foundation. Accessed January 12, 2022. https://reaganudall.org/news-and-events/events/real-world-data-webinar-series-data-standards
  5. Learn about Friends of Cancer Research. Accessed January 12, 2022. https://friendsofcancerresearch.org/