Chapter 4: Examples of COVID-19 Real-World Evidence Studies

Case-Control Study

Study citation: Thompson MG, Stenehjem E, Grannis S, Ball SW, Naleway AL, Ong TC, DeSilva MB, Natarajan K, Bozio CH, Lewis N, Dascomb K, Dixon BE, Birch RJ, Irving SA, Rao S, Kharbanda E, Han J, Reynolds S, Goddard K, Grisel N, Fadel WF, Levy ME, Ferdinands J, Fireman B, Arndorfer J, Valvi NR, Rowley EA, Patel P, Zerbo O, Griggs EP, Porter RM, Demarco M, Blanton L, Steffens A, Zhuang Y, Olson N, Barron M, Shifflett P, Schrag SJ, Verani JR, Fry A, Gaglani M, Azziz-Baumgartner E, Klein NP. Effectiveness of Covid-19 Vaccines in Ambulatory and Inpatient Care Settings. N Engl J Med. 2021 Oct 7;385(15):1355-1371. doi: 10.1056/NEJMoa2110362. Epub 2021 Sep 8. PMID: 34496194; PMCID: PMC8451184.3

Study objective: Understanding the effectiveness of COVID-19 vaccines, particularly in populations disproportionately affected by the disease, is of paramount importance to improving health outcomes. Three vaccines were initially authorized for emergency use in the United States (US) by the Food and Drug Administration (FDA): BNT162b2 (Pfizer–BioNTech), mRNA-1273 (Moderna), and Ad26.COV2.S (Johnson & Johnson/Janssen). The objective of this study was to evaluate the real-world effectiveness of the 3 vaccines.

PICO: This is a case-control study of adult visits in an ambulatory clinic and emergency department setting who were tested for COVID-19 within a multicenter health care system network. This study took place across 187 hospitals across the US, with study dates before and after the widespread availability of COVID-19 vaccines. A test negative design was used to study comparative effectiveness for test positivity between unvaccinated and vaccinated individuals. In this study, the ability of each of the 3 vaccines to prevent hospitalization, admission to an intensive care unit, or an ambulatory care visit (emergency room and urgent care) was characterized.

Data source: Electronic health record data from the VISION network, a consortium of 7 academic and private medical centers, were used for this study, with linkage to statewide and local immunization registries, claims, laboratory results, and other diagnostic data where available.

Study period: January 1 through June 22, 2021

Key sources of error and how they were handled: Data derived from several disparate sources (i.e., electronic health records, city and state immunization registries, and claims data) were used for vaccination verification, which reduced potential exposure misclassification. This study had a particularly robust analysis of results as they pertain to racial and ethnic demographics, which is important for COVID-19 (and other diseases), as there are differences in clinical course and outcomes among minority communities.