Chapter 2: Methods in Real-World Evidence Generation — Study Design

References

  1. The European Network of Centres for Pharmacoepidemiology and Pharmacovigilance (ENCePP). Guide on Methodological Standards in Pharmacoepidemiology (Revision 9). EMA/95098/2010. Accessed January 5, 2022. http://www.encepp.eu/standards_and_guidance
  2. Guidelines for Good Pharmacoepidemiology Practices (GPP) - International Society for Pharmacoepidemiology. Accessed January 5, 2022. https://www.pharmacoepi.org/resources/policies/guidelines-08027/
  3. Hernán MA, Robins JM. Using Big Data to Emulate a Target Trial When a Randomized Trial Is Not Available. Am J Epidemiol. 2016;183(8):758-764. doi:10.1093/aje/kwv254
  4. Didelez V. Commentary: Should the analysis of observational data always be preceded by specifying a target experimental trial? Int J Epidemiol. 2016;45(6):2049-2051. doi:10.1093/ije/dyw032
  5. Labrecque JA, Swanson SA. Target trial emulation: teaching epidemiology and beyond. Eur J Epidemiol. 2017;32(6):473-475. doi:10.1007/s10654-017-0293-4
  6. Ford I, Norrie J. Pragmatic Trials. N Engl J Med. 2016;375(5):454-463. doi:10.1056/NEJMra1510059
  7. Hernán MA, Sauer BC, Hernández-Díaz S, Platt R, Shrier I. Specifying a target trial prevents immortal time bias and other self-inflicted injuries in observational analyses. J Clin Epidemiol. 2016;79:70-75. doi:10.1016/j.jclinepi.2016.04.014
  8. Hernán MA. The C-Word: Scientific Euphemisms Do Not Improve Causal Inference From Observational Data. Am J Public Health. 2018;108(5):616-619. doi:10.2105/AJPH.2018.304337
  9. Sterne JA, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016;355:i4919. doi:10.1136/bmj.i4919
  10. Centers for Disease Control and Prevention. Real-World COVID-19 Vaccine Effectiveness in Healthcare Workers | CDC. Published August 25, 2021. Accessed January 5, 2022. https://www.cdc.gov/vaccines/covid-19/effectiveness-research/vaccine-effectiveness-hcp.html
  11. Dagan N, Barda N, Kepten E, et al. BNT162b2 mRNA Covid-19 Vaccine in a Nationwide Mass Vaccination Setting. N Engl J Med. 2021;384(15):1412-1423. doi:10.1056/NEJMoa2101765
  12. Dagan N Barda N, Kepten E, et al. Protocol for: Dagan N, Barda N, Kepten E, et al. BNT162b2 mRNA Covid-19 vaccine in a nationwide mass vaccination setting. N Engl J Med 2021;384:1412-23. DOI: 10.1056/NEJMoa2101765. Published 2021. Accessed September 9, 2021. https://www.nejm.org/doi/suppl/10.1056/NEJMoa2101765/suppl_file/nejmoa2101765_protocol.pdf
  13. Thomas SJ, Moreira ED, Kitchin N, et al. Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine through 6 Months. N Engl J Med. 2021;385(19):1761-1773. doi:10.1056/NEJMoa2110345
  14. Gupta S, Wang W, Hayek SS, et al. Association Between Early Treatment With Tocilizumab and Mortality Among Critically Ill Patients With COVID-19. JAMA Intern Med. 2021;181(1):41-51. doi:10.1001/jamainternmed.2020.6252
  15. Leaf D. Study of the Treatment and Outcomes in Critically Ill Patients With COVID-19. clinicaltrials.gov; 2021. Accessed January 4, 2022. https://clinicaltrials.gov/ct2/show/NCT04343898
  16. RECOVERY Collaborative Group. Tocilizumab in patients admitted to hospital with COVID-19 (RECOVERY): a randomised, controlled, open-label, platform trial. Lancet. 2021;397(10285):1637-1645. doi:10.1016/S0140-6736(21)00676-0
  17. Rosas IO, Bräu N, Waters M, et al. Tocilizumab in Hospitalized Patients with Severe Covid-19 Pneumonia. N Engl J Med. 2021;384(16):1503-1516. doi:10.1056/NEJMoa2028700
  18. Salama C, Mohan SV. Tocilizumab in Patients Hospitalized with Covid-19 Pneumonia. Reply. N Engl J Med. 2021;384(15):1473-1474. doi:10.1056/NEJMc2100217
  19. Roche Group Media Relations. Roche provides update on the phase III REMDACTA trial of Actemra/RoActemra plus Veklury in patients with severe COVID-19 pneumonia. Accessed January 5, 2022. https://www.roche.com/investors/updates/inv-update-2021-03-11.htm
  20. Al-Samkari H, Gupta S, Leaf RK, et al. Thrombosis, Bleeding, and the Observational Effect of Early Therapeutic Anticoagulation on Survival in Critically Ill Patients With COVID-19. Ann Intern Med. 2021;174(5):622-632. doi:10.7326/M20-6739
  21. Thachil J, Juffermans NP, Ranucci M, et al. ISTH DIC subcommittee communication on anticoagulation in COVID-19. J Thromb Haemost. 2020;18(9):2138-2144. doi:10.1111/jth.15004
  22. Bikdeli B, Madhavan MV, Jimenez D, et al. COVID-19 and Thrombotic or Thromboembolic Disease: Implications for Prevention, Antithrombotic Therapy, and Follow-Up: JACC State-of-the-Art Review. J Am Coll Cardiol. 2020;75(23):2950-2973. doi:10.1016/j.jacc.2020.04.031
  23. Gatto N, Garry EM, Chakravarty A. Protocol: Effect of dexamethasone on inpatient mortality among hospitalized COVID-19 patients. Accessed January 5, 2022. https://clinicaltrials.gov/ProvidedDocs/71/NCT04926571/Prot_000.pdf
  24. Mahévas M, Tran VT, Roumier M, et al. Clinical efficacy of hydroxychloroquine in patients with covid-19 pneumonia who require oxygen: observational comparative study using routine care data. BMJ. 2020;369:m1844. doi:10.1136/bmj.m1844
  25. Mahévas M et al. No evidence of clinical efficacy of hydroxychloroquine in patients hospitalised for COVID-19 infection and requiring oxygen: results of a study using routinely collected data to emulate a target trial | medRxiv. Accessed January 5, 2022. https://www.medrxiv.org/content/10.1101/2020.04.10.20060699v1
  26. Self WH, Semler MW, Leither LM, et al. Effect of Hydroxychloroquine on Clinical Status at 14 Days in Hospitalized Patients With COVID-19: A Randomized Clinical Trial. JAMA. 2020;324(21):2165-2176. doi:10.1001/jama.2020.22240
  27. Suissa S, Dell’Aniello S. Time-related biases in pharmacoepidemiology. Pharmacoepidemiol Drug Saf. 2020;29(9):1101-1110. doi:10.1002/pds.5083
  28. Grodstein F, Manson JE, Stampfer MJ. Hormone therapy and coronary heart disease: the role of time since menopause and age at hormone initiation. J Womens Health (Larchmt). 2006;15(1):35-44. doi:10.1089/jwh.2006.15.35
  29. Manson JE, Hsia J, Johnson KC, et al. Estrogen plus progestin and the risk of coronary heart disease. N Engl J Med. 2003;349(6):523-534. doi:10.1056/NEJMoa030808
  30. Renoux C, Dell’Aniello S, Brenner B, Suissa S. Bias from depletion of susceptibles: the example of hormone replacement therapy and the risk of venous thromboembolism. Pharmacoepidemiol Drug Saf. 2017;26(5):554-560. doi:10.1002/pds.4197
  31. Hernán MA, Alonso A, Logan R, et al. Observational Studies Analyzed Like Randomized Experiments: An Application to Postmenopausal Hormone Therapy and Coronary Heart Disease. Epidemiology. 2008;19(6):766-779. doi:10.1097/EDE.0b013e3181875e61
  32. Hernán MA, Alonso A, Logan R, et al. Observational studies analyzed like randomized experiments: an application to postmenopausal hormone therapy and coronary heart disease. Epidemiology. 2008;19(6):766-779. doi:10.1097/EDE.0b013e3181875e61
  33. Danaei G, Tavakkoli M, Hernán MA. Bias in Observational Studies of Prevalent Users: Lessons for Comparative Effectiveness Research From a Meta-Analysis of Statins. Am J Epidemiol. 2012;175(4):250-262. doi:10.1093/aje/kwr301
  34. Ray WA. Evaluating medication effects outside of clinical trials: new-user designs. Am J Epidemiol. 2003;158(9):915-920. doi:10.1093/aje/kwg231
  35. Catalogue of bias collaboration, Lee H, Aronson JK, Nunan D. Collider bias. Catalog of Bias. Published March 1, 2019. Accessed January 5, 2022. https://catalogofbias.org/biases/collider-bias/
  36. Rothman KJ, Greenland S, Lash TL. (2008). Modern Epidemiology, 3rd Edition. Lippincott Williams & Wilkins
  37. Strom BL, Kimmel  SE, Hennessy S. (Eds). Pharmacoepidemiology, 6th Edition. John Wiley & Sons. December 16, 2019.
  38. Dean NE, Hogan JW, Schnitzer ME. Covid-19 Vaccine Effectiveness and the Test-Negative Design. New England Journal of Medicine. 2021;385(15):1431-1433. doi:10.1056/NEJMe2113151
  39. Thompson MG, Stenehjem E, Grannis S, et al. Effectiveness of Covid-19 Vaccines in Ambulatory and Inpatient Care Settings. New England Journal of Medicine. 2021;385(15):1355-1371. doi:10.1056/NEJMoa2110362
  40. Vandenbroucke JP, Brickley EB, Vandenbroucke-Grauls CMJE, Pearce N. A Test-Negative Design with Additional Population Controls Can Be Used to Rapidly Study Causes of the SARS-CoV-2 Epidemic. Epidemiology. 2020;31(6):836-843. doi:10.1097/EDE.0000000000001251
  41. Endo A, Funk S, Kucharski AJ. Bias correction methods for test-negative designs in the presence of misclassification. Epidemiology & Infection. 2020;148. doi:10.1017/S0950268820002058
  42. Lewnard JA, Patel MM, Jewell NP, et al. Theoretical Framework for Retrospective Studies of the Effectiveness of SARS-CoV-2 Vaccines. Epidemiology. 2021;32(4):508-517. doi:10.1097/EDE.0000000000001366
  43. Schneeweiss S. A basic study design for expedited safety signal evaluation based on electronic healthcare data. Pharmacoepidemiology and Drug Safety. 2010;19(8):858-868. doi:10.1002/pds.1926
  44. Schneeweiss S, Rassen JA, Brown JS, et al. Graphical Depiction of Longitudinal Study Designs in Health Care Databases. Ann Intern Med. 2019;170(6):398-406. doi:10.7326/M18-3079
  45. Rassen JA, Bartels DB, Schneeweiss S, Patrick AR, Murk W. Measuring prevalence and incidence of chronic conditions in claims and electronic health record databases. Clin Epidemiol. 2019;11:1-15. doi:10.2147/CLEP.S181242
  46. Johnson ES, Bartman BA, Briesacher BA, et al. The incident user design in comparative effectiveness research. Pharmacoepidemiol Drug Saf. 2013;22(1):1-6. doi:10.1002/pds.3334
  47. Lund JL, Richardson DB, Stürmer T. The active comparator, new user study design in pharmacoepidemiology: historical foundations and contemporary application. Curr Epidemiol Rep. 2015;2(4):221-228. doi:10.1007/s40471-015-0053-5
  48. Franklin JM, Lin KJ, Gatto NM, Rassen JA, Glynn RJ, Schneeweiss S. Real-World Evidence for Assessing Pharmaceutical Treatments in the Context of COVID-19. Clinical Pharmacology & Therapeutics. 2021;109(4):816-828. doi:10.1002/cpt.2185
  49. Suissa S, Moodie EEM, Dell’Aniello S. Prevalent new-user cohort designs for comparative drug effect studies by time-conditional propensity scores. Pharmacoepidemiol Drug Saf. 2017;26(4):459-468. doi:10.1002/pds.4107
  50. Filion KB, Yu YH. Invited Commentary: The Prevalent New-User Design in Pharmacoepidemiology—Challenges and Opportunities. American Journal of Epidemiology. 2021;190(7):1349-1352. doi:10.1093/aje/kwaa284
  51. Webster-Clark M, Ross RK, Lund JL. Initiator Types and the Causal Question of the Prevalent New-User Design: A Simulation Study. American Journal of Epidemiology. 2021;190(7):1341-1348. doi:10.1093/aje/kwaa283
  52. Renoux C, Azoulay L, Suissa S. Biases in Evaluating the Safety and Effectiveness of Drugs for the Treatment of COVID-19: Designing Real-World Evidence Studies. Am J Epidemiol. 2021;190(8):1452-1456. doi:10.1093/aje/kwab028
  53. Suissa S. Immortal Time Bias in Pharmacoepidemiology. American Journal of Epidemiology. 2008;167(4):492-499. doi:10.1093/aje/kwm324
  54. Patorno E, Glynn RJ, Levin R, Lee MP, Huybrechts KF. Benzodiazepines and risk of all cause mortality in adults: cohort study. BMJ. 2017;358:j2941. doi:10.1136/bmj.j2941
  55. Wang SV, Schneeweiss S, Berger ML, et al. Reporting to Improve Reproducibility and Facilitate Validity Assessment for Healthcare Database Studies V1.0. Pharmacoepidemiol Drug Saf. 2017;26(9):1018-1032. doi:10.1002/pds.4295
  56. Wang SV, Pinheiro S, Hua W, et al. STaRT-RWE: structured template for planning and reporting on the implementation of real world evidence studies. BMJ. 2021;372:m4856. doi:10.1136/bmj.m4856
  57. Hernán MA, Hernández-Díaz S. Beyond the intention-to-treat in comparative effectiveness research. Clin Trials. 2012;9(1):48-55. doi:10.1177/1740774511420743
  58. Danaei G, Rodríguez LAG, Cantero OF, Logan R, Hernán MA. Observational data for comparative effectiveness research: An emulation of randomised trials of statins and primary prevention of coronary heart disease. Stat Methods Med Res. 2013;22(1):70-96. doi:10.1177/0962280211403603
  59. Stewart M, Rodriguez-Watson C, Albayrak A, et al. COVID-19 Evidence Accelerator: A parallel analysis to describe the use of Hydroxychloroquine with or without Azithromycin among hospitalized COVID-19 patients. Di Gennaro F, ed. PLoS ONE. 2021;16(3):e0248128. doi:10.1371/journal.pone.0248128
  60. Jarcho JA, Ingelfinger JR, Hamel MB, D’Agostino RB, Harrington DP. Inhibitors of the Renin-Angiotensin-Aldosterone System and Covid-19. N Engl J Med. 2020;382(25):2462-2464. doi:10.1056/NEJMe2012924
  61. Yoshida K, Solomon DH, Kim SC. Active-comparator design and new-user design in observational studies. Nat Rev Rheumatol. 2015;11(7):437-441. doi:10.1038/nrrheum.2015.30
  62. Cadarette SM, Maclure M, Delaney JAC, et al. Control yourself: ISPE‐endorsed guidance in the application of self‐controlled study designs in pharmacoepidemiology. Pharmacoepidemiol Drug Saf. 2021;30(6):671-684. doi:10.1002/pds.5227
  63. Farrington CP. Relative Incidence Estimation from Case Series for Vaccine Safety Evaluation. Biometrics. 1995;51(1):228. doi:10.2307/2533328
  64. Katsoularis I, Fonseca-Rodríguez O, Farrington P, Lindmark K, Fors Connolly AM. Risk of acute myocardial infarction and ischaemic stroke following COVID-19 in Sweden: a self-controlled case series and matched cohort study. The Lancet. 2021;398(10300):599-607. doi:10.1016/S0140-6736(21)00896-5
  65. Douglas IJ, Evans SJW, Hingorani AD, et al. Clopidogrel and interaction with proton pump inhibitors: comparison between cohort and within person study designs. BMJ. 2012;345(jul10 1):e4388-e4388. doi:10.1136/bmj.e4388
  66. Petersen I, Douglas I, Whitaker H. Self controlled case series methods: an alternative to standard epidemiological study designs. BMJ. Published online September 12, 2016:i4515. doi:10.1136/bmj.i4515
  67. Whitaker HJ, Ghebremichael-Weldeselassie Y, Douglas IJ, Smeeth L, Farrington CP. Investigating the assumptions of the self-controlled case series method: Investigating the assumptions of the self-controlled case series method. Statistics in Medicine. 2018;37(4):643-658. doi:10.1002/sim.7536
  68. Simpson CR, Shi T, Vasileiou E, et al. First-dose ChAdOx1 and BNT162b2 COVID-19 vaccines and thrombocytopenic, thromboembolic and hemorrhagic events in Scotland. Nat Med. 2021;27(7):1290-1297. doi:10.1038/s41591-021-01408-4
  69. Hippisley-Cox J, Patone M, Mei XW, et al. Risk of thrombocytopenia and thromboembolism after covid-19 vaccination and SARS-CoV-2 positive testing: self-controlled case series study. BMJ. Published online August 26, 2021:n1931. doi:10.1136/bmj.n1931
  70. Aschengrau A, Seage GR. Descriptive Epidemiology. In: Essentials of Epidemiology in Public Health. Fourth edition. Jones & Bartlett Learning; 2020.
  71. Chew MS, Kattainen S, Haase N, et al. A descriptive study of the surge response and outcomes of ICU patients with COVID‐19 during first wave in Nordic countries. Acta Anaesthesiol Scand. 2022;66(1):56-64. doi:10.1111/aas.13983
  72. Mehta HB, An H, Andersen KM, et al. Use of Hydroxychloroquine, Remdesivir, and Dexamethasone Among Adults Hospitalized With COVID-19 in the United States: A Retrospective Cohort Study. Ann Intern Med. 2021;174(10):1395-1403. doi:10.7326/M21-0857
  73. Prats-Uribe A, Sena AG, Lai LYH, et al. Use of repurposed and adjuvant drugs in hospital patients with covid-19: multinational network cohort study. BMJ. Published online May 11, 2021:n1038. doi:10.1136/bmj.n1038
  74. Savoia C, Volpe M, Kreutz R. Hypertension, a Moving Target in COVID-19: Current Views and Perspectives. Circ Res. 2021;128(7):1062-1079. doi:10.1161/CIRCRESAHA.121.318054
  75. Bradley MC, Graham DJ, Eworuke E, et al. Outpatient corticosteroid use for COVID-19 in the United States: a multi-database study. 2021.
  76. Anderson TS, O’Donoghue AL, Dechen T, Mechanic O, Stevens JP. Uptake of Outpatient Monoclonal Antibody Treatments for COVID-19 in the United States: a Cross-Sectional Analysis. J GEN INTERN MED. 2021;36(12):3922-3924. doi:10.1007/s11606-021-07109-5