Characteristics and Strength of Evidence of COVID-19 Studies Registered on ClinicalTrials.gov | Infectious Diseases | JAMA Internal Medicine | JAMA Network
[Skip to Content]
[Skip to Content Landing]
Figure.  Primary Outcome of Randomized Clinical Trials (RCTs) by Enrollment Size
Primary Outcome of Randomized Clinical Trials (RCTs) by Enrollment Size

Primary listed outcome of 664 RCTs organized by intended enrollment size. Most studies have intended enrollment fewer than 500 patients, and the most common primary end points address (1) clinical course, (2) infection rates, or (3) persistence of viral detection.

Table.  COVID-19 Studies Registered on ClinicalTrials.gov
COVID-19 Studies Registered on ClinicalTrials.gov
1.
ClinicalTrials.gov. COVID-19 search details. Accessed April 8, 2020. https://clinicaltrials.gov/ct2/results/details?cond=COVID-19
2.
OCEBM Levels of Evidence Working Group. OCEBM levels of evidence. Published May 1, 2016. Accessed April 9, 2020. https://www.cebm.net/index.aspx?o=5653
3.
Kim  AHJ, Sparks  JA, Liew  JW,  et al.  A rush to judgment? rapid reporting and dissemination of results and its consequences regarding the use of hydroxychloroquine for COVID-19.   Ann Intern Med. 2020;172(12):819-821. doi:10.7326/M20-1223 PubMedGoogle Scholar
4.
Mullard  A.  Flooded by the torrent: the COVID-19 drug pipeline.   Lancet. 2020;395(10232):1245-1246. doi:10.1016/S0140-6736(20)30894-1 PubMedGoogle ScholarCrossref
5.
World Health Organization. “Solidarity” clinical trial for COVID-19 treatments. Updated June 4, 2020. Accessed May 20, 2020. https://www.who.int/emergencies/diseases/novel-coronavirus-2019/global-research-on-novel-coronavirus-2019-ncov/solidarity-clinical-trial-for-covid-19-treatments
6.
Banno  M, Tsujimoto  Y, Kataoka  Y.  Studies registered in non-ClinicalTrials.gov accounted for an increasing proportion of protocol registrations in medical research.   J Clin Epidemiol. 2019;116:106-113. doi:10.1016/j.jclinepi.2019.09.005 PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    2 Comments for this article
    EXPAND ALL
    Strengthening Registered COVID-19 Clinical Trials
    Michael McAleer, PhD(Econometrics),Queen's | Asia University, Taiwan
    The informative research presentation by medical experts on the characteristics and strengths of registered COVID-19 clinical trials bears serious analysis for designing future clinical trials, and developing relevant and appropriate healthcare policies.

    The carefully constructed registered COVID-19 clinical trials data led to the identification and inclusion of 1,551 studies for the period March 1, 2011 - May 19, 2020, of which 451 could lead to level 2 evidence or the highest level of individual study evidence. 

    The most common primary and secondary outcomes were reported to be mortality, ventilation requirement, and treatment complications.

    In addition to the
    stated limitations, which are required by regulation to include only drug, device, or biological studies to be registered, and with one-half of non-US studies estimated to be non-registered, the following might be considered for future analysis as dealing with the limits to registered COVID-19 clinical trials :

    (1) expand registration beyond drug, device, and biological studies;

    (2) broaden the range of drugs and biological compounds, including the topical hydroxychloroquine, remdesivir, antivirals, and corticosteroids;

    (3) extend the data set beyond May 19, 2020, after which the number of confirmed COVID-19 cases exploded worldwide;

    (4) analyse the registered COVID-19 clinical trials recursively by day, week, and month as the pandemic progressed;

    (5) expand blinding, as required for level 2 evidence, and placebo-controlled clinical trials; 

    (6) using clinical trials registered on ClinicalTrials.gov as a benchmark, consider clinical trials that are not registered;

    (7) consider clinical trials that might be at alternative stages of preliminary development and outcomes;

    (8) establish alternatives to level 2 evidence beyond placebo-controlled, blinded studies.

    Current and impending COVID-19 patients are unlikely to be concerned about some relaxation in strict regulations governing registration and assessment of clinical trials, including satisfying level 2 evidence, as long as safety standards are not unduly compromised in balancing speed, efficacy, usefulness, and practicality of clinical trials outcomes.
    CONFLICT OF INTEREST: None Reported
    READ MORE
    Totality of Evidence Will Strengthen Our Understanding of COVID-19, Treatments and Recovery
    Jennifer Christian, PharmD, MPH, PhD | VP, Clinical Evidence & Epidemiology, IQVIA
    Authors: Nancy A. Dreyer, PhD, MPH; Christina D. Mack, PhD, MSPH; and Jennifer B. Christian, PhD, MPH, PharmD

    We appreciate Pundi et al’s efforts to describe the coronavirus disease 2019 (COVID-19) studies registered on ClinicalTrials.gov, but suggest that their conclusions are unduly pessimistic. (1) Using the OCEBM criteria for characterizing the level of evidence for evaluating a “therapy, prevention, etiology or harm,” they assert that studies that are not randomized clinical trials (RCT) or systematic reviews (SR) “likely will not yield meaningful evidence at a time when rapid generation of high-quality knowledge is critical”.
    This narrow lens may slow our ability to fight COVID-19, particularly when so little is known about this novel infectious disease.

    Consider that nearly all COVID-19 RCTs study the disease in hospital – an important setting, but only a small proportion of COVID-19 patients – and balance that with what can be learned from large observational research in terms of monitoring spread, evaluating symptom severity and understanding disease progression among community-dwelling people early in the disease course or experiencing moderate infection. (2,3) Just as the European Medicines Agency has demonstrated its interest in using big data to improve decision making, enhance evidentiary standards and provide patient benefits, (4) non-randomized studies are rightly being used to fight COVID-19 by the medical community to characterize those most vulnerable and those who recover quickly. These studies can provide broad, practical evidence more expeditiously than RCTs, such as guidance toward back to work decisions and evaluation of the numerous medications being repurposed for COVID-19. We support the position articulated by the Duke Margolis Center for Health Policy Research that the totality of evidence from RCTs and real world evidence provides a better understanding of a disease and a therapy’s impact in the population. (5) Undervaluing the contributions from observational studies, including those about COVID-19 outside of the hospital, actually reduces our ability to understand this disease in its totality.

    Corresponding author: Nancy.Dreyer@IQVIA.com

    References:
    1. Pundi K, Perino AC, Harrington RA, Krumholz HM, Turakhia MP. Characteristics and strength of evidence of COVID-19 studies registered on ClinicalTrials.gov. Letter to the Editor. JAMA Internal Medicine. Published online July 27, 2020.
    2. www.helpstopCOVID19.com
    3. Menni C, Valdes AM, Freidin MB, Sudre CH, Nguyen LH, Drew DA, Ganesh S, Varsavsky T, Cardoso MJ, El-Sayed Moustafa JS, Visconti A, Hysi P, Bowyer RCE, Magino M, Falchi M, Wolf J, Ourselin S, Chan AT, Steves CJ, Spector TD. Real-time tracking of self-reported symptoms to predict potential COVID-19. Nature Medicine 2020; 26:1037-1040.
    4. Heads of Medicines Agencies and European Medicines Agency. HMA-EMA Joint Big Data Taskforce Summary Report. 13 February 2019. EMA/105321/2019.
    5. https://healthpolicy.duke.edu/publications/adding-real-world-evidence-totality-evidence-approach-evaluating-marketed-product
    CONFLICT OF INTEREST: None Reported
    READ MORE
    Research Letter
    July 27, 2020

    Characteristics and Strength of Evidence of COVID-19 Studies Registered on ClinicalTrials.gov

    Author Affiliations
    • 1Department of Medicine, Stanford University School of Medicine, Stanford, California
    • 2Section of Cardiovascular Medicine, Department of Internal Medicine, Yale School of Medicine, New Haven, Connecticut
    JAMA Intern Med. 2020;180(10):1398-1400. doi:10.1001/jamainternmed.2020.2904

    The coronavirus disease 2019 (COVID-19) pandemic has led to a massive activation of clinical research. The methodological strength of these studies is not well characterized but has implications for the quality of evidence produced. We evaluated the characteristics and expected strength of evidence of COVID-19 studies registered on ClinicalTrials.gov.

    Methods

    For this cross-sectional analysis, we searched ClinicalTrials.gov on May 19, 2020, using the terms COVID-19, SARS-CoV-2, 2019-nCov, 2019 novel coronavirus, and severe acute respiratory syndrome coronavirus 2 and extracted all structured data fields.1 We excluded withdrawn, suspended, terminated, or expanded-access studies. We categorized reported outcomes and graded studies using the 2011 Oxford Centre for Evidence-Based Medicine (OCEBM) level of evidence framework.2 A single reviewer (K.P.) verified studies for inclusion and removed duplicates, and 2 reviewers (A.C.P. and M.P.T.) audited results. This study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline.

    Results

    We identified 1551 studies registered from March 1, 2011, to May 19, 2020, meeting inclusion criteria: 911 (58.7%) interventional (including 664 randomized clinical trials [RCTs]) and 640 (41.3%) observational studies (Table); 1180 (76.1%) were single center. Frequently reported primary and secondary outcomes include mortality (526 [33.9%]), ventilation requirement (413 [26.6%]), and treatment complications (359 [23.1%]). Of the 1551 studies, 451 (29.1%) could potentially yield OCEBM level 2 evidence, or the highest level of individual study evidence.2

    Across 664 RCTs, the primary outcome most frequently pertained to clinical course (323 [48.6%]); 51 (7.7%) had a primary outcome of mortality, and 42 (6.3%) had a composite end point including mortality (Figure). Blinding (required for OCEBM level 2 evidence) was reported for 364 RCTs, of which 195 (29.3%) were placebo-controlled, 238 (35.8%) planned enrollment of more than 100 participants, and 113 (17.0%) reported at least 2 study centers or sites. Only 75 RCTs (11.3%) were placebo-controlled and blinded with at least 2 study centers (60 with enrollment >100 participants; 24 with >500 participants). Most RCTs evaluated drugs and biologic compounds (486 [73.2%]); 155 (23.3%), hydroxychloroquine or chloroquine; 7 (1.1%), remdesivir; 48 (7.2%), other antivirals; 21 (3.2%), tocilizumab; and 20 (3.0%), corticosteroids.

    Of the 640 observational studies, 517 (80.8%) were single center and 123 (19.2%) were multicenter, 36 of which had 10 or more centers. Eighty-seven studies (13.6%) were prospective cohort studies that could yield level 2 evidence.

    Discussion

    Although a few large multicenter trials may generate high-quality evidence, the large proportion of studies with an expected low level of evidence is concerning. Rapid dissemination of studies with low-quality evidence studies can influence public opinion, government actions, and clinical practice in potentially harmful ways,3 especially with a rising tide of COVID-19 study dissemination via preprint or other strategies ahead of peer review.

    A number of measures can mitigate these issues. Preprint results could be accompanied by transparent data sharing ahead of peer review. Rapidly deployable systems for multicenter registries and trials should be created but with an emphasis on quality, not just speed. These systems could be activated for global health crises, leading to streamlined operations for international study coordination, data sharing, and central institutional review boards. These systems can also combine and harmonize similar observational studies into large multicenter studies or embed randomization or pragmatic features when possible. The World Health Organization’s 100-country, adaptive Solidarity trial comparing 4 treatment arms uses a common data platform and operations, but results may not be available for months.4,5 Finally, we urge institutional review boards to work with investigators to ensure that experimental research involving human participants is sufficiently well designed to achieve the goal of generating clinically meaningful evidence.

    Our study has important limitations. Current regulations only require drug, device, or biological studies to register with ClinicalTrials.gov. Half of non-US studies are estimated to not be registered with ClinicalTrials.gov,6 and OCEBM is most accurately applied to completed studies.

    This cross-sectional study found that despite the marked rise in COVID-19 studies, only 29.1% of those registered in ClinicalTrials.gov have the potential to result in OCEBM level 2 evidence. Of the RCTs, only 29.3% are placebo-controlled, blinded studies. Global decline in new cases could also stall enrollment. Even before results are known, most studies likely will not yield meaningful scientific evidence at a time when rapid generation of high-quality knowledge is critical.

    Back to top
    Article Information

    Accepted for Publication: May 27, 2020.

    Corresponding Author: Mintu Turakhia, MD, MAS, Department of Medicine, Stanford University School of Medicine, 3801 Miranda Ave, Room 111C, Palo Alto CA 94304 (mintu@stanford.edu).

    Published Online: July 27, 2020. doi:10.1001/jamainternmed.2020.2904

    Author Contributions: Drs Pundi and Turakhia had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Pundi, Perino, Turakhia.

    Acquisition, analysis, or interpretation of data: All authors.

    Drafting of the manuscript: Pundi, Perino, Turakhia.

    Critical revision of the manuscript for important intellectual content: All authors.

    Statistical analysis: Pundi, Turakhia.

    Obtained funding: Turakhia.

    Administrative, technical, or material support: Harrington, Turakhia.

    Supervision: Perino, Harrington, Turakhia.

    Conflict of Interest Disclosures: Dr Pundi reported receiving research grants from the American Heart Association (AHA) outside the submitted work. Dr Perino reported receiving research support from the AHA and Bristol-Myers Squibb/Pfizer. Dr Harrington reported serving on the AHA Board of Directors (unpaid) and on the Stanford Healthcare Board of Directors from 2016 to 2018 (unpaid). Dr Krumholz reported working under contract with the Centers for Medicare & Medicaid Services to support quality measurement programs; receiving research grants/agreements through Yale University from Medtronic plc and the US Food and Drug Administration to develop methods for postmarket surveillance of medical devices, from Johnson & Johnson to support clinical trial data sharing, and from the Shenzhen Center for Health Information for work to advance intelligent disease prevention and health promotion; collaborating with the National Center for Cardiovascular Diseases in Beijing; receiving payments from the Arnold & Porter law firm for work related to the Sanofi SA clopidogrel litigation, from the Martin Baughman law firm for work related to the Cook Celect inferior vena cava filter litigation, and from the Siegfried & Jensen law firm for work related to rofecoxib (Vioxx) litigation; chairing a cardiac scientific advisory board for UnitedHealth; serving as a member of the IBM Watson Health Life Sciences Board; serving as a member of the Advisory Board for Element Science, the Advisory Board for Facebook, and the Physician Advisory Board for Aetna; and being the cofounder of HugoHealth, a personal health information platform, and Refactor Health, an enterprise health care artificial intelligence–augmented data management company. Dr Turakhia reported receiving grants from Apple Inc, Janssen Pharmaceuticals, Boehringer Ingelheim, Bristol-Myers Squibb, the AHA, and SentreHeart, personal fees from Medtronic plc, Abbott Laboratories, iBeat Inc, iRhythm Technologies, Novartis International AG, Biotronik, Sanofi-Aventis, and Pfizer, Inc, and equity from AliveCor outside the submitted work. No other disclosures were reported.

    References
    1.
    ClinicalTrials.gov. COVID-19 search details. Accessed April 8, 2020. https://clinicaltrials.gov/ct2/results/details?cond=COVID-19
    2.
    OCEBM Levels of Evidence Working Group. OCEBM levels of evidence. Published May 1, 2016. Accessed April 9, 2020. https://www.cebm.net/index.aspx?o=5653
    3.
    Kim  AHJ, Sparks  JA, Liew  JW,  et al.  A rush to judgment? rapid reporting and dissemination of results and its consequences regarding the use of hydroxychloroquine for COVID-19.   Ann Intern Med. 2020;172(12):819-821. doi:10.7326/M20-1223 PubMedGoogle Scholar
    4.
    Mullard  A.  Flooded by the torrent: the COVID-19 drug pipeline.   Lancet. 2020;395(10232):1245-1246. doi:10.1016/S0140-6736(20)30894-1 PubMedGoogle ScholarCrossref
    5.
    World Health Organization. “Solidarity” clinical trial for COVID-19 treatments. Updated June 4, 2020. Accessed May 20, 2020. https://www.who.int/emergencies/diseases/novel-coronavirus-2019/global-research-on-novel-coronavirus-2019-ncov/solidarity-clinical-trial-for-covid-19-treatments
    6.
    Banno  M, Tsujimoto  Y, Kataoka  Y.  Studies registered in non-ClinicalTrials.gov accounted for an increasing proportion of protocol registrations in medical research.   J Clin Epidemiol. 2019;116:106-113. doi:10.1016/j.jclinepi.2019.09.005 PubMedGoogle ScholarCrossref
    ×