Skip to main content
  • Commentary
  • Published:

Rigorous scoping review of randomized trials in pediatric critical care highlights need for a rigorous rethink

Abstract

The randomized controlled trial (RCT) remains the highest-ranked study design when grading recommendations for clinical practice. In the previous issue of Critical Care, Duffett and colleagues published a scoping review of RCTs in pediatric critical care medicine and identified some serious gaps in the body of research underlying the field. Relatively few published RCTs were identified, and they were mostly small and potentially susceptible to bias. High patient heterogeneity, relatively low prevalence of specific disorders such as acute respiratory distress syndrome or septic shock, along with relatively low mortality rates, all make it difficult to improve this situation without the collaboration of pediatric critical care research networks internationally. Designing a robust RCT that can impact clinical practice has always been challenging. First, one must assess current clinical practice and disease prevalence, refine definitions and measurements, and pilot-test the intervention to be studied. The first step, however, is to rigorously assess what has already been done. This step will be facilitated by the now available, innovative, online, searchable repository of RCTs in pediatric critical care on the Evidence in Pediatric Critical Care website.

The practice of triaging critically ill infants, children, and adolescents into intensive care units staffed by clinicians trained to care for them is widespread. In the previous issue of Critical Care, Duffett and colleagues performed a scoping review of randomized controlled trials (RCTs) in pediatric critical care [1] to systematically assess the body of research in the field. They cast a wide net to identify all English and non-English language publications, and employed nine pairs of reviewers to evaluate each publication and then independently and in duplicate to extract the data and assess the risk of bias. After screening more than 7,500 studies, they included 248 RCTs randomly assigning more than 27,000 children from 31 countries. Comparatively, the authors note that this is a small fraction of the number of RCTs in adult critical care [2]; neonatology has at least that many systematic reviews of RCTs [3]. Unfortunately, 88% of the RCTs they found had a risk of systematic bias potentially threatening study validity [1]. Major obstacles plaguing pediatric critical care clinical research have been discussed before and include high patient heterogeneity, low prevalence of any single disorder, relatively low mortality rates, and high practice variability [4–6]. Given the numerous obstacles to performing high-quality RCTs in critically ill children, the rigorous review by Duffett and colleagues [1] highlights the need to rigorously rethink the RCT process.

Ideally, RCTs should enroll a representative sample of the population in which the results will be applied. The great majority of the trials (82%) Duffett and colleagues [1] identified were performed in a single center; the median number of centers participating in multicenter studies was only five. Sample sizes were generally small; 50% of RCTs randomly assigned fewer than 50 children. Most trials focused on intermediate or surrogate outcomes of unclear clinical importance. Only 43% of the trials reported a target sample size; 32% of these trials were stopped early before recruiting the target. To improve the ability to recruit a larger, more representative sample of children, pediatric critical care national networks have formed in the US, UK, Canada, and Australia-New Zealand and in other regions. The low frequency of disorders such as acute lung injury makes it essential that these international networks work together [7].

The only situation worse than not having RCT level evidence is to initiate an RCT, expose children to the untested intervention(s), and then abandon the study midway or not publish the findings (or both). Sadly, Shamliyan and Kane recently reported that of 2,290 intervention studies involving children identified in ClinicalTrials.gov, only 71% were completed and only 30% of these studies were ever published [8]. Bias in trial reporting may also be a problem in pediatric critical care RCTs [1]. The statistical significance of the outcome could be assessed in only approximately half of the RCTs, and of the 50% of these trials that reported significance, 93% of the time the results favored the experimental intervention [1]. Poor recruitment may underlie some of the abandoned studies. An adaptive research design using a Bayesian statistical approach, with the a priori estimate extrapolated from trials carried out in adults, has been proposed as one way to gain statistical power with fewer children enrolled [9].

Finally, RCTs are not always the optimal way to answer a clinical question. Aside from their extraordinary expense, they can mislead. If trial enrollment drags on too long, the intervention can become obsolete or the epidemiology can change. Protocol adherence suffers when hundreds of centers are involved, each enrolling few patients. Most RCTs also exclude many patients in whom the intervention would eventually be used. It has been argued that rigorous epidemiology studies can be equally important in creating a deeper understanding of the problem under study, often informing clinical practice [10, 11]. The Canadian Critical Care Trials Group programmatic approach to research lays out in detail the hard work leading up to a successful RCT [12]. Step 1 consists of systematically reviewing what is currently known and assessing current practice. Step 2 consists of assessing disease prevalence, identifying risk factors, and evaluating study definitions, metrics, and outcomes. Step 3 consists of assessing RCT feasibility by performing an observational study across potential enrollment sites and piloting the intervention. After all of these steps are completed, it is likely fewer pediatric critical care RCTs will be performed, but those that are will be of higher quality and have more potential impact.

To facilitate public access to the trials they identified, Duffett and colleagues [1] built an innovative online searchable repository [13]. Their spectacular website includes an interactive world map showing, by country, the number of RCTs, the number of children randomly assigned, and the median number of children per RCT. It is easily browsed by indication, intervention, and patient type, thus facilitating literature review and future meta-analyses. The work of Duffett and colleagues [1] is essential and will facilitate the initial steps in any program of research leading up to a successful randomized trial [12], that is, Step 1: to assess what has already been done.

Abbreviations

RCT:

Randomized controlled trial.

References

  1. Duffett M, Choong K, Hartling L, Menon K, Thabane L, Cook DJ: Randomized controlled trials in pediatric critical care: a scoping review. Crit Care 2013, 17: R256. 10.1186/cc13083

    Article  PubMed Central  PubMed  Google Scholar 

  2. Harhay MO, Wagner J, Cooney E, Bronheim RS, Gopal A, Green S, Prasad Kerlin M, Mikkelsen ME, Small D, Halpern SD: Epidemiology of published critical care randomized clinical trials 2007–2012. Am J Respir Crit Care Med 2013, 187: A1601.

    Google Scholar 

  3. Neonatal.org homepage. http://cochrane.neonatal.org

  4. Randolph AG, Lacroix J: Randomized clinical trials in pediatric critical care: rarely done but desperately needed. Pediatr Crit Care Med 2002, 3: 102-106. 10.1097/00130478-200204000-00002

    Article  PubMed  Google Scholar 

  5. Randolph AG: Randomized clinical trials in pediatric critical care. In 2001 Yearbook of Intensive Care and Emergency Medicine. Edited by: Vincent J-L. Berlin: Springer Berlin Heidelberg; 2001:676-684.

    Chapter  Google Scholar 

  6. Nicholson CE, Gans BM, Chang AC, Pollack MM, Blackman J, Giroir BP, Wilson D, Zimmerman JJ, Whyte J, Dalton HJ, Carcillo JA, Randolph AG, Kochanek PM: Pediatric critical care medicine: planning for our research future. Pediatric Crit Care Med 2003, 4: 196-202. 10.1097/01.PCC.0000059728.63798.DA

    Article  Google Scholar 

  7. Santschi M, Jouvet P, Leclerc F, Gauvin F, Newth CJL, Carroll CL, Flori H, Tasker RC, Rimensberger PC, Randolph AG: Acute lung injury in children: therapeutic practice and feasibility of international clinical trials. Pediatr Crit Care Med 2010, 11: 681-689. 10.1097/PCC.0b013e3181d904c0

    Article  PubMed  Google Scholar 

  8. Shamliyan T, Kane RL: Clinical research involving children: registration, completeness and publication. Pediatrics 2012, 129: e1291-e1300. 10.1542/peds.2010-2847

    Article  PubMed  Google Scholar 

  9. Halpern SD, Randolph AG, Angus DC: No child left behind: enrolling children and adults simultaneously in critical care trials. Crit Care Med 2009, 37: 2638-2641. 10.1097/CCM.0b013e3181a59357

    Article  PubMed Central  PubMed  Google Scholar 

  10. Shadish WR, Clark MH, Steiner PM: Can nonrandomized experiments yield accurate answers? A randomized experiment comparing random and nonrandom assignments. J Am Stat Assoc 2008, 484: 1334-1356.

    Article  Google Scholar 

  11. Benson K, Hartz AJ: A comparison of observational studies and randomized, controlled trials. N Engl J Med 2000, 342: 1878-1886. 10.1056/NEJM200006223422506

    Article  CAS  PubMed  Google Scholar 

  12. Marshall JC, Cook DJ, Canadian Critical Care Trials Group: Investigator-led clinical research consortia: the Canadian Critical Care Trials Group. Crit Care Med 2009, 37: S165-S172. 10.1097/CCM.0b013e3181921079

    Article  PubMed  Google Scholar 

  13. McMaster University homepage. http://epicc.mcmaster.ca

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Adrienne G Randolph.

Additional information

Competing interests

The author declares that she has no competing interests.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Randolph, A.G. Rigorous scoping review of randomized trials in pediatric critical care highlights need for a rigorous rethink. Crit Care 17, 1024 (2013). https://doi.org/10.1186/cc13166

Download citation

  • Published:

  • DOI: https://doi.org/10.1186/cc13166

Keywords