Instrument development to conduct a meta-analysis of mortality from ARDS

Recently, randomized clinical trials of ventilatory management strategies indicate a reduction in mortality associated with acute respiratory distress syndrome (ARDS). However, there is little clarity as to whether mortality has changed over time to provide a benchmark reference for measuring the effects of therapies and for identifying factors that reduce mortality. In preparation to conduct a comprehensive meta-analysis of mortality from ARDS, 14 search terms for ARDS were identified to generate titles from published, electronic, and unpublished sources. The co-investigators independently reviewed a subset of titles (1996–2000) to select abstracts about investigations of a therapy or pathophysiological mechanism likely to yield information about mortality from ARDS.

Titles were excluded from further review if the report appeared to be a single page; single case study; pediatric/neonatal or healthy subject; review or meta-analysis; study sample with chronic respiratory failure, idiopathic pneumonia syndrome, or pulmonary toxicity; or a methods paper of instrument reliability for patients with ARDS. Inter- and intra-rater agreements ranged between 98–100% across 5 years of title reviews. The co-investigators independently reviewed abstracts of acceptable titles to determine the likelihood that the manuscript would contain useful mortality data (inter-rater Kappa [corrected for chance agreement] = 0.76; intra-rater Kappa = 0.86); clarity of distinct study group(s) with ARDS (inter-rater Kappa = 0.71; intra-rater Kappa = 0.94); and sample size of greater than one subject with ARDS (inter-rater Kappa = 0.78; intra-rater Kappa = 0.79).

An instrument is being developed to assess the study quality and outcomes of each manuscript. The purpose of assessing the quality of the manuscript is to derive a total and variable-linked score by which the manuscript can be assessed for the impact of bias and precision on the meta-analysis. Four areas of the study's quality include publication demographics, study methods, statistical analysis, and presentation of results. Publication demographics include publication source, geographic location of the study, biostatistician involvement, and source of support for the study. Study methods are assessed for the design, sampling methods, description of conditions, randomization and blinding strategies, and a priori power analysis. Statistical analysis entails determining the clarity of analysis, intention to treat, compliance, and monitoring of adverse effects. Presentation of results is evaluated for the duration of enrollment, comparison of baseline characteristics and prognostic variables, and post hoc power analysis.

Inter- and intra-rater agreement of study quality are in process of being analyzed. An instrument to measure variables associated with mortality is also being developed to include general variables (ARDS definition, sample characteristics, groups compared [ARDS versus non-ARDS]), mortality linked with risk groups (sepsis/nonsepsis, direct/indirect risk), nominal/ordinal subject characteristics (gender, ARDS stage, location and cause of death), and interval/ratio subject characteristics (lung injury, illness severity). Instrument development will provide a solid methodological foundation for conducting a meta-analysis of the risks, treatment effects, and mortality associated with ARDS, using clinical studies from 1967 to the most recent findings.