Fallaces sunt rerum species et hominum spes fallunt (appearances are deceptive and betray the hopes of men)
© BioMed Central Ltd. 2013
Published: 5 September 2013
We read with interest the study by Van den Akker and colleagues  on the association between mechanical ventilation, and renal failure. We consider it is a very interesting study, but feel some comments are needed.
The core of a meta-analysis is the combination of results across different studies. A major concern about meta-analysis is the range of the studies included in the sample because the trials included may introduce large inhomogeneity. The challenge consists in deciding which set of studies is ‘combinable’ .
In the meta-analysis by Van den Akker and colleagues  the risk of bias is very high due to sampling errors of the included studies. The authors selected trials that are completely different from each other: different diseases, different clinical severity, different therapeutic treatments. Besides this, the authors neglected the results of the statistical analysis, which showed great heterogeneity within the sample. To evaluate this heterogeneity the authors used Cochran’s Q statistic, and the I2 statistic. Both the Q test and I2 statistic of all included patients were indicative of high heterogeneity .
When the P-value (heterogeneity test) is significant, two approaches are possible: first, avoid summarizing the results and look for reasons for the heterogeneity; second, utilize ‘the random effects model’. Van den Akker and colleagues chose the latter approach, failing to check whether the heterogeneity was due to differences in the clinical studies, methodological issues, and/or publication bias. As stated by DerSimonian and Laird , the ‘random effects model is not a cure for difficulty in generalising the results of meta-analysis to the real-world situation’. From our point of view, the methodological errors of the meta-analysis of Van den Akker and colleagues produce misleading results.
- Van den Akker JPC, Egal M, Groeneveld AB: Invasive mechanical ventilation as a risk factor for acute kidney injury in the critically ill: a systematic review and meta-analysis. Crit Care 2013, 17: R98. 10.1186/cc12743PubMed CentralView ArticlePubMedGoogle Scholar
- Ades AE, Higgins JPT: The interpretation of random-effect meta-analysis in decision models. Med Decis Making 2005, 25: 646-654. 10.1177/0272989X05282643View ArticlePubMedGoogle Scholar
- Higgins JPF, Thompson SG: Quantify heterogeneity in meta-analysis. Stat Med 2002, 21: 1539-1558. 10.1002/sim.1186View ArticlePubMedGoogle Scholar
- DerSimonian R, Laird N: Meta-analysis in clinical trials. Contr Clin Trials 1986, 7: 177-188. 10.1016/0197-2456(86)90046-2View ArticleGoogle Scholar