Skip to main content

Rotten and gold apples: inside and outside the gray zone of a ROC curve

The Original Article was published on 07 December 2023

The Original Article was published on 08 November 2023

Dear Editor,

We were expecting some comment about the results of our paper, since this topic has been debated for years. In the words of George Bernard Shaw “If you have an apple and I have an apple (even rotten, we add) and we exchange these apples then you and I will still each have one apple. But if you have an idea and I have an idea and we exchange these ideas, then each of us will have two ideas”. So, ideas are always welcome and we read with interest the reply from Michard and colleagues [1].

The authors describe a well-known fact. The ROC curve for PPV and SVV and their positive and negative predictive values are nearly perfect under very specific conditions, i.e., in patients undergoing non-laparoscopic surgery with a closed chest and a tidal volume of 7–9 ml/kg (and without arrhythmias). In real life, however, these represent a minority of clinical situations. We agree that the underlying studies in our metanalysis do not represent ideal conditions, however they likely reflect real-life clinical heterogeneity and they speak to the effectiveness of PPV and SPV, rather than efficacy under ideal conditions. We therefore approached this problem pragmatically and rigorously by subdividing the studies and stratifying the results according to the surgical setting [2].

We concur with Dr. Michard and co-authors that the discriminatory value of PPV and SVV is likely to be higher in the subgroup of patients meeting all the conditions conducive to its reliable use—however this data was simply not available. To support this notion, we presented pooled AUCs among patients with a closed chest and abdomen, ventilated with > 8 ml/kg tidal volume in Table 4, demonstrating an AUC close to 0.9, even in prone patients.

Regarding changes in PPV after a functional hemodynamic test, we agree with the authors that this is an interesting but different primary research question; our research group is already working on it. We also agree that PPV has practical advantages since it does not require cardiac output monitoring. Our results even suggest that PPV may be more reliable than SVV. Functional hemodynamic tests are useful and research in this field should be encouraged.

Our meta-analysis is the most comprehensive and updated reality check of the accuracy of PPV and SVV in predicting fluid responsiveness during surgery. The perioperative use of PPV and SVV is still debated despite hundreds of publications over the last decades because of a simple and unavoidable point: simplifying any physiological interaction of the human body using a number (any number) may be effective only under strict and defined conditions. Ignoring this will only result in the comparison of (rotten) apples and pears.

Threshold values of PPV were already questioned more than a decade ago by Cannesson et al., who reported an overall PPV AUC of 0.89 (95% CI 0.86–0.92) in a cohort of 413 patients during general anesthesia and mechanical ventilation in four centers (still one of the biggest trial in this field). Importantly, the average tidal volume was 7.9 ± 1.3 ml/kg body weight (so, basically 8 ml/kg), the volume expansion consisted of 500 ml colloid solution (hetastarch 6% or modified fluid gelatin) given over 10–20 min, and the hemodynamic measurements were performed within 2–5 min after volume expansion [3]. All these variables (the amount of fluid, the rate of administration, the type of fluid used and the time points evaluating the effect of the fluid challenge) may impact on the binary 1/0 fluid responsiveness response. Accordingly, studying the PPV adopting the same tidal volume, but different fluid challenge tests will inevitably lead to different results. For instance, in a previous systematic review, we showed that in the last decade fluid challenges have been infused within shorter times, as compared to the past [4].

In our metanalysis, in the studies enrolling patients with closed chest and abdomen, the pooled AUC for PPV was 0.79 (95%CI 0.73–0.84) for a threshold of 10.9%, which is increased to 0.88 (0.82–0.93) in the studies ventilating the patients with a tidal volume of > 8 ml/kg. So, also when “physiologic limitations to their use are respected”, PPV reliability may be affected by other not-PPV related mechanisms.

Finally, we would like to stress another point. At a time of exponential progress in monitoring and data management supporting systems, such as AI augmented clinical decision making, one should consider a ROC curve of 0.79 positively. This is much better than guessing or tossing a coin. However, lessons learnt from recent research in fluid administration have demonstrated that it is unwise to make a decision based on a single variable. As precisely as Michard and colleagues suggest, decisions to administer fluids should be personalized by considering clinical grounds, the assessment of the adequacy of perfusion and by testing fluid responsiveness. This should be done regardless of whether ideal conditions are fulfilled. In other words, we should search for tests that matter for patients, and not for patients that matter for tests.

Availability of data and material

Not applicable.

Abbreviations

PPV:

Pulse pressure variation

SVV:

Stroke volume variation

ROC:

Curve, receiving operating characteristics curve

AUC:

Area under the ROC curve

References

  1. Michard F, Chemla D, Teboul JL. Meta-analysis of pulse pressure variation (PPV) and stroke volume variation (SVV) studies: a few rotten apples can spoil the whole barrel. Crit Care. 2023;27(1):482.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Messina A, Caporale M, Calabro L, Lionetti G, Bono D, Matronola GM, Brunati A, Frassanito L, Morenghi E, Antonelli M, et al. Reliability of pulse pressure and stroke volume variation in assessing fluid responsiveness in the operating room: a metanalysis and a metaregression. Crit Care. 2023;27(1):431.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Cannesson M, Le Manach Y, Hofer CK, Goarin JP, Lehot JJ, Vallet B, Tavernier B. Assessing the diagnostic accuracy of pulse pressure variations for the prediction of fluid responsiveness: a “gray zone” approach. Anesthesiology. 2011;115(2):231–41.

    Article  PubMed  Google Scholar 

  4. Messina A, Calabro L, Pugliese L, Lulja A, Sopuch A, Rosalba D, Morenghi E, Hernandez G, Monnet X, Cecconi M. Fluid challenge in critically ill patients receiving hemodynamic monitoring: a systematic review and comparison of two decades. Crit Care. 2022;26(1):186.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Funding

The authors did not receive any funding for this article.

Author information

Authors and Affiliations

Authors

Contributions

AM and MC drafted the manuscript, and all authors revised the manuscript and approved the final version.

Corresponding author

Correspondence to Antonio Messina.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Competing interests

None in relation with this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Messina, A., Chew, M. & Cecconi, M. Rotten and gold apples: inside and outside the gray zone of a ROC curve. Crit Care 28, 22 (2024). https://doi.org/10.1186/s13054-023-04796-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13054-023-04796-w

Keywords