Skip to content

Advertisement

Critical Care

Open Access

Developing a feasible and valid scoring system for critically ill patients in resource-limited settings

Critical Care201822:2

https://doi.org/10.1186/s13054-017-1902-z

Received: 25 October 2017

Accepted: 29 November 2017

Published: 5 January 2018

The original article was published in Critical Care 2017 21:250

In the study by Haniffa et al., the authors have brilliantly proposed “TropICS” as the first multinational prognostic model for critically ill patients in resource-limited settings. The study used data from four South East Asian nations, involving a large number of unselected ICU patients for development and validation of three prognostic models. Of the three proposed models, TropICS performed better than APACHE II and SAPS II in terms of discrimination and calibration. It is interesting to see them test the three models, which varied in inclusion of clinical, laboratory and treatment variables, which were selected based on multivariate logistic regression. The models have tried to address the variable availability and affordability of the parameters measured [1].

However, it would be prudent to note that the complete case availability for APACHE II was only 15% [1]. Moreover, as diagnostic, therapeutic, and prognostic techniques in the ICU evolve over time, the scoring systems need to be updated [2]. Newer APACHE versions, such as APACHE IV, which is based on a larger database and in which the selection of variables and their weights is based on multiple logistic regression, perform better than older versions [3]. These factors together might have caused the lower performance of APACHE II compared to the other proposed models.

Disease-specific scoring systems are increasingly being used. The SOFA score has been proposed to quantify organ dysfunction, as per Sepsis-3 [4]. Some of the parameters of the SOFA score like PaO2, serum creatinine, and bilirubin level have poor availability (less than 50%) in the databases of the new models, raising questions about the feasibility and validity of using the SOFA score to quantify organ dysfunction in resource-poor settings [1]. There may be a need to develop a simpler and more feasible scoring system to recognize sepsis in resource-limited settings [5]. Laboratory variables of TropICS, blood urea and hemoglobin level, were available for only 50% of the patients [1]. Considering the wide variability of availability of resources between low and middle-income countries (LMICs) and even within the same nation, TropICS needs to be validated across these settings before assuming global applicability in places with limited resources.

Rather than depending on a single score at admission to ICU, change in score over time (like the delta SOFA score) may reflect the progression of organ dysfunction over time and can be helpful for better prognostication [2]. Future studies may attempt to explore the utility of the delta score for TropICS as well.

Authors’ response

  • Rashan Haniffa,
  • Mavuto Mukaka,
  • Sithum Bandara Munasinghe,
  • Ambepitiyawaduge Pubudu De Silva,
  • Kosala Saroj Amarasiri Jayasinghe,
  • Abi Beane,
  • Nicolette de Keizer and
  • Arjen M Dondorp
  • We would like to thank Dr. Shrestha for his insightful comments. Our systematic review attributes the variable performance of prognostic models in LMICs to three main factors: missing data and subsequent normal imputation; differing endpoints due to unknown hospital outcomes; and the inadvertent inclusion of paediatric patients [6]. Other reasons more commonly attributed to poor performance of prognostic models in such settings include differing case-mix when compared to high income countries (HICs) and a predominantly healthy and younger population. These are, however, often difficult to evaluate given the limited investment in infrastructure (for example medical registries) enabling the systematic collection of data. We agree that the reason for the poor performance of APACHE II in our dataset may be due to missing data [7]. However, for the same reason, newer and more complicated versions such as APACHE III or IV are unlikely to fare any better given their greater data collection burden. It is also important to note that such missingness is not limited to LMICs or indeed to the ICU [8]. The development of simplified models such as TropICS, R-MPM and qSOFA can be seen as attempts to overcome this impediment.

    As variability between and within countries is likely to persist, and as all models require regular validation and refinement (for example by recalibration), prognostic model selection and assessment are perhaps better guided, at least initially, by data availability; by matching models and their variants to the degree of missingness [7]. Subsequent validation using the classic statistical tests of discrimination, calibration and accuracy can then follow. Further research to investigate the impact of the extent of missingness on prognostic models and strategies to mitigate this seemingly ubiquitous problem, beyond traditional statistical methods, is ongoing in multiple LMIC settings by our group.

    In addition to the quest to refine statistical methodology to improve model performance, we recommend that the validation of such tools should be seen in the context of ‘real world performance and impact on patient outcomes’ [9]. To have greater applicability to improving the quality and equity of care provided, the priorities of clinicians, researchers and administrators in the target setting need to be better understood. Could the biggest barrier to the validity and applicability of such tools be the disconnect of the necessary stakeholders from processes established as indispensable in HICs: benchmarking, audit and quality improvement, and risk stratification for clinical trials?

    Notes

    Abbreviations

    APACHE: 

    Acute Physiology and Chronic Health Evaluation

    ICU: 

    Intensive Care Unit

    SAPS: 

    Simplified Acute Physiology Score

    SOFA: 

    Sequential Organ Failure Assessment.

    TropICS: 

    Tropical Intensive Care Score

    Declarations

    Acknowledgements

    None.

    Funding

    None.

    Availability of data and materials

    Not applicable.

    Authors’ contributions

    The author contributed to concept, designing, literature search, manuscript writing, editing and review.

    Ethics approval and consent to participate

    Not applicable.

    Consent for publication

    Not applicable.

    Competing interests

    The author declares that he/she has no competing interests.

    Publisher’s Note

    Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

    Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

    Authors’ Affiliations

    (1)
    Department of Anaesthesiology, Critical Care Unit, Tribhuvan University Teaching Hospital, Maharajgunj, Nepal

    References

    1. Haniffa R, Mukaka M, Munasinghe SB, et al. Simplified prognostic model for critically ill patients in resource limited settings in South Asia. Crit Care. 2017;21:250.View ArticlePubMedPubMed CentralGoogle Scholar
    2. Vincent JL, Moreno R. Clinical review: scoring systems in the critically ill. Crit Care. 2010;14:207.View ArticlePubMedPubMed CentralGoogle Scholar
    3. Zimmerman JE, Kramer AA, McNair DS, Malila FM. Acute physiology and chronic health evaluation (APACHE) IV: hospital mortality assessment for today’s critically ill patients. Crit Care Med. 2006;34:1297–310.View ArticlePubMedGoogle Scholar
    4. Singer M, Deutschman CS, Seymour CW, et al. The third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA. 2016;315:801–10.View ArticlePubMedPubMed CentralGoogle Scholar
    5. Shrestha GS. Composition of the sepsis definitions task force. JAMA. 2016;316:459–60.View ArticlePubMedGoogle Scholar
    6. Haniffa R, Isaam I, De Silva AP, Dondorp AM, De Keizer NF. Performance of critical care prognostic scoring systems in low and middle income countries - a systematic review. Critical Care. doi:10.1186/s13054-017-1930-8.
    7. Haniffa R, De Silva AP, Weerathunga P, Mukaka M, Athapattu P, Munasinghe S, Mahesh B, Mahipala P, De Silva T, Abayadeera A, Jayasinghe S. Applicability of the APACHE II model to a lower middle income country. J Crit Care. 2017;42:178–83.View ArticlePubMedGoogle Scholar
    8. Afessa B, Keegan MT, Gajic O, Hubmayr RD, Peters SG. The influence of missing components of the Acute Physiology Score of APACHE III on the measurement of ICU performance. Intensive Care Med. 2005;31(11):1537–43.View ArticlePubMedGoogle Scholar
    9. Baker T, Gerdin M. The clinical usefulness of prognostic prediction models in critical illness. Eur J Intern Med. 2017;45:37–40.View ArticlePubMedGoogle Scholar

    Copyright

    © The Author(s). 2018

    Advertisement