Validation of a computerised system to calculate the sequential organ failure assessment score

Introduction

The sequential organ failure assessment (SOFA) score was introduced with the aim of quantifying the severity of illness, based on organ dysfunction, serially over time. A previous study has suggested tha the reliability and accuracy of SOFA scoring by intensive care physicians is good but there may be room for improvement [1]. Manual calculation of the score can be time consuming. We developed a computer program that derives a SOFA score from the electronic patient record, the Innovian system (Draeger, Germany). To date, no automated method for SOFA score calculation has been validated. We validated the automated SOFA score collection method in order to assess its accuracy and reliability. We also measured the time for manual collection of SOFA scores in order to assess the resource saving potential of the computer system.

Methods

Fifty patient records were selected from the database as a stratified random sample in order to represent the patient population of our teaching hospital adult ICU. Two ICU physicians calculated a total SOFA score and an individual organ score on each of the 50 patient records. A gold standard SOFA score was then generated after discussion between the two physicians with the aid of a third adjudicating ICU physician. SOFA scores generated by the computer were compared with the gold standard to assess accuracy. Reasons for inaccurate scoring were recorded.

Results

SOFA scores varied from 1 to 15 in this sample, the mean SOFA score for the Gold standard was 8.1 and the standard deviation was 2.9. The agreement between the different ratings was very good. The computer score had a better agreement with the gold standard score (Pearson correlation coefficient 0.92), compared with the individual physician scores (Pearson correlation coefficient 0.890 and 0.895). The computer calculated the SOFA score correctly in 41 cases and the physicians calculated the SOFA score correctly in 32 cases. The average time to calculate a SOFA score was 4.91 minutes and was not significantly different between the physicians.

Conclusions

The results show that this computer system is highly accurate at calculating SOFA scores from the electronic patient record and is more accurate than physicians. The time saved is considerable.