Skip to main content
Download PDF

Prediction of cardiac arrest in critically ill patients presenting to the emergency department using a machine learning score incorporating heart rate variability compared with the modified early warning score

Critical Care volume 16, Article number: R108 (2012) Cite this article

Abstract

Introduction

A key aim of triage is to identify those with high risk of cardiac arrest, as they require intensive monitoring, resuscitation facilities, and early intervention. We aim to validate a novel machine learning (ML) score incorporating heart rate variability (HRV) for triage of critically ill patients presenting to the emergency department by comparing the area under the curve, sensitivity and specificity with the modified early warning score (MEWS).

Methods

We conducted a prospective observational study of critically ill patients (Patient Acuity Category Scale 1 and 2) in an emergency department of a tertiary hospital. At presentation, HRV parameters generated from a 5-minute electrocardiogram recording are incorporated with age and vital signs to generate the ML score for each patient. The patients are then followed up for outcomes of cardiac arrest or death.

Results

From June 2006 to June 2008 we enrolled 925 patients. The area under the receiver operating characteristic curve (AUROC) for ML scores in predicting cardiac arrest within 72 hours is 0.781, compared with 0.680 for MEWS (difference in AUROC: 0.101, 95% confidence interval: 0.006 to 0.197). As for in-hospital death, the area under the curve for ML score is 0.741, compared with 0.693 for MEWS (difference in AUROC: 0.048, 95% confidence interval: -0.023 to 0.119). A cutoff ML score ≥ 60 predicted cardiac arrest with a sensitivity of 84.1%, specificity of 72.3% and negative predictive value of 98.8%. A cutoff MEWS ≥ 3 predicted cardiac arrest with a sensitivity of 74.4%, specificity of 54.2% and negative predictive value of 97.8%.

Conclusion

We found ML scores to be more accurate than the MEWS in predicting cardiac arrest within 72 hours. There is potential to develop bedside devices for risk stratification based on cardiac arrest prediction.

Introduction

In the emergency department (ED), triage is used to assess the severity of patients' conditions and to assign appropriate treatment priorities. This clinical process entails the rapid screening of large numbers of patients to assess severity and assign treatment. Risk stratification for cardiac arrest and other adverse cardiac outcomes plays an essential role in the management of chest pain patients in the ED [1]. Medical decisions for disposition as well as the required level of intensive monitoring rest on this perceived risk [2]. Risk stratification is a necessity because medical resources are never sufficient for all patients to be attended instantaneously in busy EDs and hospitals, with limited numbers of doctors, nurses, monitored beds, resuscitation facilities, intensive care beds, operating theaters, and so forth. Quick identification of patients of higher severity, who would more urgently need and potentially benefit from such resources, is thus important.

Current risk-stratification systems are based on clinical judgment and traditional vital signs including heart rate, respiratory rate, blood pressure, temperature, and pulse oximetry [3]. Unfortunately vital signs have not been shown to correlate well with short-term or long-term clinical outcomes [4]. The modified early warning score (MEWS) is one such widely used tool (Table 1). The MEWS is based on physiological parameters: systolic blood pressure, pulse rate, respiratory rate, temperature and AVPU score (A for 'alert', V for 'reacting to vocal stimuli', P for 'reacting to pain', U for 'unconscious'). We selected the MEWS as our comparator tool because it is widely used in the UK and in Commonwealth countries to identify patients at risk of deterioration, and raised MEWS values are associated with increased mortality [5]. The MEWS can be relatively quickly calculated during triage, without the need for laboratory test results, for example. Other studies carried out in the UK have shown good results in predicting poor outcomes in their patient groups [5–7]. However, the assessment of the AVPU score is a relatively subjective element in the scoring. Also, the range of sensitivities and specificities are dependent on the cutoff score used and the MEWS requires some training to be accurate.

Table 1 Modified early warning score
Full size table

Heart rate variability (HRV) is a non-invasive measurement for investigating autonomic influence on the cardiovascular system that has generated significant interest in recent scientific literature [8]. HRV may be defined as the change in the time interval between heartbeats, from beat to beat. HRV is controlled by the autonomic nervous system, including the sympathetic nervous system and the parasympathetic nervous system [9, 10]. There is recognition of a significant relationship between the autonomic nervous system and cardiovascular mortality, including sudden cardiac death [11, 12]. Recent studies have found strong associations between HRV from short-term (2 to 30 minutes) electrocardiogram (ECG) recordings and post-acute myocardial infarction mortality [13, 14]. These associations suggest that short-term HRV measurements may serve as a rapid risk-stratification tool for adverse cardiac events.

Machine learning (ML) is based on the way the human brain approaches pattern recognition tasks, providing an artificial intelligence-based approach to solve classification problems. A model is learned during the training process using previously known input-output pairs. The trained model is then tested with new data. Various ML topologies are available, including single-layer and multi-layer feedforward networks. ML adjusts weights of hidden layers during training to minimize an error function [15].

In this study, we aim to validate a novel ML score incorporating HRV for risk stratification of critically ill patients presenting to the ED by comparing the area under the curve, sensitivity and specificity for prediction of cardiac arrest with the MEWS.

Materials and methods

Study design

We conducted a prospective, nonrandomized, observational cohort study, looking at critically ill patients attended by the Singapore General Hospital ED. Singapore General Hospital is the oldest and largest acute tertiary hospital in Singapore. The hospital accounts for about one-third of all acute-care public-sector beds and about one-quarter of acute beds nationwide. It is a Level 1 Trauma Centre for Singapore. Annually, about 60,000 patients are admitted to its wards and another 600,000 patients are attended to in its Specialist Outpatient Clinics. The ED sees between 300 and 500 patients a day.

All public hospitals in Singapore use a national Patient Acuity Category Scale (PACS) for triage at the ED. PAC 1 patients are the most critically ill and would therefore be required to be attended to without delay. They would most probably require maximum allocation of staff and equipment resources for initial management. The severity of their symptoms requires very early attention, failing which early deterioration of their medical status is likely. PAC 2 patients are nonambulant and would appear to be in a stable state on initial cardiovascular examination and are not in danger of imminent collapse. PACS 3 patients are ambulant and PACS 4 patients are nonemergencies. PACS is a symptom-based triage system and does not have strict physiological criteria (for example, vital sign cutoff values).

At Singapore General Hospital ED, all patients are initially triaged by a nurse, and those with airway, breathing and circulation problems, or those thought to be possibly unstable and needing close monitoring, are routinely put on ECG monitoring using the LIFEPAK® 12 defibrillator/monitor (Physio-Control, Redmond, WA, USA). These would be PACS 1 patients and some PACS 2 patients.

Patient recruitment and eligibility

All patients older than 18 years of age requiring continuous ECG monitoring with PACS 1 or PACS 2 were eligible. Patients in nonsinus rhythm (asystole, supraventricular arrhythmias, ventricular arrhythmias, complete heart block, and pacemaker rhythm) were excluded because HRV metrics are not reliable for nonsinus rhythms. Patients who were subsequently discharged against medical advice or transferred to another hospital for care were considered lost to follow-up and excluded from the study as clinical outcomes could not be determined. We also excluded cases with a high percentage of artifacts, nonsinus beats, and ectopics combined together (> 30% of recorded tracing); cases with ≤ 30% of artifacts and so forth were included, but the nonsinus segments of the tracings were trimmed off. Patients were only recruited during office hours. The initial set of vital signs and HRV parameters obtained during triage was recorded for this study. HRV recordings ranged from 5 to 30 minutes. Ethics approval was obtained from the Singhealth Centralised Institutional Review Board (CIRB Approval No. 2006/018/C) with a waiver of patient consent for the study. Patients were recruited from June 2006 to June 2008.

Hospital outcomes

The primary outcome was cardiac arrest within 72 hours of presentation to the ED. The event of cardiac arrest was defined as sudden unexpected death or a resuscitation event requiring cardiopulmonary assistance (chest compressions and/or defibrillation). This assistance was thought to probably reflect a primary cardiac event/etiology. Information regarding the nature of death or the resuscitation event was extracted from clinical notes. The secondary outcome was death after admission (in-hospital death during current admission, including within 72 hours). This endpoint would have included patients dying from primary cardiac as well as noncardiac etiologies. Patients were followed up until discharge or in-hospital death. Information regarding the nature of death was extracted from clinical notes. For patients who were discharged before 72 hours, electronic medical records providing information on admission to all public hospitals in Singapore were reviewed for study outcomes.

Data collection and processing

ECG tracings (long lead I, II, III and 12-lead ECG data) obtained during initial presentation from critically ill patients on a LIFEPAK 12 defibrillator/monitor were downloaded using the CODE-STAT Suite data review software (version 5.0; Physio-control). Lead II ECGs sampled at 125 Hz were extracted as text files for HRV analysis using CODE-STATâ„¢ and proprietary ECG extraction software (Physio-Control); 125 Hz is the sampling rate used by the defibrillator monitor. Since we are primarily looking at the QRS complexes and not interested in high-frequency features of the ECG, this is a sufficient rate of digitization. Cases with ECG recordings were prospectively identified and had identity confirmed by querying ED charts and records. A minimum ECG recording of 5 minutes is required in order to accurately calculate HRV metrics.

The ECG records were converted into text (ASCII) files using an extraction program available with CODE-STAT. The processing program was embedded with a MATLAB code (R2009a; The Mathworks, Natick, Massachusetts, USA), which was used to process the ECG signals to obtain the HRV variables (see Figure 1), in accordance with the guidelines outlined by the Taskforce of the European Society of Cardiology [16]. The raw ECG data were first preprocessed to reduce the effects of noise and artifacts using a 5 to 28 Hz band-pass filter. This frequency range has been found to enhance the QRS complex against the background noise for easier peak detection [17]. A modified threshold-plus-derivative method was used to detect the QRS complexes, and all ectopics and other nonsinus beats were excluded in accordance with the guidelines outlined by the Taskforce of the European Society of Cardiology [16], using an automatic detection algorithm. RR intervals were then calculated based on the sinus rhythm. The beat detection and labeling techniques have previously been validated against manually annotated data from the MIT-BIH database [18] and have been found to perform with high accuracy [19]. Ectopic beats were identified by the size and shape of the QRS complexes as well as the distances between successive beats. The height of the QRS complex, width, and RR interval were also considered. In addition, atrial fibrillation was identified manually by study engineers during retrospective verification of ECGs.

Figure 1
figure 1

List of heart rate variability parameters.

Full size image

The ECG tracings were then analyzed for heart rate variability, with both time-domain and frequency-domain analyses. Other variables collected were age, gender, medical history including ischemic heart disease, diabetes mellitus and chronic renal failure, heart rate, blood pressure, respiratory rate, Glasgow Coma Scale (GCS), etiology, and medication history. Vital signs (heart rate, blood pressure, and oxygen saturation (SpO2)) were measured using the Propaq CS Monitor (Welch Allyn, Skaneateles Falls, NY, USA) vital signs monitor in the ED. The GCS and respiratory rate were recorded at the time of vital sign measurement. AVPU scores were recorded at triage. Tympanic temperatures of the patients were taken using a tympanic thermometer. AVPU scores were scored according to the best response during data collection. The collected data were used to calculate a MEWS for each patient recruited.

HRV variables measured included time-domain, frequency-domain, and geometric parameters. The frequency-domain parameters were calculated based on estimates of power spectral density, obtained using the Lomb-Scargle periodogram that is commonly used for unevenly sampled sequences. Use of the Lomb-Scargle periodogram eliminates the need for interpolation or resampling of the sequences [20, 21].

Machine learning score prediction

A ML-based prediction model - utilizing age, HRV parameters, and vital signs - was proposed to compute risk score on patient's hospital outcome [22]. This model was run on a MATLAB code (R2009a; The Mathworks). In contrast to traditional mathematical logistic regression approaches, this is a multivariate, nonparametric, black-box approach. This approach overcomes problems faced by traditional statistical models of colinearity and overfitting. Assuming that each patient's data can be represented as a vector of HRV parameters and vital signs, the scoring system is built based on the calculation of geometric distances among a set of feature vectors (that is, multiple patients). Classifier selection plays a key role in building an efficient prediction system. In this study, the support vector machine was adopted to map feature vectors onto a higher dimensional space and find an optimal pattern-separating hyperplane [23, 24]. The support vector machine has shown satisfactory performances in many areas including ECG beat classification [25], EEG analysis [26], and text classification [27].

The calculation of the ML score is straightforward. First, cluster centers of both positive and negative samples are calculated in Euclidean space, where positive samples are patients with cardiac arrest or death as outcomes and negative samples are patients without the above outcomes. A score is then computed by evaluating Euclidean distances between a patient's data and both cluster centers. Last, the risk score is fine-tuned through a novel imbalanced learning strategy. If the predicted outcome is positive, the risk score will be increased.

As shown in Table 2 the database consists of a majority group of normal samples and a minority group of samples with abnormal outcomes (cardiac arrest or death); that is, the dataset is imbalanced. Common ML algorithms cannot be directly implemented on this imbalanced database for score tuning, because the majority class will dominate the learning process and leads to poor generalization performance on new patients from the minority class. The solution to handling data imbalance is to create a decision ensemble [28]. Our method partitions the samples of majority class into N nonoverlapped groups, with each group joined by minority samples. By doing so, N balanced datasets are created, on which a prediction model is trained to distinguish minority and majority classes. The ML algorithm was trained and validated using a leave-out-one strategy.

Table 2 Baseline characteristics of study patients
Full size table

Statistical analyses

Continuous variables are presented as means (standard deviation) or medians (interquartile range) and were analyzed using a two-tailed Student's t test and the Wilcoxon rank-sum test, respectively. Categorical variables are presented as numbers (percentage) and were analyzed using the chi-square test or Fischer's exact test when appropriate.

The ML score and the MEWS were calculated for all patients and analyzed for a significant association between the scores and the incidence of cardiac arrest or death, and adverse cardiac events. Receiver operating characteristic (ROC) curves were based on the continuous measurements of the ML score and the ordinal measurements of the MEWS.

Patients were categorized into low, intermediate, and high risk groups according to their ML scores, based on review of the data (selecting the cutoff values that provided the best discrimination): low risk, ML score 0 to 40; intermediate risk, ML score 41 to 60; and high risk, ML score 61 to 100.

The area under the receiver operating characteristic curve (AUROC) for the ML score and the MEWS was calculated and compared using a z-statistic made from the difference between the two AUROCs divided by the standard error of the difference in the AUROCs [29]. The confidence interval (CI) for reporting the difference between the two AUROCs was also derived. Statistical comparison tests between the ML score and the MEWS for sensitivity and specificity were done by applying the McNemar test to the disease group (comparison of sensitivities) and the nondiseased group (comparison of specificities) [30]. Similarly, statistical comparison tests between ML and the MEWS for the predictive values were done using the above same method [30], whereby the test statistics had a chi-square distribution with one degree of freedom. A statistical comparison test for the likelihood ratio of a positive test was not performed, however, because no well-established method of comparison was found in the literature. Whenever possible, the 95% CI for the difference in diagnostic value between the two scoring methods was provided. For the predictive values, the CI of the difference was not readily computable by well-established methods, but significance tests were carried out for these comparisons. Moreover, separate CIs of the predictive value within each scoring method are presented. A general rule of thumb is that CIs can overlap as much as 29% and the statistics can still be significantly different (see [31] chapter 2.6: overlapping confidence intervals do not imply nonsignificance). The ML score was further categorized into low, intermediate, and high risk scores and was tested for a significant relationship with the rates of cardiac arrest or death. Optimum cutoff points were determined using sensitivity and specificity analysis.

Unless otherwise specified, P < 0.05 was considered to indicate statistical significance. All data were stored with Excel (Microsoft Office 2007; Microsoft, Redmond, WA, USA) and imported into SPSS software (version 17.0; SPSS Inc., Chicago, IL, USA) and STATA software (version 11.1; STATA Corporation, College Station, TX, USA) for statistical analysis.

Results

Baseline characteristics

A total of 1,025 ECG tracings were collected during this period. Out of these tracings, 100 were excluded due to a high percentage of artifacts, nonsinus beats, ectopics, or missing data. A total of 925 patients were recruited during the study. Table 2 shows the characteristics of the study patients. The diagnosis grouping shown was based on the admitting emergency physician clinical diagnosis. The largest diagnosis grouping is the cardiovascular group at 468 (50.6%), followed by the respiratory group at 147 (15.9%).

Outcomes

Forty-three (4.6%) of the total sample developed cardiac arrest within 72 hours, while 86 (9.3%) died after admission (including those deaths within 72 hours). The respiratory diagnosis group had the largest number of primary outcomes at 10 (23.3%), followed by cardiovascular and cancer groups both at nine incidences (20.9%). Both the gastrointestinal and renal groups did not have any patients with primary outcome (cardiac arrest). The respiratory diagnosis group has the largest number of secondary outcomes at 19 (22.1%), followed by the cardiovascular diagnosis group at 18 (20.9%).

Table 3 shows the relationship of the predictor factors with the outcome of cardiac arrest within 72 hours and death after admission. Those factors found to have significant association (P < 0.05) with the primary outcome included the GCS, pulse rate, respiratory rate, SpO2, aRR, avHR, sdHR, RR triangular index, LS-VLF power, LS-HF power, LS-LF norm, LS-HF norm, and MEWS. Those factors found to have significant association (P < 0.05) with the secondary outcome included the GCS, respiratory rate, SpO2, aRR, avHR, RMSDD, RR triangular index, TINN, LS-VLF power, LS-HF power, LS-LF norm, LS-HF norm, LF/HF ratio, and MEWS.

Table 3 Measurements of MEWS, vital signs and heart rate variability of study patients
Full size table

The ROC and AUCs of the ML score and the MEWS for predicting cardiac arrest within 72 hours or death after admission are illustrated in Figure 2 and 3, respectively.

Figure 2
figure 2

Machine learning score and modified early warning score predicting cardiac arrest within 72 hours. Receiver operating characteristics (ROC) curve analysis of the machine learning (ML) score and the modified early warning score (MEWS) in predicting cardiac arrest within 72 hours.

Full size image
Figure 3
figure 3

Machine learning score and modified early warning score predicting death within 72 hours. Receiver operating characteristics (ROC) curve analysis of machine learning (ML) score and the modified early warning score (MEWS) in predicting death after admission.

Full size image

Eighty-nine patients (9.6%), 576 patients (62.3%), and 260 patients (28.1%) were in the low, intermediate, and high risk ML score groups, respectively. Rates of cardiac arrest within 72 hours were 0%, 1.6% (95% CI: 6.59 to 9.79), and 13.1% (95% CI: 1.75 to 24.45) in the low, intermediate, and high risk groups, respectively, as shown in Figure 4. Rates of death after admission were 2.3% (95% CI: 18.48 to 23.08), 29.1% (95% CI: 11.30 to 46.90), and 68.6% (95% CI: 56.76 to 80.44) in the low, intermediate, and high risk groups, respectively.

Figure 4
figure 4

Rates of cardiac arrest within 72 hours and death in relation to machine learning score. *, **.

Full size image

Table 4 shows the sensitivity, specificity, positive predictive values, and negative predictive values for the ML score and the MEWS for predicting cardiac arrest within 72 hours or death after admission. The AUROC of the ML score was higher compared with the MEWS for the primary outcome of cardiac arrest (0.781 vs. 0.680, difference in AUROC: 0.101, 95% CI: 0.006 to 0.197; P = 0.037) but not for the secondary outcome of death. For prediction of cardiac arrest within 72 hours after presentation, the sensitivity and specificity of the ML score were 81.4 and 72.3, respectively, compared with sensitivity and specificity of the MEWS being 74.4 and 54.2, respectively (difference in sensitivity: 7.0, 95% CI: -11.1 to 21.9; and difference in specificity: 18.1, 95% CI: 14.3 to 22.0). Specificity for cardiac arrest but not sensitivity was thus significantly higher in ML compared with the MEWS. The positive predictive value of the ML score was higher (12.5, 95% CI: 9.0 to 17.1) compared with the positive predictive value of the MEWS (7.4, 95% CI: 5.3 to 10.3; P < 0.001). The likelihood ratio of a positive test for the ML score was higher (2.94, 95% CI: 2.46 to 3.52) compared with the likelihood ratio of the MEWS (1.62, 95% CI: 1.34 to 1.96). As for prediction of death after admission, the specificity of the ML score (73.9) was higher compared with the specificity of the MEWS (55.7) (difference in specificity: 18.2; 95% CI: 14.3 to 22.2). The positive predictive value for the ML score (21.5, 95% CI: 16.9 to 26.9) was higher compared with the positive predictive value of the MEWS (14.7, 95% CI: 11.5 to 18.4; P < 0.001). The likelihood ratio of the ML score (2.67, 95% CI: 2.23 to 3.20) was higher compared with the likelihood ratio of the MEWS at (1.68, 95% CI: 1.45 to 1.94).

Table 4 Discriminatory values of the machine learning score and the modified early warning score
Full size table

Discussion

The results of this study showed that a ML score incorporating vital signs and HRV parameters is more predictive of cardiac arrest within 72 hours of presentation to the ED compared with the MEWS. Categorization of patients into low, intermediate, and high risk groups according to the ML score is a useful predictor of risk for cardiac arrest and death. The ML score represents a non-invasive and objective risk-stratification tool that can be determined immediately at presentation to the ED. As diagnosis in the ED setting is often time dependent, medical decisions in the ED (disposition, level of monitoring, aggressive management) are often made based on risk assessment, rather than on diagnosis. We believe this is the first study to show the potential of a ML model incorporating age, vital signs, and HRV for predicting cardiac arrest and death.

Our study also indicates that HRV measured from short-term ECG recordings (5 to 30 minutes), when combined with vital signs, provides a useful tool for risk stratification in the ED. In this study, depressed HRV parameters were associated with early (72 hours) adverse cardiac events and death after admission (Table 3). This is consistent with the findings of previous studies that suggest short-term measurement of frequency-domain HRV parameters is strongly associated with cardiac death and mortality [11–13, 32].

Decreased HRV has been found to predict increased mortality in the older patient [33], and for coronary artery disease [14, 34], post-myocardial infarction [11], congestive heart failure [35], and dilated cardiomyopathy [36]. Altered spectral HRV analysis has been found to be an indicator of severity in congestive heart failure [37], hypertension [38], coronary artery disease [39], angina [40], myocardial infarction [41], hypovolemia, hypoxia [42], chronic renal failure [43], and diabetes mellitus [44]. Decreased HRV has also been found in ICU patients following head trauma [45–49], sepsis [50], and septic shock [51, 52]. HRV has also been used as a marker of severity in ED patients with sepsis [53]. Depressed HRV may reflect a decrease in vagal activity directed to the heart that leads to prevalence of sympathetic mechanisms [54], and therefore to cardiac instability, which might explain the higher risk of arrhythmic deaths [14, 55]. However, the true sympatho-vagal correlates of HRV and the mechanisms behind reduced HRV still remain largely unknown [56].

In our previous research, we proposed using a combination of age, HRV measures, and vital signs as a predictor of patient outcomes and demonstrated that the combined features present significant improvements to predictive accuracy, sensitivity, and specificity compared with using HRV alone [22, 57]. As we can see from Table 3 not all of the vital signs were highly predictive when used in isolation. HRV parameters also tend to be highly correlated. By using a ML approach, we were able to overcome some of these limitations as well as the overfitting associated with traditional statistical methods [58]. We have investigated an extreme learning machine and a support vector machine with different activation/kernel functions as classifiers, and found that the linear support vector machine is able to provide the highest confidence in categorizing patients into two outcomes: death and survival. Furthermore, we have also presented a new segment-based decision-making strategy for outcome prediction [22].

Limitations

Several limitations are inherent in our study. This study was carried out in a single-center study at a tertiary teaching hospital in Singapore and the results may not be generalizable to other settings. However, we are confident that the current dataset accurately represents the management of ED patients in our hospital.

The diverse types of conditions in the patients recruited may have different effects on the MEWS and ML scores. The main diagnosis groupings are cardiovascular, respiratory, neurological, gastrointestinal, renal, endocrine, infectious disease, vascular, trauma, cancer, and others. The different diagnosis groupings may cause inaccuracies in predicting the ML score because the HRV in noncardiovascular conditions may differ from that in cardiovascular-related conditions. In future study, therefore, all of the different subgroups should be analyzed separately.

Another limitation of our study is that while the ML score has been shown to have good internal validity, there is a need for external validation of the score for routine clinical use.

One of the exclusion criteria of this study was the exclusion of patients in nonsinus rhythm. It remains to be seen whether the ML score will remain a good predictor of adverse outcomes in patients with irregular heart rhythms. Previous studies in patients with atrial fibrillation have found an association between HRV and an increased risk for cardiac death [59], as well as recurrence of atrial fibrillation [60]. Patients in atrial fibrillation, however, only represent a minority of patients, estimated to be approximately 2 to 4% in patients between ages 60 and 79 [61] and < 1% in patients below the age of 55 [61]. Another limitation of our study is the lack of follow-up for patients discharged from the ED. Electronic medical records of patients that were discharged before 72 hours were also checked for any admissions to public hospitals in Singapore to minimize missed cases of adverse cardiac events.

The ML score is also based on a single ECG recording. Although this represents a rapid method for risk stratification, not much is known about the change of HRV variables over time. Studies have suggested that acute changes in HRV occur before the onset of ventricular tachycardia [62–65]. Changes over time in HRV have also been found to occur in the early phase of recovery after myocardial infarction [66–68]. Serial analyses of changes in HRV were not performed in our study and should be investigated for in follow-up studies.

Future studies

The results of this study should be validated with a larger sample size, in view of the rare outcome of cardiac arrest within 72 hours or death. Further studies should also be carried out to validate the ML score in a prospective series in patients with different diagnosis groupings.

We have since developed a laptop-based prototype to acquire real-time signals and to process and analyze HRV parameters. This device incorporates ECG and other vital signs such as blood pressure, pulse oximetry, and respiratory rate, together with clinical data, for instantaneous, intelligent prediction of cardiac arrest and mortality using neural networks. In the future, we aim to prospectively validate the prediction scores generated by our device with critically ill patients, including the assessment of the effect of ongoing treatment on our prediction index and survival. Further development is also needed to produce a stand-alone device, ready for clinical use and possible clinical trials. We believe that there exists potential for the development of bedside devices capable of real-time monitoring of HRV, which may help physicians to identify patients at high risk for cardiac arrest and death.

Conclusion

In critically ill patients presenting to the ED, we found ML scores to be more accurate than the MEWS in predicting cardiac arrest within 72 hours. The results of this study also suggest that initial short-term HRV measurements, in addition to vital signs, may play a role for early, rapid, and objective risk stratification of patients during triage.

Key messages

  • We determined the relationship of the predictor factors with the outcome of cardiac arrest within 72 hours and death after admission. Significant association (P < 0.05) with the primary outcome included the GCS, pulse rate, respiratory rate, SpO2, aRR, avHR, sdHR, RR triangular index, LS-VLF power, LS-HF power, LS-LF norm, LS-HF norm, and MEWS.

  • Those factors found to have significant association (P < 0.05) with the secondary outcome included the GCS, respiratory rate, SpO2, aRR, avHR, RMSDD, RR triangular index, TINN, LS-VLF power, LS-HF power, LS-LF norm, LS-HF norm, LF/HF ratio, and MEWS.

  • The AUROCs of the ML score for both primary and secondary outcomes (0.781 and 0.741, respectively) are higher compared with those for the MEWS (0.680 and 0.693, respectively).

  • The sensitivity and specificity of the ML score are 81.4 (95% CI: 66.1 to 91.1) and 72.3 (95% CI: 69.2 to 75.2), respectively; both are higher compared with the sensitivity and specificity of MEWS (74.4, 95% CI: 58.5 to 86.0; and 54.2, 95% CI: 50.8 to 57.5, respectively).

  • In critically ill patients presenting to the ED, we found ML scores to be more accurate than the MEWS in predicting cardiac arrest within 72 hours and death.

Abbreviations

AUROC:

area under the receiver operating characteristic curve

AVPU:

A for 'alert', V for 'reacting to vocal stimuli', P for 'reacting to pain', U for 'unconscious'

CI:

confidence interval

ECG:

electrocardiogram

ED:

emergency department

GCS:

Glasgow Coma Scale

HRV:

heart rate variability

MEWS:

modified early warning score

ML:

machine learning

PACS:

Patient Acuity Category Scale

ROC:

receiver operating characteristic

SpO2:

oxygen saturation.

References

  1. Goldman L: Using prediction models and cost-effectiveness analysis to improve clinical decisions: emergency department patients with acute chest pain. Proc Assoc Am Phys 1995, 107: 329-333.

    CAS  PubMed  Google Scholar 

  2. Peacock WF, Soto-Ruiz KM: Risk stratification for suspected acute coronary syndromes and heart failure in the emergency department. Acute Cardiac Care 2009, 11: 138-145. 10.1080/17482940902989068

    Article  PubMed  Google Scholar 

  3. Gottschalk SB, Wood D, DeVries S, Wallis LA, Bruijns S: The Cape Triage Score: a new triage system South Africa. Proposal from the Cape Triage Group. Emerg Med J 2006, 23: 149-153. 10.1136/emj.2005.028332

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  4. Subbe CP, Slater A, Menon D, Gemmell L: Validation of physiological scoring systems in the accident and emergency department. Emerg Med J 2006, 23: 841-845. 10.1136/emj.2006.035816

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  5. Subbe CP, Kruger M, Rutherford P, Gemmel L: Validation of a modified Early Warning Score in medical admissions. QJM 2001, 94: 521-526. 10.1093/qjmed/94.10.521

    Article  CAS  PubMed  Google Scholar 

  6. Goldhill DR, McNarry AF, Mandersloot G, McGinley A: A physiologically-based early warning score for ward patients: the association between score and outcome. Anaesthesia 2005, 60: 547-553. 10.1111/j.1365-2044.2005.04186.x

    Article  CAS  PubMed  Google Scholar 

  7. Paterson R, MacLeod DC, Thetford D, Beattie A, Graham C, Lam S, Bell D: Prediction of in-hospital mortality and length of stay using an early warning scoring system: clinical audit. Clin Med 2006, 6: 281-284.

    Article  CAS  Google Scholar 

  8. Seely AJ, Macklem PT: Complex systems and the technology of variability analysis. Crit Care 2004, 8: R367-R384. 10.1186/cc2948

    Article  PubMed Central  PubMed  Google Scholar 

  9. Cowan MJ: Measurement of heart rate variability. West J Nurs Res 1995, 17: 32-48. 10.1177/019394599501700104

    Article  CAS  PubMed  Google Scholar 

  10. Terathongkum S, Pickler RH: Relationships among heart rate variability, hypertension, and relaxation techniques. J Vasc Nurs 2004, 22: 78-82. 10.1016/j.jvn.2004.06.003

    Article  PubMed  Google Scholar 

  11. Carpeggiani C, L'Abbate A, Landi P, Michelassi C, Raciti M, Macerata A, Emdin M: Early assessment of heart rate variability is predictive of in-hospital death and major complications after acute myocardial infarction. Int J Cardiol 2004, 96: 361-368. 10.1016/j.ijcard.2003.07.023

    Article  PubMed  Google Scholar 

  12. Bigger JT Jr, Fleiss JL, Steinman RC, Rolnitzky LM, Kleiger RE, Rottman JN: Frequency domain measures of heart period variability and mortality after myocardial infarction. Circulation 1992, 85: 164-171. 10.1161/01.CIR.85.1.164

    Article  PubMed  Google Scholar 

  13. Bigger JT, Fleiss JL, Rolnitzky LM, Steinman RC: The ability of several short-term measures of RR variability to predict mortality after myocardial infarction. Circulation 1993, 88: 927-934. 10.1161/01.CIR.88.3.927

    Article  CAS  PubMed  Google Scholar 

  14. Fei L, Copie X, Malik M, Camm AJ: Short- and long-term assessment of heart rate variability for risk stratification after acute myocardial infarction. Am J Cardiol 1996, 77: 681-684. 10.1016/S0002-9149(97)89199-0

    Article  CAS  PubMed  Google Scholar 

  15. Azuaje F, Dubitzky W, Lopes P, Black N, Adamson K, Wu X, White JA: Predicting coronary disease risk based on short-term RR interval measurements: a neural network approach. Artif Intell Med 1999, 15: 275-297. 10.1016/S0933-3657(98)00058-X

    Article  CAS  PubMed  Google Scholar 

  16. Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology: Heart rate variability. Standards of measurement, physiological interpretation, and clinical use. Eur Heart J 1996, 17: 354-381. 10.1093/oxfordjournals.eurheartj.a014868

    Article  Google Scholar 

  17. Kohler BU, Hennig C, Orglmeister R: The principles of software QRS detection. IEEE Eng Med Biol Mag 2002, 21: 42-57. 10.1109/51.993193

    Article  PubMed  Google Scholar 

  18. MIT-BIH Arrhythmia Database Directory[http://www.physionet.org/physiobank/database/mitdb/]

  19. Padmanabhan P, Lin Z, Ong MEH, Ser W, Huang G-B: Automatic extraction of HRV sequences from noisy ECG data for reliable analysis and telediagnosis. In Proceedings of the 3rd IASTED International Conference on Telehealth; 2007 Telehealth '07 The Third IASTED International Conference on Telehealth, May 31- June 1, 2007. Montreal, Quebec, Canada, ACTA Press Anaheim, CA, USA; 2007:120-123.

    Google Scholar 

  20. Thong T, Yung IO, P Zajdel DP, Ellingson RM, McNames J, Aboy M, Oken BS: Heart rate variability analysis of effect of nicotine using periodograms. Conf Proc IEEE Eng Med Biol Soc 2004, 1: 294-297.

    PubMed  Google Scholar 

  21. Van Dongen HP, Olofsen E, VanHartevelt JH, Kruyt EW: Searching for biological rhythms: peak detection in the periodogram of unequally spaced data. J Biol Rhythms 1999, 14: 617-620. 10.1177/074873099129000984

    Article  CAS  PubMed  Google Scholar 

  22. Liu N, Lin Z, Koh Z, Huang G, Ser W, Ong M: Patient outcome prediction with heart rate variability and vital signs. J Signal Process Syst 2011, 64: 265-278. 10.1007/s11265-010-0480-y

    Article  Google Scholar 

  23. Burges CJC: A tutorial on support vector machines for pattern recognition. Data Min Knowl Discov 1998, 2: 121-167. 10.1023/A:1009715923555

    Article  Google Scholar 

  24. Hearst MA, Dumais ST, Osman E, Platt J, Scholkopf B: Support vector machines. IEEE Intell Syst 1998, 13: 18-28.

    Article  Google Scholar 

  25. Ubeyli ED: ECG beats classification using multiclass support vector machines with error correcting output codes. Digit Signal Process 2007, 17: 675-684. 10.1016/j.dsp.2006.11.009

    Article  Google Scholar 

  26. Liang NY, Saratchandran P, Huang GB, Sundararajan N: Classification of mental tasks from EEG signals using extreme learning machine. Int J Neural Syst 2006, 16: 29-38. 10.1142/S0129065706000482

    Article  PubMed  Google Scholar 

  27. Liu Y, Loh HT, Tor SB: Comparison of extreme learning machine with support vector machine for text classification. Innov Appl Artif Intell 2005, 3533: 390-399. 10.1007/11504894_55

    Article  Google Scholar 

  28. Polikar R: Bootstrap-inspired techniques in computational intelligence. IEEE Signal Process Mag 2007, 24: 59-72.

    Article  Google Scholar 

  29. Hanley JA, McNeil BJ: A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 1983, 148: 839-843.

    Article  CAS  PubMed  Google Scholar 

  30. Bennett M: On comparison of sensitivity, specificity and predictive value of a number of diagnostic procedures. Biometrics 1972, 28: 793-800. 10.2307/2528763

    Article  CAS  PubMed  Google Scholar 

  31. Van Belle G: Statistical Rules of Thumb. 1st edition. John Wiley & Sons, New York, USA; 2002.

    Google Scholar 

  32. Chen WL, Tsai TH, Huang CC, Chen JH, Kuo CD: Heart rate variability predicts short-term outcome for successfully resuscitated patients with out-of-hospital cardiac arrest. Resuscitation 2009, 80: 1114-1118. 10.1016/j.resuscitation.2009.06.020

    Article  PubMed  Google Scholar 

  33. Tsuji H, Venditti FJ Jr, Manders ES, Evans JC, Larson MG, Feldman CL, Levy D: Reduced heart rate variability and mortality risk in an elderly cohort. The Framingham Heart Study. Circulation 1994, 90: 878-883. 10.1161/01.CIR.90.2.878

    Article  CAS  PubMed  Google Scholar 

  34. Rich MW, Saini JS, Kleiger RE, Carney RM, teVelde A, Freedland KE: Correlation of heart rate variability with clinical and angiographic variables and late mortality after coronary angiography. Am J Cardiol 1988, 62: 714-717. 10.1016/0002-9149(88)91208-8

    Article  CAS  PubMed  Google Scholar 

  35. Szabo BM, van Veldhuisen DJ, van der Veer N, Brouwer J, De Graeff PA, Crijns HJ: Prognostic value of heart rate variability in chronic congestive heart failure secondary to idiopathic or ischemic dilated cardiomyopathy. Am J Cardiol 1997, 79: 978-980. 10.1016/S0002-9149(97)00026-X

    Article  CAS  PubMed  Google Scholar 

  36. Ponikowski P, Anker SD, Chua TP, Szelemej R, Piepoli M, Adamopoulos S, Webb-Peploe K, Harrington D, Banasiak W, Wrabec K, Coats AJ: Depressed heart rate variability as an independent predictor of death in chronic congestive heart failure secondary to ischemic or idiopathic dilated cardiomyopathy. Am J Cardiol 1997, 79: 1645-1650. 10.1016/S0002-9149(97)00215-4

    Article  CAS  PubMed  Google Scholar 

  37. Guzzetti S, Mezzetti S, Magatelli R, Porta A, De Angelis G, Rovelli G, Malliani A: Linear and non-linear 24 h heart rate variability in chronic heart failure. Auton Neurosci 2000, 86: 114-119. 10.1016/S1566-0702(00)00239-3

    Article  CAS  PubMed  Google Scholar 

  38. Guzzetti S, Piccaluga E, Casati R, Cerutti S, Lombardi F, Pagani M, Malliani A: Sympathetic predominance in essential hypertension: a study employing spectral analysis of heart rate variability. J Hypertens 1988, 6: 711-717. 10.1097/00004872-198809000-00004

    Article  CAS  PubMed  Google Scholar 

  39. van Boven AJ, Jukema JW, Haaksma J, Zwinderman AH, Crijns HJ, Lie KI: Depressed heart rate variability is associated with events in patients with stable coronary artery disease and preserved left ventricular function. REGRESS Study Group. Am Heart J 1998, 135: 571-576. 10.1016/S0002-8703(98)70269-8

    Article  CAS  PubMed  Google Scholar 

  40. Huang J, Sopher SM, Leatham E, Redwood S, Camm AJ, Kaski JC: Heart rate variability depression in patients with unstable angina. Am Heart J 1995, 130: 772-779. 10.1016/0002-8703(95)90076-4

    Article  CAS  PubMed  Google Scholar 

  41. Poulsen SH, Jensen SE, Moller JE, Egstrup K: Prognostic value of left ventricular diastolic function and association with heart rate variability after a first acute myocardial infarction. Heart 2001, 86: 376-380. 10.1136/heart.86.4.376

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  42. Boardman A, Schlindwein FS, Thakor NV, Kimura T, Geocadin RG: Detection of asphyxia using heart rate variability. Med Biol Eng Comput 2002, 40: 618-624. 10.1007/BF02345299

    Article  CAS  PubMed  Google Scholar 

  43. Axelrod S, Lishner M, Oz O, Bernheim J, Ravid M: Spectral analysis of fluctuations in heart rate: an objective evaluation of autonomic nervous control in chronic renal failure. Nephron 1987, 45: 202-206. 10.1159/000184117

    Article  CAS  PubMed  Google Scholar 

  44. Pagani M, Malfatto G, Pierini S, Casati R, Masu AM, Poli M, Guzzetti S, Lombardi F, Cerutti S, Malliani A: Spectral analysis of heart rate variability in the assessment of autonomic diabetic neuropathy. J Auton Nerv Syst 1988, 23: 143-153. 10.1016/0165-1838(88)90078-1

    Article  CAS  PubMed  Google Scholar 

  45. Baillard C, Vivien B, Mansier P, Mangin L, Jasson S, Riou B, Swynghedauw B: Brain death assessment using instant spectral analysis of heart rate variability. Crit Care Med 2002, 30: 306-310. 10.1097/00003246-200202000-00007

    Article  PubMed  Google Scholar 

  46. Biswas AK, Summerauer SF: Heart rate variability and brain death. J Neurosurg Anesthesiol 2004, 16: 62. 10.1097/00008506-200401000-00011

    Article  PubMed  Google Scholar 

  47. Bunten DC, Warner AL, Brunnemann SR, Segal JL: Heart rate variability is altered following spinal cord injury. Clin Auton Res 1998, 8: 329-334. 10.1007/BF02309623

    Article  CAS  PubMed  Google Scholar 

  48. Rapenne T, Moreau D, Lenfant F, Boggio V, Cottin Y, Freysz M: Could heart rate variability analysis become an early predictor of imminent brain death? A pilot study. Anesth Analg 2000, 91: 329-336.

    CAS  PubMed  Google Scholar 

  49. Winchell RJ, Hoyt DB: Analysis of heart-rate variability: a noninvasive predictor of death and poor outcome in patients with severe head injury. J Trauma 1997, 43: 927-933. 10.1097/00005373-199712000-00010

    Article  CAS  PubMed  Google Scholar 

  50. Korach M, Sharshar T, Jarrin I, Fouillot JP, Raphael JC, Gajdos P, Annane D: Cardiac variability in critically ill adults: influence of sepsis. Crit Care Med 2001, 29: 1380-1385. 10.1097/00003246-200107000-00013

    Article  CAS  PubMed  Google Scholar 

  51. Garrard CS, Kontoyannis DA, Piepoli M: Spectral analysis of heart rate variability in the sepsis syndrome. Clin Auton Res 1993, 3: 5-13. 10.1007/BF01819137

    Article  CAS  PubMed  Google Scholar 

  52. Moriguchi T, Hirasawa H, Oda S, Tateishi Y: Analysis of heart rate variability is a useful tool to predict the occurrence of septic shock in patients with severe sepsis. Nippon Rinsho 2004, 62: 2285-2290.

    PubMed  Google Scholar 

  53. Barnaby D, Ferrick K, Kaplan DT, Shah S, Bijur P, Gallagher EJ: Heart rate variability in emergency department patients with sepsis. Acad Emerg Med 2002, 9: 661-670. 10.1111/j.1553-2712.2002.tb02143.x

    Article  PubMed  Google Scholar 

  54. Bigger JT Jr, La Rovere MT, Steinman RC, Fleiss JL, Rottman JN, Rolnitzky LM, Schwartz PJ: Comparison of baroreflex sensitivity and heart period variability after myocardial infarction. J Am Coll Cardiol 1989, 14: 1511-1518. 10.1016/0735-1097(89)90390-2

    Article  PubMed  Google Scholar 

  55. Huikuri HV, Makikallio T, Airaksinen KE, Mitrani R, Castellanos A, Myerburg RJ: Measurement of heart rate variability: a clinical tool or a research toy? J Am Coll Cardiol 1999, 34: 1878-1883. 10.1016/S0735-1097(99)00468-4

    Article  CAS  PubMed  Google Scholar 

  56. Lombardi F, Malliani A, Pagani M, Cerutti S: Heart rate variability and its sympatho-vagal modulation. Cardiovasc Res 1996, 32: 208-216. 10.1016/0008-6363(96)00116-2

    Article  CAS  PubMed  Google Scholar 

  57. Ong ME, Padmanabhan P, Chan YH, Lin Z, Overton J, Ward KR, Fei DY: An observational, prospective study exploring the use of heart rate variability as a predictor of clinical outcomes in pre-hospital ambulance patients. Resuscitation 2008, 78: 289-297. 10.1016/j.resuscitation.2008.03.224

    Article  PubMed  Google Scholar 

  58. Ong M, Pavitra P, Chan Y, Lin Z: Using heart rate variability as a predictor of clinical outcomes in pre-hospital ambulance patients. SGH Proc 2007, 16: S177.

    Google Scholar 

  59. Yamada A, Hayano J, Sakata S, Okada A, Mukai S, Ohte N, Kimura G: Reduced ventricular response irregularity is associated with increased mortality in patients with chronic atrial fibrillation. Circulation 2000, 102: 300-306. 10.1161/01.CIR.102.3.300

    Article  CAS  PubMed  Google Scholar 

  60. Lombardi F, Colombo A, Basilico B, Ravaglia R, Garbin M, Vergani D, Battezzati PM, Fiorentini C: Heart rate variability and early recurrence of atrial fibrillation after electrical cardioversion. J Am Coll Cardiol 2001, 37: 157-162. 10.1016/S0735-1097(00)01039-1

    Article  CAS  PubMed  Google Scholar 

  61. Wolf PA, Abbott RD, Kannel WB: Atrial fibrillation as an independent risk factor for stroke: the Framingham Study. Stroke 1991, 22: 983-988. 10.1161/01.STR.22.8.983

    Article  CAS  PubMed  Google Scholar 

  62. Huikuri HV, Valkama JO, Airaksinen KE, Seppanen T, Kessler KM, Takkunen JT, Myerburg RJ: Frequency domain measures of heart rate variability before the onset of nonsustained and sustained ventricular tachycardia in patients with coronary artery disease. Circulation 1993, 87: 1220-1228. 10.1161/01.CIR.87.4.1220

    Article  CAS  PubMed  Google Scholar 

  63. Lombardi F, Porta A, Marzegalli M, Favale S, Santini M, Vincenti A, De Rosa A: Heart rate variability patterns before ventricular tachycardia onset in patients with an implantable cardioverter defibrillator. Participating Investigators of ICD-HRV Italian Study Group. Am J Cardiol 2000, 86: 959-963. 10.1016/S0002-9149(00)01130-9

    Article  CAS  PubMed  Google Scholar 

  64. Shusterman V, Aysin B, Weiss R, Brode S, Gottipaty V, Schwartzman D, Anderson KP: Dynamics of low-frequency R-R interval oscillations preceding spontaneous ventricular tachycardia. Am Heart J 2000, 139: 126-133.

    Article  CAS  PubMed  Google Scholar 

  65. Pruvot E, Thonet G, Vesin JM, van-Melle G, Seidl K, Schmidinger H, Brachmann J, Jung W, Hoffmann E, Tavernier R, Block M, Podczeck A, Fromer M: Heart rate dynamics at the onset of ventricular tachyarrhythmias as retrieved from implantable cardioverter-defibrillators in patients with coronary artery disease. Circulation 2000, 101: 2398-2404. 10.1161/01.CIR.101.20.2398

    Article  CAS  PubMed  Google Scholar 

  66. Flapan AD, Wright RA, Nolan J, Neilson JM, Ewing DJ: Differing patterns of cardiac parasympathetic activity and their evolution in selected patients with a first myocardial infarction. J Am Coll Cardiol 1993, 21: 926-931. 10.1016/0735-1097(93)90349-6

    Article  CAS  PubMed  Google Scholar 

  67. Ortak J, Weitz G, Wiegand UK, Bode F, Eberhardt F, Katus HA, Richardt G, Schunkert H, Bonnemeier H: Changes in heart rate, heart rate variability, and heart rate turbulence during evolving reperfused myocardial infarction. Pacing Clin Electrophysiol 2005,28(Suppl 1):S227-S232.

    Article  PubMed  Google Scholar 

  68. Lombardi F, Sandrone G, Spinnler MT, Torzillo D, Lavezzaro GC, Brusca A, Malliani A: Heart rate variability in the early hours of an acute myocardial infarction. Am J Cardiol 1996, 77: 1037-1044. 10.1016/S0002-9149(96)00127-0

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

The authors would like to thank and acknowledge the contributions from all the doctors and nurses from the Department of Emergency Medicine, Singapore General Hospital. They would also like to thank Assistant Professor John C Allen Jr from the Office of Clinical Sciences, Centre for Quantitative Medicine, Duke - NUS Graduate Medical School of Singapore for his statistical input. This study was supported by grants from SingHealth Talent Development Fund, Singapore (TDF/CS001/2006) and InfoComm Research Cluster, Nanyang Technological University, Singapore (2006ICT09).

Author information

Authors and Affiliations

  1. Department of Emergency Medicine, Singapore General Hospital, Outram Road, 169608, Singapore, Singapore

    Marcus Eng Hock Ong, Nan Liu, Zhi Xiong Koh, Nur Shahidah & Tong Tong Zhang

  2. Yong Loo Lin School of Medicine, National University of Singapore, 12 Medical Drive, 17598, Singapore, Singapore

    Christina Hui Lee Ng

  3. Duke-NUS Graduate Medical School, 8 College Road, Singapore, 169857, Singapore

    Ken Goh

  4. Department of Clinical Research, Singapore General Hospital, Outram Road, Singapore, 169608, Singapore

    Stephanie Fook-Chong

  5. School of Electrical and Electronic Engineering, Nanyang Technological University, 50 Nanyang Avenue, Singapore, 639798, Singapore

    Zhiping Lin

Authors
  1. Marcus Eng Hock Ong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christina Hui Lee Ng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ken Goh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhi Xiong Koh
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Nur Shahidah
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Tong Tong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Stephanie Fook-Chong
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Zhiping Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marcus Eng Hock Ong.

Additional information

Competing interests

MEHO and ZL have a patent filing related to the technology described in the study (Method of predicting acute cardiopulmonary events and survivability of a patient; application number 13/047,348). They also have a licensing agreement with ZOLL Medical Corporation for the technology. All remaining authors declare that they have no competing interests.

Authors' contributions

MEHO planned and established the project, including the procedures for data collection, drafted the manuscript, and performed data analysis. SF-C performed detailed statistical analysis of the data. CHLN drafted the manuscript, and performed data collection and data analysis. KG, NL, ZXK, NS, and TTZ performed data collection and data analysis. ZL reviewed critical revisions to the manuscript. All authors took part in rewriting and approved the final manuscript.

Authors’ original submitted files for images

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ong, M.E.H., Lee Ng, C.H., Goh, K. et al. Prediction of cardiac arrest in critically ill patients presenting to the emergency department using a machine learning score incorporating heart rate variability compared with the modified early warning score. Crit Care 16, R108 (2012). https://doi.org/10.1186/cc11396

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/cc11396

Keywords

Critical Care

ISSN: 1364-8535