A novel cardiovascular risk stratification model incorporating ECG and heart rate variability for patients presenting to the emergency department with chest pain

Background Risk stratification models can be employed at the emergency department (ED) to evaluate patient prognosis and guide choice of treatment. We derived and validated a new cardiovascular risk stratification model comprising vital signs, heart rate variability (HRV) parameters, and demographic and electrocardiogram (ECG) variables. Methods We conducted a single-center, observational cohort study of patients presenting to the ED with chest pain. All patients above 21 years of age and in sinus rhythm were eligible. ECGs were collected and evaluated for 12-lead ECG abnormalities. Routine monitoring ECG data were processed to obtain HRV parameters. Vital signs and demographic data were obtained from electronic medical records. Thirty-day major adverse cardiac events (MACE) were the primary endpoint, including death, acute myocardial infarction, and revascularization. Candidate variables were identified using univariate analysis; the model for the final risk score was derived by multivariable logistic regression. We compared the performance of the new model with that of the thrombolysis in myocardial infarct (TIMI) score using receiver operating characteristic (ROC) analysis. Results In total, 763 patients were included in this study; 254 (33 %) met the primary endpoint, the mean age was 60 (σ = 13) years, and the majority was male (65 %). Nineteen candidate predictors were entered into the multivariable model for backward variable elimination. The final model contained 10 clinical variables, including age, gender, heart rate, three HRV parameters (average R-to-R interval (RR), triangular interpolation of normal-to-normal (NN) intervals, and high-frequency power), and four 12-lead ECG variables (ST elevation, ST depression, Q wave, and QT prolongation). Our proposed model outperformed the TIMI score for prediction of MACE (area under the ROC curve 0.780 versus 0.653). At the cutoff score of 9 (range 0–37), our model had sensitivity of 0.709 (95 % CI 0.653, 0.765), specificity of 0.674 (95 % CI 0.633, 0.715), positive predictive value of 0.520 (95 % CI 0.468, 0.573), and negative predictive value of 0.823 (95 % CI 0.786, 0.859). Conclusions A non-invasive and objective ECG- and HRV-based risk stratification tool performed well against the TIMI score, but future research warrants use of an external validation cohort.


Background
Chest pain is the most frequent complaint in patients over 45 years of age presenting to the emergency department (ED) in the USA [1]. The first priority in the evaluation of patients with chest pain is risk stratification to differentiate those who are in acute cardiovascular distress from those who are not. Risk stratification allows for an appropriate therapeutic strategy and an effective allocation of ED resources.
The American College of Cardiology/American Heart Association (ACC/AHA) practice guidelines recommend the use of validated risk scoring models such as the thrombolysis in myocardial infarction (TIMI), platelet glycoprotein IIb/IIIa in unstable angina: receptor suppression using integrilin (PURSUIT), and global registry of acute coronary events (GRACE) for cardiovascular risk stratification [2][3][4][5][6]. However, commonly employed cardiovascular risk models rely on traditional clinical indicators that are subjective, susceptible to risk factor modification (e.g., anticholesterol or antihypertensive therapy), may not be immediately available, and often do not correlate well with long-term or short-term outcomes [7][8][9][10]. Consequently, there is a need for a quantitative and rapid method to guide patient disposition in the ED.
This study aims to develop a novel risk stratification model composed of HRV parameters, demographic data, traditional vital signs and 12-lead ECG variables for the prediction of major adverse cardiac events (MACE) in patients with chest pain (henceforth referred to as the Singapore General Hospital Emergency Department risk stratification model (SEDRSM)). This study hypothesizes that the SEDRSM will perform better than an established risk stratification tool (TIMI) at predicting MACE within 30 days of a patient presenting to the ED with chest pain.

Design and setting
We conducted a prospective, non-randomized, observational study of patients presenting to the ED with chest pain from March 2010 until August 2015. This study was performed at the ED of the Singapore General Hospital (SGH), a tertiary care hospital in Singapore. ED triage is performed by nurses using the national Singaporean patient acuity category scale (PACS), a symptom-based triage system without strict physiological criteria. ED patients are classified with a PACS score, which ranges from 1 to 4 and represents the degree of urgency in patient attendance. Patients with PACS 1 are the most critically ill, those with PACS 2 are non-ambulant, those with PACS 3 are ambulant, and those with PACS 4 are non-emergencies. Our study focuses on patients presenting with chest pain, who routinely receive a 12-lead ECG investigation (Philips PageWriter TC50 Cardiograph) during triage and are placed in PACS 1 or 2 units where they receive further ECG monitoring (ZOLL X Series Monitor defibrillator). The study was approved by the local ethics committee (SingHealth Centralised Institutional Review Board, Singapore) with a waiver of patient consent.

Patient recruitment and eligibility
All patients older than 21 years of age with a primary complaint of non-traumatic chest pain were eligible. Patients presenting in non-sinus rhythm (arrhythmias, asystole, complete heart blocks, or pacemaker rhythms) were excluded due to interference of these phenomena with the interpretation of QRS complexes. Similarly, patients with a high percentage of artifacts, ectopic beats, and non-sinus beats (>30 % of ECG recordings) were excluded due to their potential biasing effect on the HRV calculations [27]. Finally, patients who were lost to follow up or transferred to other (private) hospitals within the 30-day time frame were excluded, on account of inability to ascertain whether these patients had reached our primary endpoint.

Data collection and processing
All data were collected on standardized forms in a Research Electronic Data Capture (REDCap) database. The electronic medical records (EMRs) were analyzed for demographic characteristics, medical history, presenting symptoms, clinical information, and laboratory results.
A trained research coordinator prospectively downloaded 12-lead ECG tracings from the ZOLL X Series monitor defibrillator on a daily basis. We use our inhouse software package for ECG signal processing and parameter calculation [28]. Noise was manually removed from the lead II ECG tracing and its sample of 6 minutes was stored in an Excel (Microsoft Office 2007; Microsoft, Redmond, WA, USA) file for further processing. A 5-28 Hz band-pass filter was applied to the lead II sample to facilitate peak detection [29]. QRS complexes were detected using a threshold-plus-derivative method that has been previously validated [28]. Time domain and frequency domain HRV parameters were calculated in accordance with the guidelines outlined by the Taskforce of the European Society of Cardiology [30].
Vital signs were recorded at initial ED patient presentation using the Propaq CS Vital Signs Monitor (Welch Allyn, Skaneateles, NY, USA). The first set of complete vital signs obtained at initial presentation was used for this study. The 12-lead ECG tracings recorded during triage were used for the evaluation of ECG variables. These tracings were recorded using a Philips PageWriter TC50 cardiograph and subsequently extracted for analysis and storage. A trained research associate, blinded to patient outcomes, ascertained whether the patient was in sinus rhythm, and evaluated the 12-lead ECG tracings for abnormalities. We followed the definitions of ECG variables as described in John Hampton's book "The ECG Made Easy".
We tested the SEDRSM against the TIMI score for unstable angina (UA)/non-ST elevation myocardial infarction (NSTEMI), a score that has been employed on the ED to predict MACE within 30 days of presentation to the ED with chest pain [31]. Data pertaining to the TIMI score criteria were retrieved from the EMRs and used to construct the TIMI score.

Outcomes
The primary endpoint of this study, MACE, was a composite outcome of death, acute myocardial infarction, and revascularization, including coronary artery bypass graft (CABG) or percutaneous coronary intervention (PCI), within 30 days of presentation to the ED. Patients were followed up and EMRs were reviewed to ascertain whether the patient had experienced an endpoint criterion within 30 days after presentation.

Statistical analysis
SPSS (version 21.0; SPSS Inc, Chicago, IL, USA) software was used for statistical analysis. Derivation and validation of the SEDRSM was done in the same cohort. Univariate relationships between baseline characteristics and MACE were assessed using the appropriate statistical test, based on type and distribution of data. We tested normality of distribution by inspecting normality graphs and interpreting the Kolmogorov-Smirnov quantitative normality test.
A total of 16 HRV parameters, 13 ECG variables, 7 vital signs, and 3 demographic variables (age, gender, and race) were screened as candidate predictors of MACE using the same univariate analytical method described previously. Variables associated with a p value <0.05 were selected and categorized in order to facilitate scoring and increase applicability at the ED. HRV parameter category cutoffs were chosen based on the visual comparison between HRV parameter value and frequency of MACE occurrence. ECG variables were dichotomous. Vital signs and demographics were categorized based on recognized (physiological) cutoff values.
We introduced the categorized candidate variables into an automated likelihood ratio backward stepwise logistic regression model. The retained candidate variables were used to construct the SEDRSM. All unstandardized coefficients were normalized by dividing them by the smallest coefficient, and subsequently rounded off to the nearest integer. The SEDRSM score was then calculated by a simple arithmetic sum of the integers assigned to the criteria satisfied.
The overall goodness of fit of the model was assessed by the Hosmer-Lemeshow test. The predictive accuracy of the SEDRSM and TIMI score was assessed using the area under the receiver operating characteristic (AUROC) curve. Discriminatory values (i.e., sensitivity, specificity, positive predictive value, and negative predictive value) were also determined for both risk stratification models.

Results
We included 763 patients in the study. The baseline characteristics of our total patient cohort and of those with and without a MACE (our primary endpoint) are shown in Table 1. A total of 254 patients experienced a MACE, versus a total of 509 who did not. The mean age of our cohort was 60 (SD = 13) years. The majority of the population was male (65 %). In the group that experienced a MACE, the patients were older (61.75 years) (p = 0.001), there were more male patients (72.8 %, p = 0.001) and more patients with diabetes mellitus (44.5 %, p = 0.001), and fewer patients with respiratory disease (1.6 %, p = 0.012). We also found that patients who experienced a MACE were more frequently admitted, specifically more often to general wards and intensive care wards. No other significant differences were found between the groups who did or not experience. The risk factors hypertension and hyperlipidemia were present in over half the cohort. The frequencies of all different types of MACE are shown in Table 2. The most frequent MACE was revascularization (24.5 %) by either PCI, or CABG, or both, followed closely by MI (23.6 %). Death (2.0 %) was the least frequent MACE experienced by patients in this cohort. Table 3 shows the univariate association between vital signs, HRV parameters, 12-lead ECG variables, and our endpoint. A total of 19 candidates for the SEDRSM, including gender and age, were identified (p < 0.05). Heart rate and diastolic blood pressure were found to be significantly elevated in the cohort that experienced a MACE within 30 days. The presence of five ECG variables was strongly associated with the occurrence of a MACE within 30 days. Lastly, nine HRV parameters were found to be significantly different in the cohort that experienced a MACE versus the cohort that did not experience a MACE; these were average R-to-R interval (RR), SDRR (SD R-to-R interval), average HR, root mean square of successive differences (RMSSD), the number of interval differences of successive normal-to-normal (NN) intervals greater than 50 ms (NN50), the proportion derived by dividing NN50 by the total number of NN intervals (pNN50), triangular interpolation of NN interval histogram (TINN), very low frequency (VLF), and high frequency (HF).  The final set of criteria for the SEDRSM is shown in Table 5. The unstandardized coefficients were normalized by dividing the total set by 0.277, which was the smallest common multiplicative factor. The normalized unstandardized coefficients were subsequently rounded off to the nearest integer. The SEDRSM has a range of 0 to 37.
The SEDRSM performed with an AUROC (or C statistic) of 0.780 (95 % CI 0.743, 0.817), compared to an AUROC of 0.653 (95 % CI 0.611, 0.695) for the TIMI (UA/NSTEMI) score in the prediction of 30-day MACE. The SEDRSM performed significantly better than the TIMI score (p < 0.001). The ROC curves for the SEDRSM and the TIMI score are shown in Fig. 1. Figure 2 shows the SEDRSM score in relation to the rate of MACE. With an increase of the score constructed in the SEDRSM there is a proportionate increase in patients experiencing MACE. For example, 71.0 % of patients with the risk score of 15 or 16 have 30-day MACE. In Fig. 3, we did a further investigation on the performance of the SEDRSM where the distributions of the risk scores are illustrated by outcome categories, that is, with and without 30-day MACE. The gray bars indicate the score distributions for patients

Discussion
In this study we constructed a risk stratification model (the SEDRSM) incorporating vital signs, demographic data, ECG variables, and HRV parameters for the prediction of 30-day MACE in patients presenting to the ED with chest pain. In validation in the same cohort, the SEDRSM significantly outperformed the TIMI score in terms of AUROC (0.780 versus 0.653, p < 0.001). Additionally, the SEDRSM performs better than the TIMI score in terms of sensitivity, specificity, positive predictive value, and negative predictive value.
Vital signs and 12-lead ECG variables are well-established and frequently employed during clinical risk prediction on the ED [11,32]. In contrast, many studies have reported the clinical and prognostic value of HRV parameters in the evaluation of patients with possible cardiovascular pathological conditions, but they have yet to be clinically applied [33]. This study demonstrates that HRV parameters can be successfully implemented in cardiovascular risk stratification on the ED, even amidst other established prognosticators. The TIMI score has been used as a benchmark due to its popularity and accuracy in predicting 30-day MACE in patients presenting to the ED with chest pain [31]. We used short-term (6-minute) recordings of 12-lead ECG as it has practical advantages in the time-scarce ED setting. Long-term (24-h) HRV analysis is prone to data analysis difficulties (e.g., failure to detect low-frequency oscillations and data-filtering difficulties). The SEDRSM Score Fig. 2 The relationship between the Singapore Emergency Department risk stratification model (SEDRSM) and the rate of major adverse cardiac events (MACE) 3  Recent evidence demonstrated a comparable predictive value for short-term vs long-term HRV analysis [34,35]. The role of HRV in cardiovascular risk prediction has been examined previously but we cannot make direct comparisons with our study due to fundamental differences in methodology (i.e., no inclusion of 12-lead ECG variables) and a low yield of patients meeting the primary endpoints in previous reports [26].
Calculating HRV requires only ECG monitoring and standard analysis software [36,37]. The SEDRSM had an AUROC of 0.780 versus an AUROC of 0.653 for the TIMI score. Risk stratification models are considered reasonable when the AUROC (or C statistic) is higher than 0.7 and strong when it exceeds 0.8 [38]. Our findings reaffirm the potential role of HRV amidst clinical cardiovascular predictors such as 12-lead ECG variables, and demonstrate a good performance compared to a widely used cardiovascular ED risk prediction model; however, there are still many opportunities for enhancement.
Our results show that three HRV parameters are significant predictors in a multivariable prediction model. Additionally, four 12-lead ECG variables, and two demographic variables were strong significant predictors in the multivariable model. Only one vital sign, heart rate, was incorporated as a criterion into the model. The heart rate can also be obtained through processing of the 12-lead ECG. This tool would allow wider diffusion of HRV parameters into clinical use as it enables HRV parameter interpretation by the physician, which is nearly impossible by manual methods. The SEDRSM allows early clinical anticipation of MACE on the ED, and thus facilitates early intervention.
HRV is a highly complex nonlinear phenomenon; even though time-domain and frequency-domain HRV parameters are most frequently employed (designated as the "classical analysis") they are not fully representative due to their linear and stationary nature. Evidence suggests that nonlinear HRV parameters are much more suitable for the assessment and prognosis of cardiovascular risk than the "classical analysis" [39]. However, more highquality longitudinal studies are required to establish the applicability of nonlinear HRV parameters [13].
Fragmented QRS complex is an easily obtainable ECG index that might further enhance cardiovascular risk prediction models [40]. Signal averaged ECG analysis has been demonstrated to have a high negative predictive value [41], and could be employed simultaneously with HRV processing. Microvolt T-wave alternans, a measure of repolarization dispersion, has been employed in the prediction of cardiac death and may add value to HRV and signal averaged ECG (SAECG) analysis [42]. Last, the occurrence of specific early repolarization morphological features that are frequently difficult to distinguish from true ST elevation, have recently been associated with increased risk of unexpected death. These patterns are considered particularly useful to refine risk stratification and identify a high-risk subset of patients; future research could explore the value of incorporating certain repolarization morphological features into risk prediction using the HRV and 12-lead ECG analysis [43,44].

Limitations
In our study we have compared the performance of the SEDRSM and the TIMI score for the prediction of 30day MACE in patients presenting to the ED with chest pain. The SEDRSM score was only evaluated in our local population, whereas the TIMI score has been popular and widely validated, despite the fact that it was designed for a slightly different specification [4]. Though there is now evidence of its usefulness in this setting [31], the usefulness of some of the criteria in the TIMI score is questionable in the ED. For example, the TIMI score requires knowledge of results of prior cardiac catheterization, which might not be available. Newer risk stratification scores have been developed that are specifically calibrated towards patients presenting to the ED with chest pain. For example, the history, ECG, age, risk factors (HEART) score [45] and the Emergency Department assessment of chest pain score (EDACS) score [46] could serve as better benchmarks due to their applicability to, and specific design for, an ED setting in which the risk of 30-day MACE needs to be assessed [47].
Our study evaluates HRV as a quantitative measure of a supposed quantitative primary endpoint, MACE. However, MACE includes revascularization (either through PCI or CABG) as a category, which is not necessarily a purely quantitative endpoint. Patient choice, i.e., the decision by patients to reject revascularization at their own risk, is a qualitative phenomenon. The revascularization endpoint therefore interferes with the computation of quantitative linear relationships between predictor and endpoint. In our study we have not corrected for, or The Singapore Emergency Department risk stratification model (SEDRSM) has a range of 0 to 37; the thrombolysis in myocardial infarct (TIMI) score has a range of 0 to 7. AUROC area under the receiver operating characteristic, CI confidence interval, PPV positive predictive value, NPV negative predictive value quantified the number of patients rejecting medical diagnostics or therapy at any point after presentation to the ED. A modification of endpoint or exclusion of these patients would likely facilitate a more accurate linear relationship between predictor and endpoint. In addition to the above limitations, we note that HRV parameters can not be manually calculated or interpreted. In this study, we used our in-house software to derive HRV measurements. We are currently developing a portable hardware device to integrate data acquisition and analysis. We believe that such a device will help clinicians quickly identify patients at high risk of developing MACE.

Conclusions
In our single-center, single-cohort study of patients presenting to the ED with chest pain, our risk stratification model (the SEDRSM) outperformed the TIMI score for the prediction of MACE, a composite endpoint of MI, revascularization, and death (AUROC of 0.780 versus 0.653 respectively). The SEDRSM incorporated the following criteria: age, gender, heart rate, three HRV parameters, and four 12-lead ECG variables. The SEDRSM provides useful information for making decisions about the placement of ED patients with chest pain under observation, and in determining the therapeutic strategy. The SEDRSM contains eight criteria that can be acquired by processing electrocardiographic data, allowing for a 12-lead ECG-based risk prediction device, setting aside only the manual input of demographic criteria. A risk stratification device could employ machine learning techniques that reduce information loss occurring during the construction of multivariable linear association models. There are several other ECGbased variables that can be valuable additions to our risk stratification score; these include nonlinear HRV parameters and novel depolarization or repolarization ECG variables. Our study demonstrates the potency and suitability of the SEDRSM for cardiovascular risk prediction in the ED, but also warrants evaluation and possibly resolution of existing limitations before it can be implemented in clinical practice.

Key messages
The Singapore General Hospital (SGH) Emergency Department risk stratification model (SEDRSM) was proposed for patients with chest pain The SEDRSM outperformed the TIMI score in predicting 30-day major adverse cardiac events Abbreviations ACC/AHA, American College of Cardiology/American Heart Association; BP, blood pressure; CABG, coronary artery bypass graft; CHF, congestive heart failure; CI, confidence interval; DM, diabetes mellitus; ECG, electrocardiogram; ED, emergency department; EDACS, Emergency Department assessment of chest pain score; EMR, electronic medical records; GRACE, global registry of acute coronary events; GW, general ward; HEART, history, ECG, age, risk factors, Troponin; HF, high frequency; HR, heart rate; HRV, heart rate variability; ICW, intensive care ward; IHD, ischemic heart disease; IVCD, intraventricular conduction defect; LAA, left atrial abnormality; LBBB, left bundle branch block; LF, low frequency; LVH, left ventricular hypertrophy; MACE, major adverse cardiac event; MI, myocardial infarction; NHCS, National Heart Centre Singapore; NN, normal-to-normal intervals; NN50, number of interval differences of successive NN intervals greater than 50 ms; PACS, patient acuity category scale; PCI, percutaneous coronary intervention; pNN50, proportion derived by dividing NN50 by the total number of NN intervals; PURSUIT, Platelet glycoprotein IIb/IIIa in unstable angina: receptor suppression using integrilin therapy; RBBB, right bundle branch block; REDCap, research electronic data capture; RMSSD, root mean square of successive differences; ROC, receiver operator characteristic; RR, R-to-R interval; RVH, right ventricular hypertrophy; SAECG, signal averaged ECG; SDRR, standard deviation R-to-R interval; SEDRSM, Singapore Emergency Department risk stratification model; SGH, Singapore General Hospital; TIMI, thrombolysis in myocardial infarct; TINN, triangular interpolation of NN interval histogram; UA/NSTEMI, unstable angina/ non-ST-elevation myocardial infarct; VLF, very low frequency