Development and validation of a prehospital prediction model for acute traumatic coagulopathy

Background Acute traumatic coagulopathy (ATC) is a syndrome of early, endogenous clotting dysfunction that afflicts up to 30% of severely injured patients, signaling an increased likelihood of all-cause and hemorrhage-associated mortality. To aid identification of patients within the likely therapeutic window for ATC and facilitate study of its mechanisms and targeted treatment, we developed and validated a prehospital ATC prediction model. Methods Construction of a parsimonious multivariable logistic regression model predicting ATC — defined as an admission international normalized ratio >1.5 — employed data from 1963 severely injured patients admitted to an Oregon trauma system hospital between 2008 and 2012 who received prehospital care but did not have isolated head injury. The prediction model was validated using data from 285 severely injured patients admitted to a level 1 trauma center in Seattle, WA, USA between 2009 and 2013. Results The final Prediction of Acute Coagulopathy of Trauma (PACT) score incorporated age, injury mechanism, prehospital shock index and Glasgow Coma Score values, and prehospital cardiopulmonary resuscitation and endotracheal intubation. In the validation cohort, the PACT score demonstrated better discrimination (area under the receiver operating characteristic curve 0.80 vs. 0.70, p = 0.032) and likely improved calibration compared to a previously published prehospital ATC prediction score. Designating PACT scores ≥196 as positive resulted in sensitivity and specificity for ATC of 73% and 74%, respectively. Conclusions Our prediction model uses routinely available and objective prehospital data to identify patients at increased risk of ATC. The PACT score could facilitate subject selection for studies of targeted treatment of ATC. Electronic supplementary material The online version of this article (doi:10.1186/s13054-016-1541-9) contains supplementary material, which is available to authorized users.


Background
Over the last 15 years, randomized trials have often failed to validate previously promising therapies for critically ill patients [1][2][3][4]. The study of traumatic injury, which was the cause over 130,000 deaths in the USA in 2013 and remains the leading killer of adults and children ages 1-44 years [5], is no exception. Uncontrolled hemorrhage and post-traumatic coagulopathy contribute to half of injury-related deaths [6], but interventions including recombinant factor VIIa [7][8][9] and balanced transfusion [10] have demonstrated no benefit in broad populations of injured patients. At least some such negative trials seem to occur because researchers, who are lacking tools to quickly identify the subset of patients with disease biology amenable to targeted therapy, are forced to include heterogeneous subject populations [11,12].
The study of acute traumatic coagulopathy (ATC) poses particular challenges. Present in up to 30% of severely injured patients on emergency department (ED) arrival, ATC is an endogenous biologic syndrome contributing to, but distinct from, traumatic hemorrhage in general [13][14][15][16]. When defined as an international normalized ratio (INR) >1.5 on hospital admission, ATC is associated with a significantly increased risk-adjusted probability of not only all-cause and hemorrhageassociated mortality but also multiple organ failure and venous thromboembolism [13,14,17]. As most bleeding-related deaths occur early after injury, treatment to prevent or mitigate ATC also needs to begin quickly, potentially even in the prehospital setting. Diagnosis of ATC in this time frame, however, remains difficult: the conventional coagulation tests consistently linked to risk-adjusted outcomes are slow to return, but issues of validity, reliability, availability, and interpretation hinder broad implementation of otherwise promising point-of-care testing and viscoelastic measures [15,[18][19][20][21]. A simple, validated, predictive index using data available prior to ED admission to identify patients at high risk of ATCas opposed to major hemorrhage more generallycould advance research and patient care by facilitating trial enrollment, efficient specimen collection, and, ultimately, targeted ATC treatment.
The only prehospital ATC prediction tool reported so far, the Coagulopathy of Severe Trauma (COAST) score, is based on vehicle entrapment, chest decompression by paramedics, and prehospital assessment of blood pressure, temperature, and abdominal/pelvic content injury [22]. As the score was not externally validated after development in a single-center Australian cohort, its generalizability is uncertain [23]. Marked differences in ambulance crew practice patterns in the USA also pose obstacles to the application of the COAST score in trauma settings within the USA.
In the current study, we developed and internally validated a prediction model for ATC using patient demographic information, injury characteristics, and clinical data available to providers before patients' arrival in the ED. We then externally validated our score in an independent trauma cohort and compared its performance to that of the COAST score.

Derivation cohort
To derive a multivariable model predicting ATC, we studied severely injured non-pregnant patients ages 18-89 years, who were entered in the Oregon Trauma Registry from 2008 to 2012 [24]. Trained staff at the 44 certified trauma centers in Oregon enter details of injured patients treated at their facility into the registry if they meet any of the following criteria: intensive care unit (ICU) admission ≤24 hours from ED arrival; trauma team activation; prehospital trauma triage criteria met; surgical intervention; or injury severity score (ISS) >8 [25]. The registry excludes patients who die before ED arrival or who have isolated hip fracture after a groundlevel fall.
For model derivation, we used data from registry patients who met one or more of the following criteria for severe injury: death prior to discharge; admission directly from the initial trauma center ED to the ICU or operating room; or transfer from the initial ED to another state-certified trauma center ED followed by admission directly to the receiving facility ICU or operating room. Exclusion criteria included missing admission INR; initial care outside the trauma system; preadmission anticoagulant medication; blood transfusion during prehospital care; and no prehospital care. We also excluded patients with isolated burn or traumatic brain injury (no abbreviated injury score (AIS) ≥3 except for the head) because coagulopathy in these conditions differs from polytrauma-associated ATC [26]. The Oregon Health Authority and University of Washington Institutional Review Boards approved the use of Oregon Trauma Registry data.

Validation cohort
We validated our model in a prospective cohort (Age of Transfused Blood and Lung Injury After Trauma Study) collected at Harborview Medical Center, a level 1 trauma center in Seattle, WA, USA [27]. Patients with blunt trauma, age ≥18 years, admitted to the ICU from the ED (directly or via the operating room) between March 2010 and December 2013 were eligible for enrollment if transfused ≥1 units of red blood cells within 24 hours of injury. Study exclusion criteria were acute respiratory distress syndrome on admission, isolated traumatic brain injury (radiologic brain injury without non-brain injury), transfusion ≤6 months prior to admission, pregnancy, being in police custody, and expected survival <24 hours. The validation cohort excluded subjects on warfarin, with no prehospital care, or missing initial INR values. Trained research staff unaware of coagulopathy status collected data on patient characteristics, prehospital and ED care, and outcomes. The University of Washington Institutional Review Board approved the original study and granted exempt status to the current secondary analysis.

Predictor and outcome definitions
ATC was defined as an INR >1.5 on initial measurement in the first ED [17]. Potential ATC predictors identified a priori included patient and injury characteristics, and clinical and management data available before hospital arrival. Consistent with prior reports [28], we observed ≤1 point difference between prehospital and ED GCS in 85% of subjects not intubated in the field. We therefore substituted initial ED values for missing prehospital GCS in subjects not intubated prehospital. GCS was analyzed as the difference between the measured GCS and a normal GCS (15) to provide a positive regression coefficient. Shock indexthe ratio of the first prehospital heart rate to first prehospital systolic blood pressure (SBP)was considered elevated if ≥1 [29]. Prehospital treatments included cardiopulmonary resuscitation, chest decompression (needle or tube thoracostomy), and endotracheal intubation or invasive airway. In addition to ISS and AIS [30], injury severity indicators included rollover motor vehicle crash, ejection or need for extrication from vehicle ("entrapment"), and death of another person on scene [31].
COAST scores were calculated as previously described (Table 1) [22]. As prehospital providers in the USA do not systematically evaluate abdominal/pelvic content injury [32], we applied a secondary definitionabdominal/pelvic AIS ≥1used in the original description of the COAST score. Similarly, we employed the first ED temperature in place of the prehospital value [33].

Missing data
To minimize bias due to missing data, we performed multiple imputation based on chained equations to create 50 imputed datasets for both cohorts [34][35][36]. Missing values were imputed using predictive mean matching from three nearest neighbors for continuous variables [37] and logistic regression for binary variables. Imputation model variables (Additional file 1: Table S1) included missing and non-missing candidate predictors, hospital and coagulopathy outcomes, and other correlates of missing variables [38].

Model development
We constructed a multivariable ATC prediction model from prehospital variables in three steps: candidate predictor modeling, selection of a parsimonious final predictor set, and coefficient estimation. To minimize predictive bias and optimism, we ensured a >10:1 ratio of outcome events to predictors entered in the model selection algorithm [36,39,40]. To achieve this ratio, we (1) discarded variables with p values >0.2 in bivariable analyses or missingness >25%; (2) "forced" a variable based on the SBP into the final prediction model given its strong epidemiologic association with ATC and evidence for a causal mechanism underlying this association; and (3) created merged or collapsed candidate predictors (non-vehicular injury mechanism, shock index) when feasible and supported by bivariable analysis [23,29,36]. Continuous candidate predictors were evaluated without transformation as locally weighted scatterplot smoothing (LOW-ESS) plots revealed no major non-linearity in predictor/ INR relationships.
We adapted the "majority rules" approach to model selection described by Vergouwe et al. [41]. Within each imputed dataset, we evaluated all possible combinations of predictor variables using a best-subsets approach and a leaps-and-bounds algorithm adapted for logistic regression [42][43][44], choosing the model with the lowest Akaike information criterion. This likelihood-based measure of model fit penalizes larger models to reduce overfitting [45]. The final prediction model included predictors selected in 50% or more of the imputationderived models (Additional file 2: Figure S1). Coefficients for the final prediction model were obtained by combining regression coefficients from the 50 imputed datasets using Rubin's rules [46]. We created the Prediction of Acute Coagulopathy of Trauma (PACT) score by rounding raw model coefficients to one decimal place and multiplying by 100.

Evaluation of model performance
We estimated model optimism in the multiply-imputed derivation cohort using bootstrap techniques [47]. After sampling with replacement for 1000 iterations, we performed the previously described model selection procedure on each bootstrap sample and compared model discrimination in the bootstrapped vs. original derivation cohort. The average difference for the 1000 bootstrapped samples is an estimate of the deterioration in model discrimination attributable to sampling bias. To formally test generalizability, we evaluated the discrimination and calibration of the PACT and COAST scores when applied to the validation cohort.

Statistical analysis
For bivariable analyses we employed the unpaired t test with unequal variance or the Mann-Whitney test for continuous variables and the chi-square or Fisher's exact test for categorical variables. Regression coefficients are reported with robust standard errors. Model discrimination measured using the area under the receiver operating characteristic curve (AUROC) is reported with 95% confidence intervals and compared using the method of Delong et al. [48]. Model calibration was evaluated (1) graphically by plotting the observed versus predicted ATC probability across equal quantiles of predicted ATC risk and (2) using the Hosmer-Lemeshow goodness-offit statistic [49]. A p value >0.1 for this statistic indicates no significant divergence of observed from predicted probabilities. As the 7-point COAST score cannot be divided into >7 quantiles, the primary PACT score calibration analysis also used 7 quantiles of predicted risk. For other tests, a p value ≤0.05 was considered significant. We used Stata version 14.0 (StataCorp LP, College Station, TX, USA) for all analyses and adhered to published guidelines for reporting of prediction models [50]. We performed two sensitivity analyses. We tested whether an alternate ATC definition adding partial thromboplastin time (PTT) >60 seconds to INR >1.5 altered our results. We also reevaluated the calibration of our model using deciles of ATC risk predicted by the PACT score.

Results
The model derivation cohort included 1963 patients enrolled in the Oregon Trauma Registry between 2008 and 2012 (Additional file 3: Figure S2). ATC was present in 115 patients (5.9%). Coagulopathic patients were more severely injured, less likely to be injured while operating or riding in a motor vehicle, motorcycle or bicycle, more likely to undergo prehospital interventions and had lower prehospital SBP and GCS (Table 2). Compared to the derivation cohort, the 285 subjects included in the validation cohort (Additional file 3: Figure S3) had a slightly higher ATC incidence (9.1%), were more severely injured, and displayed greater physiologic derangements (Table 3). In-hospital mortality was 46% in subjects with ATC compared to 7% in subjects without ATC (p < 0.001) in the derivation cohort and 24% vs. 7% (p = 0.001) in the validation cohort.
Application of the PACT score to the independent validation cohort yielded an AUROC of 0.80 (95% CI 0.72-0.88). The PACT score AUROC was significantly greater than the COAST score AUROC (0.70, 95% CI 0.60-0.80, p = 0.032 for comparison; Fig. 1). Including PTT >60 seconds in the definition of ATC yielded similar results (AUROC 0.80 vs. 0.71, p = 0.038). There was no statistical evidence of inadequate calibration for either the PACT score (Hosmer-Lemeshow goodness-of-fit statistic χ df=7 = 4.02, p = 0.77), or the COAST score (χ df=7 = 11.25, p = 0.13). However, graphical evaluation suggested good calibration of the PACT score but an inconsistent relationship between observed and predicted ATC risk at higher COAST score values (Fig. 2). Dividing the PACT score into deciles rather than seven quantiles of predicted risk did not alter these conclusions (χ df=10 = 8.30, p = 0.59).
Setting the PACT score cutoff at ≥196 maximized sensitivity and specificity at 73.1% and 73.8%, respectively (Table 5). Applying this threshold to the validation cohort, 191 of 198 patients (96.5%) with a PACT score <196 were correctly identified as not having coagulopathy. Among those with a positive PACT score, 19 of 87 (21.8%) had coagulopathy. At the COAST score recommended threshold of ≥3, sensitivity was 26.9% and specificity was 86.1%. Of 43 COAST scores ≥3, 36 (84.7%) were false positives (Table 5).

Discussion
We developed and externally validated a model predicting ATC prior to ED arrival in patients with severe trauma. The PACT score, incorporating a small number of objective and readily measured data elements routinely available to prehospital providers, exhibited good discrimination and calibration when tested in an independent trauma cohort and performed better in both domains than the only previously published prehospital ATC prediction tool.
Benefits of prehospital identification, expedited triage, and receiving hospital notification are well-recognized for conditions where time to treatment affects outcomes [31,52,53]. Given the time course of exsanguinationrelated mortality and the early separation of survival curves for patients with and without ATC, the best time to intervene in ATC appears to be within minutes of  [14,54]. We created the PACT score in response to calls for improved ATC recognition within this window of opportunity [55,56]. The implementation of the score in clinical care must await clinical trials of PACT score-guided therapy. In the meantime, stratification of trauma patients according to ATC risk using the PACT score could aid study of this condition's mechanisms and facilitate interventional trials of its treatment. Enrolling patients at high risk of ATC would foster efficient resource use, reduce heterogeneity, and enrich cohorts with the subjects most likely to benefit from a particular treatment, thereby increasing study power. The PACT score demonstrated good ability to discriminate patients with ATC. Discrimination improved in the validation cohort compared to the derivation cohort, suggesting the score has better accuracy in patients who are sicker and/or suffered blunt injury. The PACT score cannot, however, diagnose ATC with perfect accuracy and would benefit from testing against physician judgment. Clinical application therefore largely awaits studies investigating targeted prehospital or "ED doorway" therapies. The appropriate PACT score cutoff will, moreover, depend on the specific application. In a low-prevalence environment, a PACT score ≥160 (92% sensitivity, 59% specificity) could guide treatment selection for low-risk interventions. Alternatively, for a theoretical study recruiting high-risk patients from the validation cohort, a PACT score ≥250 would enroll 38 patients of whom 29% would have ATC. This compares favorably with the COAST score at its recommended threshold (27 subjects, 19% ATC) or unselected enrollment (285 subjects, 9% ATC).
Viscoelastic assays deliver partial results within 10-15 minutes of test initiation, allowing attractively rapid post-admission coagulopathy evaluation at the minority of level 1 trauma centers where these assays are available [57,58]. However, startup costs, assay system interchangeability and reliability issues, and particularly the absence of a consensus outcome-linked viscoelastic ATC definition pose barriers to the application of viscoelastic assays in clinical care and research outside high-volume, high-resource trauma centers [20,57,59,60]. Because the PACT score accelerates ATC risk stratification relative to viscoelastic assays and is applicable in the settings without access to these tests where most trauma patients receive their initial care, we believe the PACT score has a role in ATC research and, eventually, in clinical care.  Consistent with previous studies, patients with ATC had substantially increased mortality. Higher ATC mortality in the less severely injured derivation cohort may reflect differences in timing of cohort entry or outcome variation between a multilevel trauma system and a single high-volume level 1 trauma center [61]. Overall, variables in our model indicate greater injury relative to physiologic reserve, in line with prior research correlating ATC prevalence with injury severity and hypoperfusion [13,62]. Besides suggesting particularly severe injury, the predictive utility of prehospital CPR may also signal a contribution from the type of coagulopathy previously observed in survivors of non-traumatic cardiac arrest [63]. However, this study was not designed to identify ATC risk factors and our results should not be interpreted as evidence of causal associations between the studied predictors and ATC.
Our study strengths include an independent cohort for external model validation, sufficient events per variable tested, and a model selection algorithm balancing the predictive utility of the variables against the risk of overfitting. Whereas complete case analysis would have limited our effective sample size and introduced bias into model development and evaluation [36,64], our approach using multiple imputation to manage missing data avoided excluding patients with missing predictor values and has been widely recommended in recent literature on predictive models [36,50,64]. We nevertheless cannot exclude residual bias due to missing predictor or outcome data.
The COAST score calculation required several approximations due to differences between its derivation dataset and our datasets. We estimated prehospital temperature using a validated extrapolation and applied the original   [22,33]. These modifications may have penalized the COAST score in comparisons with the PACT score.
In parallel to past studies [22], we focused on severely injured patients in order to create a tool for stratifying among patients at risk of ATC rather than for screening unselected trauma patients for ATC. Selection of severely injured patients for both cohorts, however, relied on retrospective application of severity markers and other data available only after hospital admission. Though parallel in this respect to the procedures applied for development and validation of the COAST score [22] and a well-known prediction model for massive transfusion [65,66], cohort selection for prehospital prediction model building and testing would ideally use prehospital data. The PACT score may perform differently if applied to patients identified as severely injured solely from information available prehospital or upon ED arrival.
Our study has several additional limitations. We defined ATC as an INR >1.5 on hospital admission, a validated definition [17] which may nevertheless not capture all mechanismsincluding hyperfibrinolysisrelevant to the impact of the syndrome on trauma outcomes. Substituting the ATC definition employed by Mitra et al. (INR >1.5 or PTT >60 seconds) did not alter our results. As we were unable to exclude subjects with liver disease, the derangement of INR in some subjects may have resulted from preexisting conditions. The model derivation cohort was less severely injured and, as a result, had less physiologic derangement and lower mortality than the validation cohort. Compared to model evaluation in an identically defined cohort, the two cohorts' entry criteria and mortality actually provided a more rigorous generalizability test. Global variations in injury patterns and prehospital care could decrease the accuracy of our prediction model outside of North America. Finally, the lower-than-expected ATC incidence in the validation cohort yielded a suboptimal sample size for model validation [67]. Though it represents one of the few validated prediction tools, repeating the PACT score validation in a larger, more diverse trauma cohort identified from prehospital criteria would be useful to further confirm its generalizability.

Conclusions
In conclusion, we report derivation and external validation of a prediction model that employs objective, routinely collected prehospital data to identify patients at increased risk of ATC. The PACT score exhibited improved discrimination and calibration relative to a previously reported ATC prediction model. Application of the PACT score during study recruitment could aid therapeutic trials by enriching enrolled cohorts with the patients most likely to benefit from treatments targeting coagulopathy.

Funding
This study was supported by training grant T32 HL728735 (IDP), career development grant K23 GM086729 (TRW, EC), and clinical and translational sciences award UL1TR000423 (University of Washington) from the National Institutes of Health, which had no role in study design, data collection, analysis or interpretation, or manuscript preparation.

Availability of data and materials
The derivation cohort dataset supporting the conclusions of this article is available by application to the Oregon Health Authority after approval by the appropriate ethics committees. The validation cohort dataset supporting the conclusions of this article is available, after ethics committee approval, from Dr. Lisa Vande Vusse, the custodian of the dataset from the Age of Transfused Blood and Lung Injury After Trauma Study.
Authors' contributions IDP and TRW conceived the study. IDP, TRW, RVM, TDR, ARR, and LKVV participated in study design. EC, IDP, TRW, and LKVV participated in data collection. EC and IDP processed data. IDP, TRW, TDR, ARR, and LKVV designed the statistical analyses. IDP performed statistical analyses. IDP drafted the manuscript. All authors read, revised, and approved the final manuscript.

Competing interests
Dr. Watkins is currently employed by Gilead Sciences, which is not involved in commercial research relevant to this work. The remaining authors declare that they have no competing interests.

Consent for publication
Not applicable.
Ethics approval and consent to participate The Oregon Health Authority and University of Washington Institutional Review Boards approved use of Oregon Trauma Registry data and granted a waiver of informed consent for retrospective data analysis. Subjects provided informed consent to participate in the observational Age of Transfused Blood and Lung Injury After Trauma Study under a protocol approved by the University of Washington Institutional Review Board, which subsequently granted exempt status to the current secondary analysis.