Skip to main content

Validation of the SOS-PD scale for assessment of pediatric delirium: a multicenter study

Abstract

Backgrounds

Reports of increasing incidence rates of delirium in critically ill children are reason for concern. We evaluated the measurement properties of the pediatric delirium component (PD-scale) of the Sophia Observation Withdrawal Symptoms scale Pediatric Delirium scale (SOS-PD scale).

Methods

In a multicenter prospective observational study in four Dutch pediatric ICUs (PICUs), patients aged ≥ 3 months and admitted for ≥ 48 h were assessed with the PD-scale thrice daily. Criterion validity was assessed: if the PD-scale score was ≥ 4, a child psychiatrist clinically assessed the presence or absence of PD according to the Diagnostic and statistical manual of mental disorders (DSM)-IV. In addition, the child psychiatrist assessed a randomly selected group to establish the false-negative rate. The construct validity was assessed by calculating the Pearson coefficient (rp) for correlation between the PD-scale and Cornell Assessment Pediatric Delirium (CAP-D) scores. Interrater reliability was determined by comparing paired nurse-researcher PD-scale assessments and calculating the intraclass correlation coefficient (ICC).

Results

Four hundred eighty-five patients with a median age of 27.0 months (IQR 8–102) were included, of whom 48 patients were diagnosed with delirium by the child psychiatrist. The PD-scale had overall sensitivity of 92.3% and specificity of 96.5% compared to the psychiatrist diagnosis for a cutoff score ≥4 points. The rp between the PD-scale and the CAP-D was 0.89 (CI 95%, 0.82–0.93; p < 0.001). The ICC of 75 paired nurse-researcher observations was 0.99 (95% CI, 0.98–0.99).

Conclusions

The PD-scale has good reliability and validity for early screening of PD in critically ill children. It can be validly and reliably used by nurses to this aim.

Background

Delirium is an acute dysfunction of the brain and is common in adult intensive care unit (ICU) patients with incidence up to 80% [1,2,3]. Reported prevalence rates of delirium in critically ill children range from 4% to 47% [4,5,6,7,8,9,10], and in a recent study even up to 56% in children below 2 years of age [9]. Delirious patients have disturbed consciousness and/or attention and other changes in cognition that develop over a short period of time. Their behavior may be described as hyperactive, hypoactive, or mixed [11]. In adults, delirium has been associated with, among other things, increased mortality risk, longer ICU stay, complications, and long-term cognitive deterioration [12,13,14]. In pediatric care, delirium is associated with prolonged stay and increased healthcare costs [15, 16]. Therefore, early identification of delirium in children is an urgent medical matter, as treatment of the underlying causes may minimize long-term consequences.

Three tools for assessing delirium in critically ill children have become available: the Cornell Assessment of Pediatric Delirium (CAP-D) [7, 10], the pediatric Confusion Assessment Method for ICU patients (pCAM-ICU) [8], and the preschool CAM-ICU (psCAM-ICU) [9]. Each has its own strengths and limitations.

Earlier we developed and validated the Sophia Observation Withdrawal Symptoms-scale (SOS) to assess iatrogenic withdrawal symptoms (IWS) in critically ill children aged 0–16 years [17, 18]. As IWS is a possible cause of delirium, symptoms of both conditions overlap [19, 20]. From a pilot study we concluded that the SOS scale, extended with a pediatric delirium (PD) component (SOS-PD scale), had promising validity and reliability [21]. We aimed to determine the measurement properties of the PD part of the SOS-PD scale for use by nurses, in terms of construct validity, criterion validity, and interrater reliability.

Methods

Design

A multicenter prospective observational study with repeated measures was conducted in four PICUs of university hospitals in the Netherlands. Data were collected over 6 months in each center, in the period from December 2015 through August 2016. Nurses applied the PD-scale (the PD component of the SOS-PD scale) at least three times a day, once every shift, in all eligible patients.

Setting and study population

Children between 3 months and 18 years of age admitted to a PICU and with an expected length of stay of at least 48 h were eligible for inclusion. Exclusion criteria were the following: anticipated death within 48 h; neurological abnormality (e.g. severe psychomotor delay, encephalitis); comatose, or deeply sedated (COMFORT behavior score < 11, or Richmond Agitation-Sedation Scale (RASS) < −3), or paralysis through neuromuscular blocking agents during the whole admission period, which would make behavioral assessment impossible. The local institutional review board (EMC-2013-545) approved the study and patients were included after informed consent from the parents.

Instruments

The PD-scale consists of 17 items that represent symptoms of PD and an item that reflects the perspective of the child’s parents (“parents do not recognize their child’s behavior as normal”) (Fig. 1) [21]. Presence of a symptom is scored as “yes” if it was observed at any moment during the previous 4 h. If the items “hallucinations” or “parents do not recognize their child’s behavior” were positive, a score of 4 points was assigned. Therefore, if only the item hallucinations is scored as yes, it overrides the PD score < 4 – by which the threshold for the index test (SOS-PD scale) is reached. The same holds for the parents’ item. The maximum sum score of the PD-scale is 17 points. In our pilot study, the sensitivity was 96.8% (95% CI, 80.4–99.5%) and the specificity was 92.0% (95% CI, 59.7–98.9%) for the cutoff point ≥4 [21]. The intraclass correlation coefficient (ICC) from 16 paired nurse-researcher observations was 0.90 (95% CI, 82.7–99.4) [21]. Given the lack of data on false-negatives and the small sample size of the pilot study, the validity should be reproduced in a large multicenter study.

Fig. 1
figure 1

Sophia Observation Withdrawal Symptoms Scale (SOS-PD)

The construct validity was tested by comparing outcomes on the PD-scale with outcomes on the revised CAP-D [10]. The CAP-D is a reliable and valid tool for assessing delirium in critically ill children of 0–18 years of age and consists of eight items scored 0–4. A total score ≥9 is consistent with a diagnosis of delirium [10].

Study procedure

Local research nurses who had been trained by the principal investigator (EI) trained all nurses involved in the study in applying the PD-scale. The research nurses were also trained in applying the CAP-D. After the unit nurses had received theoretical instruction, they applied the PD-scale while watching video material of three cases of PD diagnosed by a child psychiatrist (one case of hypoactive delirium and two cases of hyperactive delirium). The nurses’ scores were compared with the reference score provided by the instructor. Discrepancies between the reference score and nurses’ scores were explained and advice was given for observation in clinical practice.

Patients meeting the inclusion criteria were included in the study 48 h after admission. Assessments were not performed if the patient was over sedated (COMFORT behavior score <11 or RASS < −3) or comatose [10, 22, 23]. In this situation, clinical assessment for delirium is useless. Caregiving nurses applied the PD-scale every shift at set times, i.e. 0400, 1400, and 2000 h.

A PD-scale score ≥4 or a positive score on the item “hallucination” was reason to consult a child psychiatrist (“suspected delirium”, Additional file 1: File S1). This threshold was based on our previous study and defined prior to the data collection [21]. If the patient was still admitted the next day, PD-scale assessments and psychiatrist consultation were repeated.

To estimate the number of false negatives, a random selection of patients with a PD-scale score <4 was assessed by a child psychiatrist on weekdays (Additional file 1: File S1). If at a particular moment more than one patient had been assigned a SOS-PD score <4 (random sample) the the local study coordinator determined, with the use of opaque envelops containing the bed numbers of the eligble patients, which patient (with a maximum of 2 patients) would undergo psychiatric evaluation. The psychiatric standardized examination is non-invasive (no physical examination) and includes a succinct assessment of consciousness, attention, orientation, or other disturbances in cognitive functioning according to the DSM-IV criteria (“gold standar”) [11]. The psychiatrist requested additional information (allo-anamnesis) from the parents, treating physician, and nurse, and reviewed the medical history (medication use, type of illness, etc.). The full examination took 40–50 min. The attending nurse performed the PD assessment within 1 h before the psychiatric evaluation and did not discuss the results with the child psychiatrist. A diagnosis of delirium was reported to the medical team so that appropriate measures could be taken. Regarding the first item on the PD-scale, the nurses asked the parents whether their child behaved differently from “normal” in relation to illness and previous days (e.g. hallucinations, withdrawn, making no eye contact). If parents said “This now is not my child, I don’t recognize my child anymore”, this could be a sign of the hyperactive as well as the hypoactive form of delirium, especially in very young children.

In addition, to establish the interrater reliability in clinical (bedside) circumstances a research nurse in each center performed at least 15 paired observations, a convenience sample, with a caregiving nurse, based on the availability of patients and nurses. Further, during weekdays the research nurse independently assessed patients with the CAP-D while simultaneously the attending nurse was performing the PD assessment.

Additional clinical data

The following additional data were collected: birth date, gender, reason for admission, type of respiratory support, length of stay in the PICU, Pediatric Risk of Mortality Score (PRISM) III-24 (first 24 h), type of continuous infusion of sedatives and opioids, and significant clinical developmental delay, based on clinical assessment and/or parental report of developmental problems that affected the child’s behavior or ability to communicate. Children with mild or transient history of developmental problems (i.e., needing occupational therapy for motor or speech delays) but who currently did not have communication or behavior problems were classified as normal [10].

Sample size calculation

Assuming a delirium prevalence of 15%, and point estimate for sensitivity of 90%, a sample size of 22 patients with delirium was required for the analysis. The sample size was calculated to ensure the appropriate number of patients necessary to provide the lower bound for the 95% CI of 70%. With 126 patients without delirium, the lower bound of the 95% CI for specificity was estimated to be 96%. So, a sample size of 148 patients was required for the analysis. We expected that 10% of patients would have missing data. Thus, the total sample size required for the investigation was 163 patients. This corresponded with 45 patients per participating PICU (n = 4).

Statistical analysis

Demographics were summarized using descriptive statistics. The frequencies of the PD-scale items were divided into 3e groups: (1) assessments in the whole patient group; (2) assessments in delirious patients after diagnosis by the child psychiatrist; and (3) assessments in delirious patients till 48 h after diagnosis by the child psychiatrist. Data on patients with and without delirium were compared using the chi-square test for categorical variables and the Mann-Whitney test for continuous variables. These tests were also used to compare the demographic variables (age, reasons for admission, respiratory support, and severity of illness) of patients with suspected delirium and random psychiatric assessment.

Reliability

Interrater reliability of the PD-scale was assessed using Cohen’s kappa for the dichotomous items and using an intraclass correlation coefficient (ICC) for average measures with a two-way random effects model for continuous data [24]. Cohen’s kappa > 0.65 was considered satisfactory [25].

Validity

Criterion validity was defined as the ability of the PD-scale to classify patients into normal and PD categories compared with the psychiatrist’s verdict using the DSM-IV criteria. This was assessed by calculating the sensitivity, specificity, positive and negative predictive value, and the positive and negative likelihood ratios. Because the child psychiatrist was consulted for only a random selection of patients with a PD score <4, and in all patients with a PD score ≥4, naive estimates of test characteristics will be biased due to partial verification bias. To correct for this problem, we applied the formulas of Begg et al. [26] with the PD score as an ordinal variable. A receiver-operating-characteristic curve and associated area under the curve with adjustment for partial verification bias were calculated using the method of Zhou [27]. For this analysis one observation for each patient was used, that is the first assessment by the psychiatrist for patients with at least one such assessment, and the first observation of the PD-score for all patients without an assessment by the psychiatrist.

We tested the construct validity of the PD-scale by comparing the scores with the CAPD scale scores, on the assumption that both measure delirium in critically ill children. In line with the Consensus-based standards for the selection of health measurement instruments (COSMIN) checklist, we hypothesized that the correlation coefficient between the CAP-D and PD-scale was moderate and at least 0.65 [28, 29].

Results

Of 585 patients screened for eligibility, 100 were excluded (see Fig. 2). The 485 patients included in the analysis had a median age of 27.0 months (IQR 8–102) and 59.6% of them were boys. Almost half (42.5%) were admitted with respiratory failure and 56.9% (276/485) spent time on the ventilator (Table 1). More than half of the patients received sedatives (e.g. midazolam, lorazepam) or opioids, 56.1% (272/485) and 57.3% (278/485), respectively (Table 2).

Fig. 2
figure 2

Inclusion flowchart. PICU, pediatric intensive care unit

Table 1 Demographic variables for the patient groups with and without confirmed delirium (n = 485)
Table 2 Continuous infusion of sedatives and opioids (n = 485)

In total, 48 patients (9.9%) were diagnosed with delirium. One patient was identified as delirious twice during the PICU admission (Table 3). These patients were admitted for significantly longer (p < 0.001), were significantly older (p < 0.042) and significantly more of them spent time on the ventilator (p < 0.001) than patients without delirium. Overall, 182 psychiatric evaluations were performed in 168 patients, 62 evaluations for suspected delirium and 120 evaluations for at-random assessment (Table 3). Informed consent was provided for 168 of the 173 patients (97%) who were eligible for psychiatric evaluation. No data were excluded in the case of inconclusive or uninterpretable index or reference tests. The time interval between the SOS-PD assessment and psychiatrist’s assessment was less than 1 h in more than 95% of cases. These two groups of patients were not significantly different with respect to demographic and clinical variables.

Table 3 Total numbers of psychiatric assessments performed

PD-scale scores

Nurses performed 5207 PD-scale assessments in 485 patients, which corresponds to a median number of 5 (IQR 2–13) assessments per patient. The median PD score was 0 (IQR 0–1; n = 5207) for the total population, and the median number of days that patients were assessed was 3 days (IQR 1–7). The frequencies of items of the PD-scale are shown in Fig. 3 for the different groups. Purposeful acting, agitation, anxiety, motor disturbance, attentiveness, lack of eye contact, and sleeplessness (34.9–46.3%) were the most prevalent in patients with delirium.

Fig. 3
figure 3

Observed delirium symptoms of the Pediatric Delirium Scale (PD scale). Every item is represented by 3 bars. The blue bars (Group 1) show the frequency of the item for all assessments (5207) in the whole patient group (n = 485). The yellow bars (Group 2) show the frequency of the item for assessments (758) in delirious patients after diagnosis by the child psychiatrist (n = 48). The green bars (Group 3) show the frequency of the item for assessments (229) in delirious patients up to 48 h after diagnosis by the child psychiatrist (n = 48)

Measurement properties of the PD-scale

Reliability

The attending nurse and the researcher made 75 paired PD-scale assessments. The ICC for the total sum score between the attending nurse and researcher was 0.99 (95% CI, 0.98–0.99). The interrater reliability (Cohen’s kappa) for the individual items ranged from 0.79 to 1.0 (See Additional file 1: File S2).

Criterion validity

To establish the criterion validity of the PD-scale, the first child psychiatrist assessment was compared with the PD score in the randomly selected patients (PD-scale score <4; n = 114) and for the patients with suspected delirium (PD-scale score ≥4; n = 54), respectively (Table 4). In three cases in which the PD score was ≥ 4, the child psychiatrist was not consulted for logistic reasons. The sensitivity was 92.3% and the specificity was 96.5% after correction for verification bias, for a PD score ≥4. Figure 4 shows the receiver-operating characteristic-curve, in which the area under the curve was 0.989. The positive predictive and the negative predictive values (without adjustment for verification bias) were 76.4% and 99.1%, respectively. The positive likelihood ratio and the negative likelihood ratio (based on partial verification bias) were 26.5 and 0.08, respectively. Delirium was diagnosed on median day 8 (IQR 5–14) after admission. The estimated prevalence of PD was 10% (with adjustment for verification bias). The observed prevalence of delirium (without adjustment for verification bias) was highest in patients who had been on invasive ventilator support at any time (44/232, 18.9%) followed by the age group > 12 years of age (10/63, 15.9%). The hyperactive subtype of delirium was the most common, affecting 43.8% of patients. The hypoactive subtype occurred in 33.3% and the mixed-subtype in 22.9% of patients. At a cutoff point ≥4, there were one false-negative PD assessment and 14 false-positive screens among the observations with an assessment by the child psychiatrist. The false negative PD assessment was a 16-month-old boy with an infection who was on non-invasive ventilation. The caregiving nurse assessed the child and scored 3 points on the PD-scale (agitation, purposeful acting, and inconsolable crying). The child psychiatrist reported signs of decreased awareness and attention, cognition, withdrawn affection, and fluctuation, and concluded the patient had hypoactive delirium. In the next shift the score on the PD-scale was 6.

Table 4 Numbers of psychiatric assessments performed for criterion validity – correcting for verification bias
Fig. 4
figure 4

Receiver-operating characteristic (ROC) curve for performance of the Pediatric Delirium (PD) part of the Sophia Observation Withdrawal Symptoms Scale (SOS-PD). The area under the cure is 0.989. The calculation of the ROC curve has been adjusted for partial verification bias using the method of Zhou [27]

Construct validity

In 144 cases, we compared the PD-scale scores of the caregiving nurse with the CAP-D scores of the research nurse. The median PD-scale score was 1 (IQR 0–1; n = 144) and the median CAP-D score was 4 (IQR 2–6). The Pearson coefficient of correlation between the sum scores of the PD-scale and the CAP-D was 0.89 (95% CI, 0.82 to 0.93; p < 0.001).

Discussion

Delirium in critically ill children is a global problem and is increasingly recognized in this vulnerable patient group with a prevalence ranging from 4 up to 47% [4,5,6,7,8,9,10], and 10% in the current study. In this multicenter validation study, the PD-scale demonstrated high sensitivity (92%) and specificity (97%) for the detection of pediatric delirium – with one false-negative case and 14 false-positive cases. The likelihood ratios in this study can be labeled “strong” and provide robust evidence to rule in or rule out, respectively, the diagnosis of PD in most circumstances [30]. Further, the interrater reliability for the individual items had improved compared to our pilot study [21]. Altogether, these results confirm the measurement properties of the PD-scale established in our single-center pilot study [21]. Furthermore, the correlation between the PD-scale and the CAP-D was high, reflecting that both measure the same construct. Overall, the measurement properties of the PD-component of the SOS-PD scale are in line with those of the CAPD, pCAM-ICU, and psCAM-ICU [7,8,9,10, 31].

Schieveld and Zwieten recommended to develop a uniform observational screening tool across the entire age span [32] that could create a common diagnostic language and standardize the process of diagnosing delirium. However, developmental differences across the patient populations, such as in the elderly with dementia or comorbidity, with concomitant different expression patterns of delirium, make it unrealistic to develop a one-size-fits-all tool [33]. The primary goal is to implement delirium screening in daily practice, which currently is by far not the case [34]. Nurses are already expected to assess pain and level of sedation and adding a delirium screening tool increases the workload. However, not performing routine delirium screening might delay diagnosis and therapeutic interventions. The availability of different pediatric delirium screening tools will allow nurses and other healthcare professionals to choose a tool that best fits their needs, based on resources and ease of implementation. For instance, a screening tool that assesses the clinical picture “at the moment” such as the psCAM-ICU and the pCAM-ICU, or a tool such as the CAPD or the SOS-PD to assess the course during a nursing shift. Both the CAPD and the SOS-PD will detect delirium-associated fluctuations in brain dysfunction, such as consciousness and inattention, and therefore we believe the latter can be more appropriately used by caregiving nurses.

We propose that the effects of preventive and pharmacological and non-pharmacological treatment interventions be evaluated with the currently available validated delirium tools. Haloperidol is currently one of the most used antipsychotics for PD, although its efficacy for this indication is not proven [35,36,37]. There may be a larger role for nonpharmacological interventions than has been the case so far [38].

In this study, we found an overall estimated prevalence rate of delirium of 10% in critically ill children. This is lower than rates reported in recent studies, ranging from 17% up to 47% [8,9,10, 39]. The lower prevalence could be explained by the fact that our study patients were included 48 h after admission, in line with the Dutch guideline Pediatric Delirium [37], and therefore missing many self-limiting cases of emergence delirium. In this study, most of the delirious patients (75%) were diagnosed at day 5 or later. We may have missed patients with delirium; however, this study was not set up as an epidemiological study. Interestingly, although a single-center study on risk factors found that young age (≤2 years) is an independent risk factor of delirium, we identified the lowest estimated prevalence for this age group compared to older children in our multicenter study [39].

This multicenter validation study in four Dutch PICUs was performed in daily practice, in which the PD-scale assessments were made by well-trained PICU nurses. This increases the accuracy of the results of this study. Still, several limitations need to be addressed. First, the design of the study could have introduced observer bias in the “random sample”. To counteract this, we corrected for verification bias. Second, the study design may have introduced selection bias, because selected patients with a PD score <4 were chosen for psychiatric evaluation. Also, the paired observations, assessing the interrater reliability and construct validity, were performed in convenience samples and mostly in patients who were not delirious. The latter is a challenge in future studies. Third, the psychiatrist would know whether the consultation was for suspected delirium (SOS-PD score ≥4), as the attending physician customarily reported this 24 h/7 days. Besides, the psychiatric examination required additional information from the parents, attending physician, and attending nurse. These aspects could have introduced test bias. Fourth, the PD-scores were compared to the psychiatric examination according to the DSM-IV criteria, which was standardized, which is a strength. Yet we cannot rule out that the psychiatric teams in the participating hospitals assessed in different ways, with risk of subjectivity. Better accuracy could have been ensured by one team of child psychiatrists assessing all random samples, but this was impossible for logistical reasons. Fifth, even though the care-giving nurses were asked not to share the results of their PD-scale assessments with the psychiatrists, they could have non-verbally influenced the child psychiatrist.

Conclusions

The SOS-PD scale has good interrater reliability and validity for screening by nurses of delirium in critically ill children. Early screening and routine monitoring of delirium with the SOS-PD scale could facilitate early capture and management of critically ill children with suspected delirium.

Abbreviations

CAPD-D:

Cornell Assessment Pediatric Delirium

COSMIN:

Consensus-based Standards for the Selection of Health Measurement Instruments

DSM:

Diagnostic and statistical manual of mental disorders

ICC:

Intraclass correlation coefficient

ICU:

Intensive Care Unit

IWS:

Iatrogenic withdrawal symptoms

pCAM-ICU:

Pediatric Confusion Assessment Method for ICU patients

PD-scale:

Pediatric Delirium Scale

PICU:

Pediatric Intensive Care Unit

PRISM:

Pediatric Risk of Mortality Score

psCAM-ICU:

Preschool Confusion Assessment Method for ICU patients

RASS:

Richmond Agitation Sedation Scale

SOS scale:

Sophia Observation Withdrawal Symptoms Scale

SOS-PD scale:

Sophia Observation Withdrawal Symptoms-Pediatric Delirium scale

References

  1. Ely EW, Gautam S, Margolin R, Francis J, May L, Speroff T, Truman B, Dittus R, Bernard R, Inouye SK. The impact of delirium in the intensive care unit on hospital length of stay. Intensive Care Med. 2001;27(12):1892–900.

    Article  CAS  Google Scholar 

  2. Ouimet S, Kavanagh BP, Gottfried SB, Skrobik Y. Incidence, risk factors and consequences of ICU delirium. Intensive Care Med. 2007;33(1):66–73.

    Article  Google Scholar 

  3. van den Boogaard M, Schoonhoven L, van der Hoeven JG, van Achterberg T, Pickkers P. Incidence and short-term consequences of delirium in critically ill patients: a prospective observational cohort study. Int J Nurs Stud. 2012;49(7):775–83.

    Article  Google Scholar 

  4. Janssen NJ, Tan EY, Staal M, Janssen EP, Leroy PL, Lousberg R, van Os J, Schieveld JN. On the utility of diagnostic instruments for pediatric delirium in critical illness: an evaluation of the Pediatric Anesthesia Emergence Delirium Scale, the Delirium Rating Scale 88, and the Delirium Rating Scale-Revised R-98. Intensive Care Med. 2011;37(8):1331–7.

    Article  Google Scholar 

  5. Larsen GY, Donaldson AE, Parker HB, Grant MJ. Preventable harm occurring to critically ill children. Pediatr Crit Care Med. 2007;8(4):331–6.

    PubMed  Google Scholar 

  6. Schieveld JN, Leroy PL, van Os J, Nicolai J, Vos GD, Leentjens AF. Pediatric delirium in critical illness: phenomenology, clinical correlates and treatment response in 40 cases in the pediatric intensive care unit. Intensive Care Med. 2007;33(6):1033–40.

    Article  CAS  Google Scholar 

  7. Silver G, Traube C, Kearney J, Kelly D, Yoon MJ, Moyal WN, Gangopadhyay M, Shao HB, Ward MJ. Detecting pediatric delirium: development of a rapid observational assessment tool. Intens Care Med. 2012;38(6):1025–31.

    Article  Google Scholar 

  8. Smith HA, Boyd J, Fuchs DC, Melvin K, Berry P, Shintani A, Eden SK, Terrell MK, Boswell T, Wolfram K, et al. Diagnosing delirium in critically ill children: validity and reliability of the Pediatric Confusion Assessment Method for the intensive care unit. Crit Care Med. 2011;39(1):150–7.

    Article  Google Scholar 

  9. Smith HA, Gangopadhyay M, Goben CM, Jacobowski NL, Chestnut MH, Savage S, Rutherford MT, Denton D, Thompson JL, Chandrasekhar R, et al. The Preschool Confusion Assessment Method for the ICU: valid and reliable delirium monitoring for critically ill infants and children. Crit Care Med. 2016;44(3):592–600.

    Article  Google Scholar 

  10. Traube C, Silver G, Kearney J, Patel A, Atkinson TM, Yoon MJ, Halpert S, Augenstein J, Sickles LE, Li C, et al. Cornell Assessment of Pediatric Delirium: a valid, rapid, observational tool for screening delirium in the PICU. Crit Care Med. 2014;42(3):656–63.

    Article  Google Scholar 

  11. American Psychiatric Association. Diagnostic and statistical manual of mental disorders, DSM-IV-TR. Washington, DC: American Psychiatric Association; 2000.

    Google Scholar 

  12. Pandharipande PP, Girard TD, Jackson JC, Morandi A, Thompson JL, Pun BT, Brummel NE, Hughes CG, Vasilevskis EE, Shintani AK, et al. Long-term cognitive impairment after critical illness. N Engl J Med. 2013;369(14):1306–16.

    Article  CAS  Google Scholar 

  13. Mehta S, Cook D, Devlin JW, Skrobik Y, Meade M, Fergusson D, Herridge M, Steinberg M, Granton J, Ferguson N, et al. Prevalence, risk factors, and outcomes of delirium in mechanically ventilated adults. Crit Care Med. 2015;43(3):557–66.

    Article  Google Scholar 

  14. Silver G, Traube C, Gerber LM, Sun XM, Kearney J, Patel A, Greenwald B. Pediatric Delirium and Associated Risk Factors: a single-center prospective observational study. Pediatr Crit Care Me. 2015;16(4):303–9.

    Article  Google Scholar 

  15. Smeets IA, Tan EY, Vossen HG, Leroy PL, Lousberg RH, van Os J, Schieveld JN. Prolonged stay at the paediatric intensive care unit associated with paediatric delirium. Eur Child Adolesc Psychiatry. 2010;19(4):389–93.

    Article  Google Scholar 

  16. Traube C, Mauer EA, Gerber LM, Kaur S, Joyce C, Kerson A, Carlo C, Notterman D, Worgall S, Silver G, et al. Cost associated with pediatric delirium in the ICU. Crit Care Med. 2016;44(12):e1175–9.

    Article  Google Scholar 

  17. Ista E, van Dijk M, de Hoog M, Tibboel D, Duivenvoorden HJ. Construction of the Sophia Observation withdrawal Symptoms-scale (SOS) for critically ill children. Intensive Care Med. 2009;35(6):1075–81.

    Article  Google Scholar 

  18. Ista E, de Hoog M, Tibboel D, Duivenvoorden HJ, van Dijk M. Psychometric evaluation of the Sophia Observation Withdrawal Symptoms Scale in critically ill children. Pediatr Crit Care Med. 2013;14(8):761–9.

    Article  Google Scholar 

  19. van Dijk M, Knoester H, van Beusekom BS, Ista E. Screening pediatric delirium with an adapted version of the Sophia Observation Withdrawal Symptoms scale (SOS). Intensive Care Med. 2012;38(3):531–2.

    Article  Google Scholar 

  20. Madden K, Burns MM, Tasker RC. Differentiating delirium from sedative/hypnotic-related iatrogenic withdrawal syndrome: lack of specificity in pediatric critical care assessment tools. Pediatr Crit Care Med. 2017;18(6):580–8.

    Article  Google Scholar 

  21. Ista E, Te Beest H, van Rosmalen J, de Hoog M, Tibboel D, van Beusekom B, van Dijk M. Sophia Observation Withdrawal Symptoms-Paediatric Delirium scale: a tool for early screening of delirium in the PICU. Aust Crit Care. 2018;31(5):266–73.

    Article  Google Scholar 

  22. Ista E, van Dijk M, Tibboel D, de Hoog M. Assessment of sedation levels in pediatric intensive care patients can be improved by using the COMFORT “behavior” scale. Pediatr Crit Care Med. 2005;6(1):58–63.

    Article  Google Scholar 

  23. Kerson AG, DeMaria R, Mauer E, Joyce C, Gerber LM, Greenwald BM, Silver G, Traube C. Validity of the Richmond Agitation-Sedation Scale (RASS) in critically ill children. J Intensive Care. 2016;4:65.

    Article  Google Scholar 

  24. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86(2):420–8.

    Article  CAS  Google Scholar 

  25. Fleiss JL. Statistical methods for rates and proportions. New York: Wiley; 1981.

    Google Scholar 

  26. Begg CB, Greenes RA. Assessment of diagnostic tests when disease verification is subject to selection bias. Biometrics. 1983;39(1):207–15.

    Article  CAS  Google Scholar 

  27. Zhou XH. A nonparametric maximum likelihood estimator for the receiver operating characteristic curve area in the presence of verification bias. Biometrics. 1996;52(1):299–305.

    Article  CAS  Google Scholar 

  28. Mokkink LB, Terwee CB, Knol DL, Stratford PW, Alonso J, Patrick DL, Bouter LM, de Vet HC. The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: a clarification of its content. BMC Med Res Methodol. 2010;10:22.

    Article  Google Scholar 

  29. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, Bouter LM, de Vet HC. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res. 2010;19(4):539–49.

    Article  Google Scholar 

  30. Deeks JJ, Altman DG. Diagnostic tests 4: likelihood ratios. BMJ. 2004;329(7458):168–9.

    Article  Google Scholar 

  31. Daoud A, Duff JP, Joffe AR, Alberta Sepsis N. Diagnostic accuracy of delirium diagnosis in pediatric intensive care: a systematic review. Crit Care. 2014;18(5):489.

    Article  Google Scholar 

  32. Schieveld JN, van Zwieten JJ. From pediatrics to geriatrics: toward a unified standardized screening tool for delirium: a thought experiment. Crit Care Med. 2016;44(9):1778–80.

    Article  Google Scholar 

  33. Smith HA, Han JH, Ely EW. Meeting the challenges of delirium assessment across the aging spectrum. Crit Care Med. 2016;44(9):1775–7.

    Article  Google Scholar 

  34. Kudchadkar SR, Yaster M, Punjabi NM. Sedation, sleep promotion, and delirium screening practices in the care of mechanically ventilated children: a wake-up call for the pediatric critical care community. Crit Care Med. 2014;42(7):1592–600.

    Article  CAS  Google Scholar 

  35. Barr J, Fraser GL, Puntillo K, Ely EW, Gelinas C, Dasta JF, Davidson JE, Devlin JW, Kress JP, Joffe AM, et al. Clinical practice guidelines for the management of pain, agitation, and delirium in adult patients in the intensive care unit. Crit Care Med. 2013;41(1):263–306.

    Article  Google Scholar 

  36. Bledowski J, Trutia A. A review of pharmacologic management and prevention strategies for delirium in the intensive care unit. Psychosomatics. 2012;53(3):203–11.

    Article  CAS  Google Scholar 

  37. Schieveld JN, de Graeff-Meeder ER, Kalverdijk LJ, Gerver JAM, Knoester H, de Neef M, Ista E, de Hoog M, Koomen E, Catsman-Berrevoets CE, et al. Dutch multidiciplinary guideline for pediatric delirium. Utrecht: Dutch Psychiatric Association (Nederlands Vereniging voor Psychiatric - NVvP; 2014.

    Google Scholar 

  38. Simone S, Edwards S, Lardieri A, Walker LK, Graciano AL, Kishk OA, Custer JW. Implementation of an ICU bundle: an interprofessional quality improvement project to enhance delirium management and monitor delirium prevalence in a single PICU. Pediatr Crit Care Med. 2017;18(6):531–40.

    Article  Google Scholar 

  39. Traube C, Silver G, Gerber LM, Kaur S, Mauer EA, Kerson A, Joyce C, Greenwald BM. Delirium and mortality in critically ill children: epidemiology and outcomes of pediatric delirium. Crit Care Med. 2017;45(5):891–8.

    Article  Google Scholar 

Download references

Acknowledgements

Ko Hagoort MA, Erasmus MC, for editing the manuscript; Sandra Dijkstra RN, Department of Pediatrics, Division of Pediatric Critical Care Medicine, Beatrix Children’s Hospital, University Medical Center Groningen, The Netherlands; Leo Bakker MSc RN, Department of Intensive Care, Radboud University Medical Center, Nijmegen, The Netherlands; Nathalie Wils MD, child psychiatrist, Karakter, Nijmegen, The Netherlands; Gijs Vos MD PhD, pediatric-intensivist PICU, MUMC+, Maastricht, The Netherlands; Jan Schieveld MD PhD, child psychiatrist, MUMC+, Maastricht, The Netherlands; Aimee Bogman MANP, MUMC+, Maastricht, The Netherlands; Wilma Kamp MD, child psychiatrist, Accare, Groningen, The Netherlands.

Funding

This study was partly funded by the Coolsingel Foundation (389), Rotterdam, The Netherlands, and the Sophia Foundation (B15–03), Rotterdam, The Netherlands. The funding organizations had no role in the statistical analyses or publication decisions.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Author information

Authors and Affiliations

Authors

Contributions

EI, MvD, BvB, GD, and BD designed the study. EI, MK, JL, and AB coordinated the study. EI, MK, JL, and AB were responsible for patient screening and enrollment. EI, JR, and MvD analyzed the data. EI, MvD, and JR wrote the manuscript. All authors had full access to all study data, contributed to drafting the manuscript or critical revision of it for important intellectual content, approved the final version of the manuscript, and took responsibility for the integrity of the data and the accuracy of the data analysis. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Erwin Ista.

Ethics declarations

Ethics approval and consent to participate

This study was approved by the local institutional review board of the Erasmus Medical Center (EMC-2013-545) and patients were included after informed consent from the parents.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional file

Additional file 1:

File S1. Study procedure, flowchart study procedure. File S2 Interrater reliability (Cohen’s kappa) for the individual items of the PD-scale. (DOCX 77 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ista, E., van Beusekom, B., van Rosmalen, J. et al. Validation of the SOS-PD scale for assessment of pediatric delirium: a multicenter study. Crit Care 22, 309 (2018). https://doi.org/10.1186/s13054-018-2238-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13054-018-2238-z

Keywords