Open Access

Satisfaction with quality of ICU care for patients and families: the euroQ2 project

  • Hanne Irene Jensen1, 2Email author,
  • Rik T. Gerritsen3,
  • Matty Koopmans3,
  • Lois Downey4, 5,
  • Ruth A. Engelberg4, 5,
  • J. Randall Curtis4, 5,
  • Peter E. Spronk6, 7,
  • Jan G. Zijlstra8 and
  • Helle Ørding1
Critical Care201721:239

https://doi.org/10.1186/s13054-017-1826-7

Received: 22 May 2017

Accepted: 24 August 2017

Published: 7 September 2017

Abstract

Background

Families’ perspectives are of great importance in evaluating quality of care in the intensive care unit (ICU). This Danish-Dutch study tested a European adaptation of the “Family Satisfaction in the ICU” (euroFS-ICU). The aim of the study was to examine assessments of satisfaction with care in a large cohort of Danish and Dutch family members and to examine the measurement characteristics of the euroFS-ICU.

Methods

Data were from 11 Danish and 10 Dutch ICUs and included family members of patients admitted to the ICU for 48 hours or more. Surveys were mailed 3 weeks after patient discharge from the ICU. Selected patient characteristics were retrieved from hospital records.

Results

A total of 1077 family members of 920 ICU patients participated. The response rate among family members who were approached was 72%. “Excellent” or “Very good” ratings on all items ranged from 58% to 96%. Items with the highest ratings were concern toward patients, ICU atmosphere, opportunities to be present at the bedside, and ease of getting information. Items with room for improvement were management of patient agitation, emotional support of the family, consistency of information, and inclusion in and support during decision-making processes. Exploratory factor analysis suggested four underlying factors, but confirmatory factor analysis failed to yield a multi-factor model with between-country measurement invariance. A hypothesis that this failure was due to misspecification of causal indicators as reflective indicators was supported by analysis of a factor representing satisfaction with communication, measured with a combination of causal and reflective indicators.

Conclusions

Most family members were moderately or very satisfied with patient care, family care, information and decision-making, but areas with room for improvement were also identified. Psychometric assessments suggest that composite scores constructed from these items as representations of either overall satisfaction or satisfaction with specific sub-domains do not meet rigorous measurement standards. The euroFS-ICU and other similar instruments may benefit from adding reflective indicators.

Keywords

Quality of care ICU Family Satisfaction Questionnaire survey FS-ICU

Background

In order to improve quality of care, the involvement of patients and their families in health care is a focal point in many countries [1]. This involvement may extend to a variety of healthcare components, from participation in informed decision-making to the provision of feedback on care provided [24]. In the intensive care unit (ICU), although both patients’ and families’ experiences are of great importance [5], patient involvement is complicated by the patient’s critical condition. Approximately 10–20% of patients die in the ICU [68] and a substantial percentage of surviving patients are too sick to be actively involved during their ICU stay, with many unable to remember their ICU experience altogether [9, 10]. Family members often spend considerable time in the ICU and their assessment of the quality of patient care correlates well with patients’ assessments, making it reasonable to use family members to assess care for both the patient and family [11].

Families’ assessments can be obtained in a number of ways, the most common being through interviews and self-administered questionnaires [12]. Open-ended interviews and cognitive debriefing techniques provide valuable, detailed information about individual experiences but generally rely on small samples [12]. By contrast, self-administered questionnaires that use a set of standard items allow a larger number of respondents to provide information, but they do not allow the same in-depth exploration as is afforded by qualitative methods. If such questionnaires are to provide accurate assessments of respondents’ experiences, they must show evidence of strong psychometric characteristics, such as reliability, validity and responsiveness, to ensure that the items and the constructs they represent are appropriate for the populations in whom they are used [12].

A number of instruments are available to measure satisfaction and quality of care in the ICU and are designed to be completed by families of ICU patients [13]. Two of the best known and well-validated are the European “Family Satisfaction in the ICU” (euroFS-ICU), looking at general satisfaction [13, 14], and the “Quality of Dying and Death” (QODD), looking at the quality of events that occur at the end of life [15, 16]. However, both were developed and validated in North America and, as there are cultural differences between North America and Europe [17], use of the instruments without cultural adaptation may decrease validity. Gerritsen and colleagues conducted a Dutch QODD study and found a high prevalence of “not applicable” responses and other missing data, suggesting a need for cultural adaptation [18]. Therefore, in 2012, a Danish-Dutch study aimed at developing a European adaptation of both the FS-ICU and the QODD in a combined “European quality questionnaire” (euroQ2) was undertaken in collaboration with some of the North American developers of the FS-ICU and QODD [19]. The first qualitative and quantitative components of the study showed high face and content validity, suggesting that the instrument may be promising for capturing European ICU families’ experiences and assessments [19].

The goal of this component of the study was to examine assessments of satisfaction with care in a large cohort of Danish and Dutch family members and to conduct a detailed examination of the measurement characteristics of the euroFS-ICU.

Methods

Settings

Participants came from 21 ICUs (11 from Denmark and 10 from The Netherlands) including both university affiliated and regional ICUs from different parts of the two countries.

Inclusion criteria

Family members of patients admitted to the ICU for 48 hours or more, independent of ICU outcome, were eligible for participation. Up to three family members per patient could participate. Family members were defined as the persons closest to the patient (as identified by the patient), including partners, siblings, children, parents and friends. If more than three family members wanted to participate, the family members themselves chose the participants based on who had spent the most time in the ICU.

Exclusion criteria

Family members were excluded who met the following criteria: (1) under age 18 years; (2) with cognitive impairment; or (3) unable to read or write Danish or Dutch.

Recruitment of participants

Family members who fulfilled the eligibility criteria were approached during the patient’s ICU stay by either ICU nurses or physicians; most family members were approached although sometimes ICU nurses and physicians forgot to do so. Family members received oral and written information about the study and, if they agreed to participate, they provided their name and home address. Three weeks after patient discharge from the ICU, family members received the questionnaire by mail, together with written information and a prepaid envelope. In Denmark, the individual ICUs were responsible for sending out the questionnaires, and the cover letter was signed by the local investigators. In the Netherlands, all questionnaires were sent out by the investigators. In both countries, the completed questionnaires were returned to the investigators. If the questionnaire was not returned, one reminder with a new questionnaire was sent.

Patient and respondent data

For participating families, the following patient data were obtained from the medical record: gender, age, medical or surgical specialty of the admitting physician, diagnosis, length of stay in the ICU, and decisions about withholding or withdrawing life-sustaining treatments. The Acute Physiology and Chronic Health Evaluation II (APACHE II) and Simplified Acute Physiology Score (SAPS) were also included when available (from 12 and 13 ICUs, respectively). Data on family respondents included age, gender and relationship to the patient.

Instrument

The euroQ2 questionnaire (see Additional file 1), consists of two sections: the euroFS-ICU, which all participating family members completed, and an extra section containing the euroQODD, which was completed only by family members of patients who died in the ICU. In this paper, we present results for the euroFS-ICU portion of the questionnaire.

Statistics

Statistical analyses were conducted using Stata 13 [20] and Mplus 7.4 [21]. For comparing background characteristics of Danish and Dutch family members and patients we used the chi-squared (χ2) or Fisher’s exact test and the Mann-Whitney U test as appropriate. To compare family members’ responses between countries, we used clustered regression models with country as predictor and the five-point satisfaction items as outcomes. We tested associations of family and patient characteristics with family members’ responses on the family satisfaction items with clustered single-predictor probit regression models (family respondents nested under patients; outcomes defined as ordered categorical variables) estimated with weighted least squares with mean and variance adjustment (WLSMV). P values were based on Wald’s test. Clustered analyses were used to adjust for participation of more than one family member for some of the patients.

Earlier analysis of the North American version of the FS-ICU had suggested that the questionnaire encompassed two domains (care and decision-making), resulting in a recommendation for computing composite scores for those two domains and for total satisfaction [14]. However, that analysis was based on exploratory factor analysis (EFA), with indicators defined as normally distributed continuous variables, and without the use of strict tests of empirical fit. More recent analyses, based on exploratory factor analysis within a confirmatory factor analysis framework (E/CFA) [22] and using a larger sample, with indicators defined as ordered categorical variables, have suggested that the instrument likely encompasses four domains of family satisfaction: (1) communication with the family; (2) empathy shown to the family; (3) support of the family during decision making and (4) management of patients’ symptoms (work by LD, JRC and RAE) (see Additional file 2). Although the euroFS-ICU is an adapted version of the 24-item FS-ICU, many of the items in the two instruments are identical. Therefore, we hypothesized that the euroFS-ICU would encompass dimensions that are conceptually similar to the four domains identified previously in the North American questionnaire.

Examination of the measurement characteristics of the euroFS-ICU included four aspects: (1) positing a conceptual framework for the domain structure of the euroFS-ICU; (2) using exploratory factor analysis (EFA) to simplify the conceptual structure by removing items that contributed to statistically significant misfit (i.e., the χ2 test of fit with p < 0.05) to data from the combined samples; (3) investigating whether the simplified structure was equally appropriate for Denmark and the Netherlands, considered separately and (4) assessing whether a set of “pure” factors (i.e., each indicator contributing to the measurement of only one factor) could be identified, with the resulting factors having equivalent meaning in the two countries. Evidence supporting equivalent meaning between countries required that a model in which the loadings and thresholds for each indicator were constrained to equality between countries produced non-significant misfit to the observed data (i.e., the χ2 test of fit with p > 0.05). Equivalent meaning must be established in order to provide legitimacy for between-country comparisons of mean levels on the factors. Detailed descriptions of the analyses are presented in Additional file 2.

Results

Characteristics of patients and family members

A total of 1077 family members participated, 573 from Denmark and 504 from The Netherlands, representing 920 ICU patients. In Denmark, 185 of the 573 participants were second and third family members of the same patient. In The Netherlands, 6 of the 504 participants were second and third family members. The overall response rate was 72% among family members who were approached and reportedly willing to participate, 75% in Denmark and 68% in The Netherlands. The Dutch and Danish participants differed significantly on a number of demographic and clinical characteristics such as age, relationship to patient, reason for admission and level of therapy (Table 1).
Table 1

Background characteristics of participating family members and patients

 

Total sample

Denmark

The Netherlands

p a

 

Valid n b

Statisticc

Valid n b

Statisticc

Valid n b

Statisticc

 

Family member

 Age, median years (IQRd)

1055

57 (22)

553

54 (22)

502

60 (20)

<0.001

 Female

1056

724 (69)

554

399 (72)

502

325 (65)

0.01

 Relationship to patient, n (%)

1061

 

559

 

502

 

<0.001

 Spouse or partner

 

499 (47)

 

209 (37)

 

290 (58)

 

 Child

 

372 (35)

 

235 (42)

 

137 (27)

 

 Sibling

 

64 ( 6)

 

32 ( 6)

 

32 ( 6)

 

 Parent

 

60 ( 6)

 

37 ( 7)

 

23 ( 5)

 

 Other

 

66 ( 6)

 

46 ( 8)

 

20 ( 4)

 

Patient

 Age, median years (IQR)

894

69 (16)

408

70 (15)

486

68(17)

0.33

 Female, n (%)

894

340 (38)

408

144 (35)

486

196 (40)

0.12

 Days in ICU, median days (IQR)

893

8 (10)

406

9 (11)

487

7 (10)

0.16

 Level of therapy, n (%)

856

 

408

 

448

 

<0.001

 Full

 

630 (74)

 

315 (77)

 

315 (70)

 

 Life-sustaining therapy withheld

 

123 (14)

 

38 (9)

 

85 (19)

 

 Life-sustaining therapy withdrawn

 

103 (12)

 

55 (13)

 

48 (11)

 

 Discharge, n (%)

895

 

408

 

487

 

<0.001

 Planned

 

658 (74)

 

266 (65)

 

392 (81)

 

 Dead

 

178 (20)

 

88 (22)

 

90 (18)

 

 Othere

 

59 ( 7)

 

54 (13)

 

5 ( 1)

 

 Reason for admission, n (%)

894

 

407

 

487

 

<0.001

 Respiratory

 

311 (35)

 

142 (35)

 

169 (35)

 

 Sepsis

 

152 (17)

 

52 (13)

 

100 (21)

 

 Cardiovascular

 

274 (31)

 

119 (29)

 

155 (32)

 

 Other

 

157 (18)

 

94 (23)

 

63 (13)

 

 Mechanical ventilation, n (%)

894

783 (88)

408

346 (85)

486

437 (90)

0.02

 APACHE II, median score (IQR)

509

21 (10)

59

24 (12)

450

21 (10)

0.01

 SAPS II, median score (IQR)

638

50 (24)

277

51 (22)

361

48 (26)

0.09

APACHE II Acute Physiology and Chronic Health Evaluation II, SAPS II Simplified Acute Physiology Score II

aThe Mann-Whitney U test or χ2/Fisher exact test as appropriate

bDifferent numbers due to missing data

cExcept where noted, the statistics provided are number (percentage)

dInterquartile range (percentile75–25)

eIncludes patients who were transferred to other hospitals or who were discharged because of a lack of available beds in the ICU

Between-country comparisons of responses to individual family satisfaction items

Except for inclusion in decision-making processes, the Danish ratings were significantly higher than the Dutch ratings (Table 2). Items with the greatest number of “Excellent” endorsements were concern and caring towards the patient, dyspnea management, atmosphere of the ICU, presence at the bedside and ease of getting information. Items with fewer “Excellent” endorsements and suggesting the need for improvement were management of agitation, emotional support, consistency of information and inclusion in decision-making (Table 2).
Table 2

Family members’ perceptions of ICU quality of care (euroFS-ICU)

 

Total

Denmark

The Netherlands

p a

 

Valid n b

% “Excellent”c

% “Very good”d

Valid n b

% “Excellent”c

% “Very good”d

Valid n b

% “Excellent”c

% “Very good”d

 

Concern and caring toward patient

1070

55.2

34.2

566

63.8

32.0

504

45.6

36.7

<0.001

Symptom management

 Pain

1008

42.8

41.4

547

50.6

41.5

461

33.4

41.2

<0.001

 Breathlessness

928

45.2

37.7

500

55.8

34.6

428

32.7

41.4

<0.001

 Agitation

970

35.8

37.9

513

42.1

39.8

457

28.7

35.9

<0.001

 Atmosphere of the ICU

1075

47.2

35.9

571

53.4

34.7

504

40.1

37.3

<0.001

 Consideration of family needs

1066

40.5

35.9

567

46.0

36.7

499

34.3

35.1

<0.001

 Emotional support

1034

36.3

36.2

550

42.0

37.3

484

29.8

34.9

<0.001

 Opportunity to be present at bedside

1076

51.2

33.0

572

57.0

31.5

504

44.6

34.7

<0.001

 Ease of getting information

1071

45.6

37.6

570

52.5

36.7

501

37.7

38.7

<0.001

 Understanding of information

1070

41.2

40.6

568

43.7

44.7

502

38.5

35.9

0.001

 Honesty of information

1070

44.6

35.7

567

52.7

35.3

503

35.4

36.2

<0.001

Completeness of information

 What was happening

1065

36.7

39.7

566

42.8

41.2

499

29.9

38.1

<0.001

 Why things were being done

1063

37.7

38.2

565

44.8

39.5

498

29.7

36.8

<0.001

 Consistency of information

1057

31.3

36.7

558

37.1

39.6

499

24.9

33.5

<0.001

Overall quality of information

 By doctors

1045

37.2

37..1

550

41.3

37.5

495

32.7

36.8

0.004

 By nurses

1067

40.0

38.7

565

49.7

37.7

502

29.1

39.8

<0.001

 Inclusion in decision-making processes

906

30.8

36.3

466

33.7

36.3

440

27.7

36.4

0.094

 Support during decision-making processes

839

35.8

36.6

436

39.7

42.0

403

31.5

30.8

<0.001

The “Family Satisfaction in the ICU” (euroFS-ICU) is Part 1 of the European quality questionnaire (euroQ2)

a P values for differences between countries were based on a clustered regression model with country as predictor and the five-category satisfaction item as outcome

bExcludes respondents who indicated that the item was inapplicable or who failed to provide a response

cPercentage of family members who indicated that this aspect of care was “Excellent” with the valid number as the denominator

dPercentage of family members who indicated that this aspect of care was “Very good,” with the valid number as the denominator

In addition to the questions presented in Table 2, the euroFS-ICU contains three items that do not use 5-point Likert scale response options: (1) those who chose “Fair” or “Poor” when asked about inclusion in the decision-making processes were subsequently asked why they gave these responses: 114 family members responded to this question (Denmark, n = 65, The Netherlands, n = 49), with 9% stating that they were involved too much, 63% that they were not involved enough, and 28% that their low satisfaction was due to other reasons; (2) the participants were also asked whether they felt they had adequate time to have their concerns addressed and questions answered when major decisions were made, with 72% answering that they had enough time and 9% that they could have used more time: for these two questions there were no statistical differences between the two countries; (3) finally, the participants were asked to assess overall satisfaction with the care the patient had received from all doctors, nurses and other healthcare professionals: the assessment was made on a scale from 0 to 10, with 0 being worst care possible and 10 best care possible; the median assessment was 9 (inter-quartile range 8–10) with significantly higher scores in Denmark (median 9 (9–10)) than in The Netherlands (median 9 (8–9)) (p < 0.001).

Association between respondent characteristics and responses on individual family satisfaction ratings

Whereas there was a significant difference between the two countries for almost all ratings, the respondents’ age, gender and relation to the patient had only a small impact on level of satisfaction. Respondent age influenced six of the items, with higher ratings as age increased. These items were agitation management, atmosphere of the ICU, emotional support, opportunity to be present at the bedside, consistency of information and overall satisfaction with care. The respondents’ gender was significantly associated with four items, with female respondents providing higher ratings, on average, than their male counterparts. Two of the items were about symptom management (management of pain and dyspnea) and two concerned staff communication (willingness to answer questions and provision of understandable explanations). The respondent’s relationship to the patient was not associated with any of the satisfaction ratings (see Additional file 2: Table S1a and S1b for details.)

Association between patient characteristics and responses to individual family satisfaction ratings

The SAPS was significantly associated with satisfaction, with higher scores associated with higher family satisfaction. The SAPS score was associated with 15 items. The items not associated with the SAPS were symptom management (pain, breathlessness and agitation) and adequate time to have concerns addressed. Death in the ICU was associated with higher ratings on seven items including consideration of family needs, emotional support and overall satisfaction with care. The remaining patient characteristics (i.e., gender, age and hours in the ICU) were associated with few or none of the satisfaction items (see Additional file 2: Table S2a-f for details).

Domains of family satisfaction underlying the euroFS-ICU instrument

The first step in investigating the structure of the euroFS-ICU items was to assign each of the 20 items a priori to one of the four conceptual domains (communication, empathy, patient care and symptom management, and decision-making) that have been identified in the North American version of the instrument (Additional file 2: Table S3). To achieve acceptable fit to data from the combined Danish and Dutch samples (please see Additional file 2, p. 1 for details), we generated a series of EFA models, using modification indices that eliminated nine items (five from the communication domain, one from empathy, two from patient care and symptom management, and one from decision-making) from the a priori structure. This produced a four-domain model with strong primary loadings, relatively weak cross-loadings, and good fit to the observed data from the combined countries (Table 3; see Additional file 2, p. 11 for details).
Table 3

Exploratory factor analysis, four-factor eleven-indicator model, merged data from Denmark and the Netherlands (n = 1077): indicator loadings and factor correlations

Indicator

Communication

Empathy

Symptom management

Decision-making

Provision of understandable explanations

0.848 *

0.013

−0.021

0.038

Honesty of information

0.839 *

-0.010

0.015

0.043

Overall quality of information from nurses

0.765 *

0.083*

0.068*

−0.005

Appreciation for family presence

0.195*

0.720 *

0.065*

−0.050*

Consideration of family needs

0.029

0.976 *

−0.059*

0.037

Emotional support of family

−0.029

0.766 *

0.038*

0.165*

Pain management

0.028

0.063*

0.811 *

0.012

Breathlessness management

0.053

−0.076*

0.897 *

0.017

Agitation management

−0.031

0.067*

0.856 *

0.012

Inclusion in decision-making processes

0.134

−0.009

−0.031*

0.785 *

Support during decision-making processes

−0.007

0.038

0.102*

0.873 *

 

Factor correlations

Communication

----

   

Empathy

0.774*

----

  

Symptom management

0.736*

0.730*

----

 

Decision-making support

0.793*

0.689*

0.667*

----

*Statistically significant at p ≤ 0.05

However, although analysis of this EFA model within countries showed acceptable fit to the within-country data, the countries were dissimilar in their pattern of loadings, portending difficulties in establishing a factor structure where the factors had equivalent meanings in the two countries (see Additional file 2, p. 11 for details). Moreover, a confirmatory factor analysis (CFA) in which each indicator was allowed to load on only one of the four factors required further elimination of indicators in order to obtain adequate fit to data from the separate countries, and even this model failed when indicator loadings and thresholds were constrained to equality between countries (see Additional file 2, pp. 12–14). As a result of this failure, we could not conclude that the euroFS-ICU contains elements supporting a four-factor structure for which the factors can be legitimately compared between countries.

Correcting a source of model misspecification

All of the models tested with these data use a methodology that is widely reported for similar instruments. However, it is based on an important type of model misspecification: the modeling of factor indicators as reflective (or effect) indicators, when they are more appropriately modeled as causal indicators [2325]. Reflective indicators are indicators that are caused by (i.e., reflect) a construct, with an individual’s position on all of the indicators tending to rise or fall in concert with that individual’s position on the underlying construct. By contrast, causal indicators are variables that contribute to, rather than reflect, the construct; an individual’s position on some, but not necessarily all, of the causal indicators is expected to rise and fall in concert with the individual’s position on the construct. The difference is in the direction of causation: reflective indicators are caused by the construct; causal indicators contribute to the construct. To achieve statistical identification, modeling a construct with causal indicators requires that there be at least two additional variables that can be used as outcomes of the construct. Ideally, these would be reflective indicators, but they may alternatively be more distal outcomes of the construct. Although the euroFS-ICU includes only one hypothesized domain (the “Communication” domain), for which there are, arguably, reflective indicators, the existence of reflective indicators for this one domain allowed us to test an alternative measurement method.

Figure 1 shows a model in which the quality of ICU communication is measured with a combination of causal and reflective indicators with the additional imposition of between-country measurement invariance. In this model the regression coefficients for the causal indicators and both the factor loadings and thresholds for the reflective indicators were constrained to equality between the two countries. This model provided good fit to the data (p for the χ2 test of fit = 0.4147), thereby providing evidence that the combination of causal and reflective indicators measure a latent communication construct that has equivalent meaning in the two countries and on which the two countries can be legitimately compared.
Fig. 1

Quality of ICU communication, measured with causal and reflective indicators. Measurement invariance imposed between Denmark and The Netherlands. Unstandardized coefficients are shown

The remaining hypothesized domains were not represented by a sufficient set of variables for use as outcomes (either as reflective indicators or as more distal outcomes) to allow causal-indicator tests of those domains.

Discussion

This study was based on a large sample of family members of patients treated in a variety of ICUs in two countries. All questions were assessed as understandable and relevant in the first qualitative and quantitative analyses [19] and, as single items, provide important information about families’ experiences. Overall, family members rated the care provided by ICUs moderately highly, with a large majority of respondents from both countries rating each aspect of care as either excellent or very good, but with respondents from Denmark typically providing higher ratings than were offered by respondents from the Netherlands. Similarly, family members from both countries provided high marks on a single-item rating of overall care provided to patients, but Danish respondents gave higher ratings, on average, than respondents from the Netherlands. However, if the goal is to provide care rated as excellent, many of the items were rated as excellent by only a third to a half of family members. Areas with the highest scores were concern and caring toward patient, dyspnea management, ICU atmosphere, opportunities for family members to be present at the bedside and ease of getting information. Areas with most room for improvement were management of patients’ agitation, emotional support of the family, consistency of information, and inclusion in and support during decision-making processes. Similar levels of satisfaction have been found in a number of ICU family satisfaction studies [14, 2628]. Furthermore, areas for improvement are similar to results from a recent German FS-ICU study [26]. The reasons for Danish ratings being higher than Dutch ratings are unknown. A generally higher nurse-patient ratio (1 nurse to 1–1.4 patients) in Denmark versus 1 nurse to 1–2.5 patients in The Netherlands and more restricted visiting policies in The Netherlands could be contributing factors.

Earlier studies have identified the needs of ICU families, including honest and consistent information [5, 29, 30], possibilities to support, protect and advocate for the patient [29, 30] and emotional support [29, 31]. The development of the euroFS-ICU part of the euroQ2 is based on the substantial work conducted with the FS-ICU [14], subsequent work with the FS-ICU demonstrating a valid domain structure, interviews with Danish families [19] and both qualitative and quantitative tests of whether the questions were relevant, understandable and comprehensive [19]. The literature and our preliminary research therefore support the four hypothesized domains (communication with the family, empathy shown to the family; support of the family during decision-making, and management of patients’ symptoms) as highly relevant for ICU families.

Although exploratory factor analyses identified a set of four domains potentially underlying family satisfaction, successive confirmatory factor analyses (aimed at producing a model in which each indicator measured only one factor) retained only a few indicators from the original set of 20 and failed to fit the data when between-country measurement invariance was imposed. The analyses suggested that the the euroFS-ICU instrument does not measure a unidimensional construct representing overall family satisfaction, nor does it measure four constructs that are comparable between countries. We posited that an important misspecification related to our definition of the component indicators as reflective indicators (i.e., indicators that are caused by a construct), when most of the variables in this instrument function conceptually as causal indicators of their respective constructs (i.e., variables that contribute to, rather than reflect, the construct). Analysis of a single construct (satisfaction with communication) for which the euroFS-ICU instrument includes both causal and reflective indicators provided evidence in support of this hypothesis. One potential approach for the next phase of development of the euroFS-ICU instrument is the addition and testing of a set of reflective indicators of overall satisfaction with the ICU experience, and the addition and testing of at least two reflective indicators for each of the four hypothesized domains. Based on results from this study, we have begun development of extra items that can be used as true reflective indicators. These items will be pilot tested in future research and added to the euroQ2.

Strengths and limitations

Strengths of the study include enrollment of more than 1000 family members from two countries, affiliated with patients who were treated in a large number of ICUs of different types and located in several geographic areas. The response rate among family members approached by ICU staff and willing to consider participation was relatively high (72%) and respondents left few questions unanswered. Despite this high response rate, it was lower than that experienced in an earlier phase of the study (87%), perhaps because the earlier phase included phone contacts to respondents, whereas the current phase used mailed reminders. In addition, the analytic approach in this study was more rigorous than that used for most other measures of family experience. The analyses show the importance of using newer statistical approaches to ensure that multi-item constructs are unidimensional and meet quality standards, as we suspect that other measures may encounter similar challenges of model misspecification in the measurement of latent constructs.

There are also important limitations. SAPS scores were only available for approximately 70% of the sample and from 62% of the ICUs, and the generalizability of these findings may therefore be limited. Additionally, SAPS scores may not discriminate and describe disease severity as well as the APACHE-III scoring and APACHE-IV prediction model, but these scores were not available. If an ultimate objective is to construct multi-item constructs of overall satisfaction and its sub-domains, an important limitation is the absence of reflective indicators of those constructs in the current instrument. Modification of the instrument is already in progress and may allow an exploration of whether such constructs exist and are consistent between countries, or whether contributors to satisfaction vary by country. The validity, reliability, and responsiveness of such measures remain to be determined. Because the current instrument consists primarily of casual indicators, most future analyses with this data set, except for satisfaction with communication, are best limited to the use of single-item measures. A second limitation is the omission of some eligible family members during the study period, owing to ICU staff forgetting to mention the study to them. However, there is nothing to indicate that these omissions were other than random. Likewise, exact numbers of families who refused to participate when approached are missing, but are estimated at less than 10%. A third limitation is that the effect of ethnicity is not examined. As the vast majority of patients in both Denmark and The Netherlands are Caucasians, groups of non-Caucasian family members would be too small for analysis. The lack of ethnic subsamples reduces the generalizability of the study. A fourth limitation is the fact that both of the countries represented in the study are from Northern Europe. Although we identified a model of satisfaction with communication that was invariant for these two countries, it may not fit data provided by ICU families from other parts of the Europe or the world. Addition of data from other European countries and other regions of the world will be important for future studies.

Conclusion

The euroFS-ICU part of the euroQ2 provides information about families’ experiences with ICU quality of care. Areas with the highest scores were concern and caring toward patient, dyspnea management, atmosphere of the ICU, family members’ opportunity to be present at the bedside and ease of getting information. Areas with most room for improvement were management of patients’ agitation, emotional support of the family, consistency of information and inclusion in and support during decision-making processes.

Rigorous psychometric assessments showed that it is problematic to measure overall satisfaction with a composite score or latent construct based on items in the current euroFS-ICU, although a latent construct of one domain (satisfaction with communication) appears to be possible, using a combination of causal and reflective indicators. In the future, this and other instruments may benefit from adding reflective indicators that will allow measuring overall satisfaction, and the three other hypothesized satisfaction sub-domains (satisfaction with symptom management, empathy, and decision-making) as multi-indicator constructs.

Declarations

Acknowledgements

The authors thank all participating families and ICUs.

Funding

The study was supported by The Region of Southern Denmark, The Novo Nordic Foundation, Denmark (11415), The Augustinus Foundation, Denmark (14-2421), and The Frisian ICU Research Fund, The Netherlands. The funding bodies had no influence on the study or the paper.

Availability of data and materials

The datasets generated and/or analyzed during the current study are available from the corresponding author on reasonable request.

Authors’ contributions

HIJ, RTG, MK, JRC and HØ designed the study. HIJ, RTG and MK were responsible for data collection. JRC, RE, PS, JZ and HØ were advisory contributors during data collection. LD had main responsibility for the statistical analyses together with HIJ, RTG, MK, JRC and RE. HIJ was the main contributor writing the manuscript, but all authors were involved in revising the manuscript, and all authors read and approved the final manuscript.

Ethics approval and consent to participate

In accordance with Danish law, the study did not need permission from the Regional Ethics Committees, but permission to access patient files was obtained from the Danish National Health Authorities (3-3013-353/1/) for all participating centers, and the study was registered with the Danish Data Protection Agency. In The Netherlands, the Leeuwarden IRB (R-TPO) nr. nWMO 21a approved the study and provided a waiver for patient consent according to Dutch law, valid for all participating centers.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Department of Anaesthesiology and Intensive Care, Vejle and Middelfart Hospitals
(2)
Institute of Regional Health Research, University of Southern Denmark
(3)
Center of Intensive Care, Medisch Centrum Leeuwarden
(4)
Pulmonary, Critical Care and Sleep Medicine, University of Washington
(5)
Cambia Palliative Care Center of Excellence, University of Washington
(6)
Department of Intensive Care Medicine Gelre Hospitals Apeldoorn
(7)
Academic Medical Center, University of Amsterdam
(8)
University Medical Center Groningen, University of Groningen

References

  1. Haywood K, Brett J, Salek S, et al. Patient and public engagement in health-related quality of life and patient-reported outcomes research: what is important and why should we care? Findings from the first ISOQOL patient engagement symposium. Qual Life Res. 2015;24:1069–76.View ArticlePubMedGoogle Scholar
  2. Wiering B, de Boer D, Delnoij D. Patient involvement in the development of patient-reported outcome measures: a scoping review. Health Expect. 2017;20:11–23.View ArticlePubMedGoogle Scholar
  3. Kurtzman ET, Greene J. Effective presentation of health care performance information for consumer decision making: a systematic review. Patient Educ Couns. 2016;99:36–43.View ArticlePubMedGoogle Scholar
  4. Boothroyd LJ, Lambert LJ, Ducharme A, et al. Challenge of informing patient decision making: what can we tell patients considering long-term mechanical circulatory support about outcomes, daily life, and end-of-life issues? Circ Cardiovasc Qual Outcomes. 2014;7:179–87.View ArticlePubMedGoogle Scholar
  5. Curtis JR, Vincent JL. Ethics and end-of-life care for adults in the intensive care unit. Lancet. 2010;376:1347–53.View ArticlePubMedGoogle Scholar
  6. Adhikari NK, Fowler RA, Bhagwanjee S, Rubenfeld GD. Critical care and the global burden of critical illness in adults. Lancet. 2010;376:1339–46.View ArticlePubMedGoogle Scholar
  7. Azoulay E, Metnitz B, Sprung CL, et al. End-of-life practices in 282 intensive care units: data from the SAPS 3 database. Intensive Care Med. 2009;35:623–30.View ArticlePubMedGoogle Scholar
  8. Jensen HI, Ammentorp J, Ording H. Withholding or withdrawing therapy in Danish regional ICUs: frequency, patient characteristics and decision process. Acta Anaesthesiol Scand. 2011;55:344–51.View ArticlePubMedGoogle Scholar
  9. Thompson BT, Cox PN, Antonelli M, et al. Challenges in end-of-life care in the ICU: statement of the 5th International Consensus Conference in Critical Care: Brussels, Belgium, April 2003: executive summary. Crit Care Med. 2003;32:1781–4.View ArticleGoogle Scholar
  10. Aitken LM, Castillo MI, Ullman A, et al. What is the relationship between elements of ICU treatment and memories after discharge in adult ICU survivors? Aust Crit Care. 2016;29:5–14.View ArticlePubMedGoogle Scholar
  11. Stricker KH, Kimberger O, Brunner L, Rothen HU. Patient satisfaction with care in the intensive care unit: can we rely on proxies? Acta Anaesthesiol Scand. 2011;55:149–56.View ArticlePubMedGoogle Scholar
  12. Bowling A. Research Methods in Health. Investing Health and Health Services. 3rd ed. Buckingham: Open University Press; 2009.Google Scholar
  13. van den Broek JM, Brunsveld-Reinders AH, Zedlitz AM, et al. Questionnaires on family satisfaction in the adult ICU: a systematic review including psychometric properties. Crit Care Med. 2015;43:1731–44.View ArticlePubMedGoogle Scholar
  14. Wall RJ, Engelberg RA, Downey L, et al. Refinement, scoring, and validation of the Family Satisfaction in the Intensive Care Unit (FS-ICU) survey. Crit Care Med. 2007;35:271–9.View ArticlePubMedGoogle Scholar
  15. Curtis JR, Patrick DL, Engelberg RA, et al. A measure of the quality of dying and death. Initial validation using after-death interviews with family members. J Pain Symptom Manage. 2002;24:17–31.View ArticlePubMedGoogle Scholar
  16. Downey L, Curtis JR, Lafferty WE, et al. The Quality of Dying and Death Questionnaire (QODD): empirical domains and theoretical perspectives. J Pain Symptom Manage. 2010;39:9–22.View ArticlePubMedGoogle Scholar
  17. Moselli NM, Debernardi F, Piovano F. Forgoing life sustaining treatments: differences and similarities between North America and Europe. Acta Anaesthesiol Scand. 2006;50:1177–86.View ArticlePubMedGoogle Scholar
  18. Gerritsen RT, Hofhuis JG, Koopmans M, et al. Perception by family members and ICU staff of the quality of dying and death in the ICU: a prospective multicenter study in The Netherlands. Chest. 2013;143:357–63.View ArticlePubMedGoogle Scholar
  19. Jensen HI, Gerritsen RT, Koopmans M, et al. Families’ experiences of intensive care unit quality of care: Development and validation of a European questionnaire (euroQ2). J Crit Care. 2015;30:884–90.View ArticlePubMedGoogle Scholar
  20. Stata. StataCorp LP, Texas, USA. 2016. http://www.stata.com/. Accessed 4 May 2017.
  21. Mplus. Muthén & Muthén, Los Angeles, USA. 2016. https://www.statmodel.com/index.shtml. Accessed 4 May 2017.
  22. Brown TA. Confirmatory factor analysis for applied research. 2nd ed. NY: The Guilford Press; 2015.Google Scholar
  23. Edwards JR, Bagozzi RP. On the nature and direction of relationships between constructs and measures. Psychol Methods. 2000;5:155–74.View ArticlePubMedGoogle Scholar
  24. Bollen KA, Bauldry S. Three Cs in measurement models: causal indicators, composite indicators, and covariates. Psychol Methods. 2011;16:265–84.View ArticlePubMedPubMed CentralGoogle Scholar
  25. Jarvis CB, Mackenzie SB, Podsakoff PM. A critical review of construct indicators and measurement model misspecification in marketing and consumer research. J Consum Res. 2013;30:199–218.View ArticleGoogle Scholar
  26. Schwarzkopf D, Behrend S, Skupin H, et al. Family satisfaction in the intensive care unit: a quantitative and qualitative analysis. Intensive Care Med. 2013;39:1071–9.View ArticlePubMedGoogle Scholar
  27. Wright SE, Walmsley E, Harvey SE, et al. Family-Reported Experiences Evaluation (FREE) study: a mixed-methods study to evaluate families’ satisfaction with adult critical care services in the NHS. NIHR Journals Library: Southampton; 2015.Google Scholar
  28. Sarode V, Sage D, Phong J, Reeves J. Intensive care patient and family satisfaction. Int J Health Care Qual Assur. 2015;28:75–81.View ArticlePubMedGoogle Scholar
  29. Linnarsson JR, Bubini J, Perseius KI. A meta-synthesis of qualitative research into needs and experiences of significant others to critically ill or injured patients. J Clin Nurs. 2010;19:3102–11.View ArticlePubMedGoogle Scholar
  30. Lind R, Lorem GF, Nortvedt P, Hevroy O. Family members’ experiences of “wait and see” as a communication strategy in end-of-life decisions. Intensive Care Med. 2011;37:1143–50.View ArticlePubMedPubMed CentralGoogle Scholar
  31. Stricker KH, Kimberger O, Schmidlin K, et al. Family satisfaction in the intensive care unit: what makes the difference? Intensive Care Med. 2009;35:2051–9.View ArticlePubMedGoogle Scholar

Copyright

© The Author(s). 2017

Advertisement