- Open Access
Long-term health-related quality of life and burden of disease after intensive care: development of a patient-reported outcome measure
Critical Care volume 25, Article number: 82 (2021)
ICU survivorship includes a diverse burden of disease. Current questionnaires used for collecting information about health-related problems and their relation to quality of life lack detailed questions in several areas relevant to ICU survivors. Our aim was to construct a provisional questionnaire on health-related issues based on interviews with ICU survivors and to test if this questionnaire was able to show differences between ICU survivors and a control group.
Thirty-two ICU survivors were identified at a post-ICU clinic and interviewed at least six months after ICU discharge. Using an established qualitative methodology from oncology, all dysfunctions and disabilities were extracted, rephrased as questions and compiled into a provisional questionnaire. In a second part, this questionnaire was tested on ICU survivors and controls. Inclusion criteria for the ICU survivors were ICU stay at least 72 h with ICU discharge six months to three years prior to the study. A non-ICU-treated control group was obtained from the Swedish Population Register, matched for age and sex. Eligible participants received an invitation letter and were contacted by phone. If willing to participate, they were sent the questionnaire. Descriptive statistics were applied.
Analysis of the interviews yielded 238 questions in 13 domains: cognition, fatigue, physical health, pain, psychological health, activities of daily living, sleep, appetite and alcohol, sexual health, sensory functions, gastrointestinal functions, urinary functions and work life. In the second part, 395 of 518 ICU survivors and 197 of 231 controls returned a completed questionnaire, the response rates being 76.2% and 85.3%, respectively. The two groups differed significantly in 13 of 22 comorbidities. ICU survivors differed in a majority of questions (p ≤ 0.05) distributed over all 13 domains compared with controls.
This study describes the development of a provisional questionnaire to identify health-related quality of life issues and long-term burden of disease after intensive care. The questionnaire was answered by 395 ICU survivors. The questionnaire could identify that they experience severe difficulties in a wide range of domains compared with a control group.
Trial registry ClinicalTrials.gov Ref# NCT 02767180
ICU survivorship may come at a price—the price of cognitive  and physical dysfunction , psychiatric and psychological problems , financial and work-related shortcomings  and healthcare consumption .
To identify and describe these problems, the intensive care community uses a synthesis of tests, examinations and questionnaires depending on the context: SF-36 and EQ-5D are the most commonly used questionnaires for measuring health-related quality of life [6, 7], but concerns regarding their ability to identify issues valued by ICU survivors have been raised . Within the domains of physical, cognitive and mental health, the concept of PICS (post-intensive care syndrome) points out directions for investigations rather than provides scales. All three of these domains have numerous specific measurements, with for example at least 26 different tools to measure functional outcome . Furthermore, discriminating post-ICU issues from prevalent psychological and physical ill-being in the general population is challenging when problems overlap [10, 11].
We hypothesized that a questionnaire mainly based on interviews with ICU survivors would contain a majority of issues experienced after intensive care, as well as carry a discriminative capacity to identify those issues with a magnitude distinct from a non-ICU-treated population. Influenced by advances made in oncology , we applied an established qualitative methodology [13, 14], using interviews to identify and extract issues from survivors. In an attempt to encase the full extent of the problems, interviews were not limited to particular areas.
Our aim was to develop a provisional questionnaire based on the content of such interviews, test its practicality in a scientific setting, as well as its ability to identify differences in the magnitude of issues between ICU survivors and non-ICU-treated controls. This would be a first step toward a questionnaire for long-term follow-up after intensive care. It could be useful to healthcare providers in post-ICU clinics as well as in primary care to identify clinical problems in need of specific treatments, consulting or referral to rehabilitation. In a research setting, it could be practical as an outcome measurement when evaluating issues and their trajectories.
Comparing the results from an ICU survivor group with those from a non-ICU-treated group gives two advantages. First, a comparison between an ICU survivor group and a non-ICU-treated group will aid a future reduction in the number of items. Second, by being able to measure the degree to which issues are related to intensive care and not to problems common in a non-ICU-treated population, the questionnaire could be used to identify domains suitable for interventional trials.
Interviews and development of a provisional questionnaire
Methodological framework and considerations
We took a pragmatic approach inspired by our earlier experience of developing instruments in oncology based on interviews [13, 14]. Thus, the method applied in this study follows recommendations of EORTC (European Organization of Research and Treatment of Cancer) and the Division of Clinical Cancer Epidemiology, Gothenburg, Sweden [15, 16].
While the methodology has similarities with Grounded Theory such as using data saturation as an endpoint and the parallel process of data collection and analyses, there are important differences. For example, we use an interviewer with clinical experience and domain knowledge. As recommended by the EORTC findings from the literature, other scales and questionnaires may be shown to the interviewee to evoke further thoughts in the second part of the interview. In summary, our methodology aims at creating an as comprehensive list of symptoms and issues as possible, where keeping the exact wording of the interviewees is important to minimize interpretations.
Setting and study population
Our post-ICU (16-bed mixed ICU in a university hospital) plans a scheduled visit six months after ICU discharge for survivors with an ICU length of stay of at least 72 h. Survivors may also contact the clinic for a visit or be invited at the discretion of the post-ICU clinic nurses. All survivors visiting the post-ICU clinic between February and May in 2015, with at least six months from ICU discharge, were eligible for the study.
Using a purposive, maximum variation sampling approach, potentially “information-rich” interviewees representative for different ages, gender, admission diagnoses, ICU length of stay, time from ICU discharge and postal areas as a marker of socioeconomic status were invited to participate . They were selected by one of the researchers (J.M.), who had not been involved in their care but met them at their visit to the post-ICU clinic to gain trust and explain the study. Sample size was based on data saturation—the point where no new information emerged . To confirm saturation, we decided a priori to continue the data collection for an additional three interviews.
Interviews were conducted either in the post-ICU clinic or in their home, based on the interviewee’s own choice. There was no time limitation for the interviews, and participants were interviewed only once. Using a semi-structured technique, we explored their current situation as well as symptoms, difficulties, quality-of-life issues and social effects arising at any point after ICU discharge. All interviews started with the question “We are asking for your help in creating a questionnaire which will be used to identify and follow the experiences of patients who have survived intensive care. I would like to ask you a few things about your health. Can you tell me about the experiences you may have had as a result of your intensive care stay, and the time between discharge and today?”. While initial questions were open-ended, as interviews progressed details about findings were sought for. Once the interviewee could think of nothing further, domains and issues from previous interviews, literature or other scales and questionnaires were discussed (Additional file 1: Table S1). Examples of interview questions, probes and prompts can be seen in Additional file 4: Figure S1. Field notes were taken by the interviewer and read to the interviewee at the end of the interview to allow for comments or corrections.
In parallel with conducting the interviews, already transcribed interviews were independently analyzed by two of the researchers (J.M. and A-C.W.). Analyses were made manually: Long quotes were shortened while preserving the core meaning of the issues . All extracted issues were categorized into domains, and duplicates were removed. To ensure that important issues were retained as well as to minimize the risk of recall bias, issues only had to be mentioned once to be included in the provisional questionnaire. No items were excluded in this phase since an item reduction will be performed at a later stage. The remaining issues were rephrased as questions, where care was taken to maintain the wording used by the interviewee.
At the time of data analysis, the first researcher (J.M.), a male intensive care physician, had two years of experience in qualitative research and several years of experience with post-ICU care. The second researcher (A-C.W.), a female gynaecological oncologist, had ten years of experience in qualitative research in the Division of Clinical Cancer Epidemiology, Gothenburg University, Sweden and 14 years of experience in the EORTC Quality of Life Group.
Composite questions about domain-specific quality of life and domain-specific future concerns were added at the end of each domain (How much do you think problems within [domain] affects your quality of life? For the past month, have you been worried about your future regarding [domain]?). Empty space and a request for missing issues or other comments were provided after each domain.
Questions regarding demographics and comorbidities were added at the end of the questionnaire.
The response scales used are based on the established experience of the Division of Clinical Cancer Epidemiology, Gothenburg, Sweden, and were created to match each conceptual entity as closely as possible using incidence, prevalence, intensity and agreement when applicable (Table 1) .
Because our interest lies in long-term effects, the time frame asked about in most response scales was "the last month." This would also minimize the problem of recall bias. Care was taken not to overlap between alternatives and to include "Not applicable" if needed.
Content validity and cognitive interviews
Evidence of content validity as a measurement property refers to the extent of which an instrument contains the relevant aspects of the construct it intends to measure, in our case “issues experienced after intensive care” . However, the content is not only the issues raised within the questions, but also such aspects as the wording of questions, the clarity of instructions and proper response scales . In accordance with the ISPOR (International Society for Pharmacoeconomics and Outcomes Research) guidelines on testing for evidence of content validity, all questions were tested with cognitive interviews on additional ICU survivors chosen with the same criteria as the initial interviewees, and with the same saturation-based sample size . These interviews were recorded as well. The cognitive interviews were the final opportunity to make content changes before administering the questionnaire to a larger group and included appropriate response scales and recall period. The aim was to ensure that the questions were conceptually clear, easily understood, perceived as relevant and to make sure no important issues are missing. Interviewees were initially instructed to complete the questionnaire while thinking aloud, but as the two first interviewees failed to follow these instructions, we changed to a retrospective probing technique, where questions were asked after finishing each domain, in line with EORTC’s guidelines when a questionnaire has a substantial number of questions .
Application of the questionnaire
Eligible patients were all adult ICU survivors admitted between February 2013 and December 2015 to one of three mixed ICUs in Sahlgrenska University Hospital, Gothenburg, Sweden (in total 31 ICU beds), and with a minimum ICU length of stay of 72 h. They all had been discharged from the ICU between six months and three years prior to the study. Exclusion criteria were primary neurological/neurosurgical reason for admission, limited understanding of Swedish as judged by study personnel, no Swedish personal identity number, no Swedish address or phone number or a secret Swedish personal identity. We obtained a non-ICU-treated control group from the Swedish Population Register, matched for age and sex with respect to ICU survivors having returned a completed questionnaire. For the version of the questionnaire addressing the control group, we removed all questions requiring a previous ICU stay (e.g., Have you had difficulties describing your ICU experiences?) and added one question checking for previous intensive care. Exclusion criteria for the control group were previous ICU stay or a limited understanding of Swedish.
All eligible participants received an initial letter with information about the study, and within a week they received a phone call asking for participation. The questionnaire was sent together with a pre-paid return envelope, and reminder phone calls were made if the questionnaire was not returned within two weeks. The questionnaire was sent to the ICU survivors between April 2016 and October 2017, and to the control group between March 2017 and December 2017.
Univariate descriptive statistics are presented as frequencies and percentages for all categorical variables. Continuous variables were screened for normality using Shapiro-Wilks (p > 0.05) and box-plots. For non-normally distributed continuous variables, median and range or median and interquartile range (IQR) are reported. Bivariable comparisons were made between ICU survivors and the control group for all ordered categorical variables and continuous variables in order to identify differences between the groups by applying the Mann–Whitney U test. These results are presented as means and mean rank sums and the associated p-value calculated in the Mann–Whitney U test. In addition, all the bivariable comparisons for ordered categorical variables were analyzed with Fisher's exact test as a robustness check. Dichotomous variables were also assessed with Fisher's exact test. All tests were two-tailed, and significance level was set to 0.05.
Questionnaires were scanned with Remark Office OMR (Remark Office OMR 10, Gravic Inc, Malvern, USA). Statistical analyses were performed using the IBM SPSS v26 package (IBM SPSS v26 Statistics, IBM, Armonk, USA).
Interviews and development of a provisional questionnaire
The median age of the interviewees was 55.5 years (range 20–82), and 33% were females. The interviews took place at a median of 14.7 months (range 7.6–68.0) after ICU discharge. The median ICU length of stay was 4.9 days (range 1.7–76.1), and the median SAPS 3 score was 57.5 (range 24–81). Seventy per cent were treated with mechanical ventilation for a median time of five days (range 1–62). The most common primary diagnosis was infection/sepsis (18.8%), followed by trauma and cardiac arrest as second and third most common (both 12.5%) diagnosis (Table 2).
All invited patients accepted to be interviewed. In total, 32 interviews including six cognitive interviews were performed. Ten of the interviews were conducted in the presence of a partner. Apart from one interview conducted in the interviewee’s home and one conducted in a public location, all interviews were conducted in the post-ICU clinic. The average time of interviews was 49 min (range 15–113). Minor language corrections were made based on the cognitive interviews, but no new issues were identified. No question was considered upsetting, no response scale was changed, and no time frame was adjusted.
Quotes from the interviews generated 437 issues. By removing duplicates and similarities, they were reduced to 195 unique issues (Additional file 5: Figure S2). These were rephrased as questions and categorized into 13 domains: cognition, fatigue, physical health, pain, psychological health, activities of daily living, sleep, appetite and alcohol, sexual health, sensory functions, gastrointestinal functions, urinary functions and work life.
For the questionnaire, 31 composite questions regarding domain-specific quality of life and domain-specific future worries were added. Twelve questions from other scales and questionnaires were considered relevant by the interviewees and were added: All three questions from AUDIT-C , four questions from the KATZ-ADL index , four questions from the Work Ability Index  and one question about the ability to walk for six minutes . The distribution of questions is shown in Table 3. In the version of the questionnaire for the control group, twenty questions requiring a previous ICU-stay were removed.
A majority of questions was measured on an ordered category scale: 113 questions on a 6-point scale, 91 questions on a 5-point scale, eight questions on a 4-point scale, two questions on a 3-point scale. Twenty-two questions were measured on a dichotomous scale and two questions were quantitative. Higher scores indicated higher levels of difficulties or problems except in eleven reversely coded questions where higher scores indicated lower levels of problems (e.g., Do you have the ability to look forward to things?; No—Rarely—Sometimes—Quite often—Very often—All the time).
Application of the questionnaires
A total of 518 ICU survivors and 289 controls received a questionnaire. Among these, 395 ICU survivors and 195 controls returned a completed questionnaire, the return rates being 76.2% and 85.3%, respectively (Fig. 1). The most commonly stated reason for declining participation among ICU survivors was family members declining (1.8%; 10 of 567) and among controls the reason was "No time" (2.2%; 6 of 276).
The most frequent ICU admission diagnosis for survivors was infection/sepsis (27.8%; n = 110) followed by trauma (13.4%; n = 53) and respiratory failure (10.9%; n = 43). Median SAPS 3 score was 59 (range 16–100), and median ICU length of stay was 5.6 days (range 3.0–78.6). Most ICU survivors were mechanically ventilated (78.5%), with a median time of 4.0 days (range 0–74). The representation of the major diagnosis groups was fairly similar between the ICU survivors and the interviewees (Table 3).
Demographics and comorbidities
There were no differences in age and gender between the ICU survivors and the control group (Table 4). While there were no differences in educational levels between the two groups, significantly more ICU survivors were on sick leave/sickness benefit compared to the control group (p = 0.000). The ICU survivors were also sicker compared with controls, differing significantly in 13 of 22 comorbidities (Table 5). Cardiovascular disease (hypertension, angina pectoris, myocardial infarction and heart failure) was more common among the ICU-survivors as was respiratory disease and pulmonary embolus. The ICU survivors suffered more often from depression and anxiety. Diabetes, kidney disease and bowel disease were also more common in this group. The need for walking aids or wheelchair due to physical impairment occurred only among ICU survivors as did having amputated limb(s).
Symptoms and burden of disease
At the time of completing the questionnaire, which was between six months and three years after discharge, the ICU survivors differed significantly in a majority of questions across all domains when compared with the control group (Additional file 2: Table S2a and Table 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18). Examples are:
Cognitive difficulties affecting quality of life such as losing the thread easily, think you have done something you haven’t, mistaken which day of the week it is, or difficulties taking initiatives.
Fatigue was a common symptom often with the need for a daytime rest. It could be tough getting started doing things, difficult finishing things due to feeling exhausted or doing things under pressure of time or multitasking. Getting tired from reading, or from conversation between more than two people could affect work and limit social activities.
Physical health in general was more often affected among ICU survivors. They could suffer from reduced body feeling, muscle weakness in arms or legs, dizziness when standing up, losing balance easily, difficulties climbing stairs, unsteady gait, contractures and shortness of breath, many of them limiting physical activities.
Pain was reported from different parts of the body or as general body pain by survivors making painkillers necessary for managing ADL or to get sufficient sleep.
Psychological health problems were also overrepresented among the survivors. It made them cry more easily. Feeling low-spirited or depressed, or suffering from panic attacks, were also more common as was feelings of hopelessness and feelings of life being meaningless. Many suffered from low self-confidence.
Activities of daily living (ADL) were often more difficult for the survivors. They could need help with things like getting dressed, moving from bed to chair, visiting the toilet, shopping, cooking and doing housework. Help with medication and managing bills was also more common.
Sleep could be affected in many ways, at worst as nightmares.
Poor appetite, bothersome thirst and difficulties chewing more often affected quality of life for the ICU-survivors. Together with reduced taste, mouth dryness, mouth soreness or mouth pain, swallowing were difficult and made it easier to choke.
Sexual health issues like libido and sex life were less satisfactory in ICU-survivors.
Work life differed between the two groups. For the survivors the capacity for work was negatively affected both due to physical demands and psychological demands. Work problems as well as financial problems were more common among ICU-survivors.
A complete list of all questions and their response rates is shown in Additional file 2: table S2a/Additional file 3: S2b. All continuous variables were found to deviate from normality in both groups. No additional valuable information was added to the space after each domain.
Our study describes a first step toward an intensive care long-term follow-up questionnaire with the capacity to detect the burden of ICU survivorship and the effect on quality of life. By creating a questionnaire from interviews with ICU survivors and testing it on ICU survivors and a non-ICU-treated control group, we were able to show that the questionnaire contained most issues experienced by survivors and was able to identify differences between the two groups. While issues found in our interviews may apply to a general population to a certain degree, it is our belief that many might worsen after intensive care. A comparison with a non-ICU-treated control group may help describe to what extent intensive care can be attributed to a change in magnitude rather than simply describe a prevalence.
At a stakeholders' conference in 2010, the concept of PICS was created to enclose impairments in mental health, cognition and physical functions . At a follow-up conference in 2012, the PICS group pointed out the need for outcome assessment tools created with qualitative methods . Several groups have addressed this issue, either by developing new instruments or by examining the evidence of content validity of existing ones. Jeong and Kang reported the development and validation of a questionnaire specifically for the three domains of PICS, using a methodology similar to ours . In 2018, Nedergaard et al. interviewed 18 ICU survivors and extracted the most important issues . Although symptoms from the PICS domains were well represented, additional symptoms were also considered important, for example incontinence, short temper and the feeling of being isolated. Furthermore, large differences between patients and clinicians when ranking the importance of symptoms have been found in areas as diverse as bariatric surgery , diabetes  and aphasia . These findings would argue toward instruments developed with input from former patients.
Regarding measuring HRQoL (Health-Related Quality of Life), SF-36 and EQ-5D are currently the most commonly used tools after intensive care. Lim et al. extracted post-ICU issues from 30 ICU survivors and let the same patients compare these issues with SF-36 and EQ-5D . Of the domains identified as relevant by the ICU survivors, only one was considered adequately covered by SF-36 or EQ-5D. The remaining domains were either inadequately covered or completely missing, suggesting that the use of either of these instruments as a measurement of post-ICU HRQoL will miss important issues. In another study, Jensen et al. were unable to show improvement in HRQoL measured by SF-36 after their ICU recovery program and recommend new instruments to be developed and validated to assess the particular HRQoL problems of post-ICU patients .
Our questionnaire focuses on different measures of ICU survivorship: First of all, it covers the areas of PICS; physical, cognitive and mental health. Secondly, it describes quality-of-life related to the survivor's health status. Further, areas not covered by PICS or SF-36/EQ-5D such as dysphagia , joint contractions , sleep disturbances  and personal finances  are included, all previously described problems after intensive care.
The response rate of 76.2% from the ICU survivors and 85.3% from the control group indicates not only the usability of the questionnaire in a trial context, but that questions were considered relevant. Participants did not provide any additional issues in the comment areas in the questionnaire, only encouraging comments, arguing toward evidence of content validity in our questionnaire rather than "questionnaire fatigue."
The development of the questionnaire follows international recommendations for development of patient-reported outcome measures . Choosing interviewees purposively instead of in a consecutive order has been the most effective for reaching data saturation with minimum sample size in simulations  Data saturation is the most commonly used delimiter for sample size but not randomizing the order of the interviews poses a hypothetical risk of affecting the saturation point . Therefore, we decided à priori to set the sample size to when three consecutive interviews did not provide any new information.
We took several steps to show evidence of content validity: First, we based this provisional questionnaire mainly on issues reported by ICU survivors themselves. Second, all interviewees were read the field notes to ensure our proper understanding of issues. Third, we used cognitive interviews, and finally we allowed all participants to add potentially missing issues in the quantitative phase.
This being a provisional questionnaire awaiting further analyses, there are some limitations. Even if a high response rate indicates the relevance of the questions asked, this questionnaire will be reduced. Differences in comorbidities between ICU survivors and the control group, where 13 of 22 differed significantly, align with previous findings that chronic comorbidities are common in ICU patients . We do not know to what extent these comorbidities explain the differences between the two groups. The questionnaire has not been developed from, nor tested on, patients with a shorter time from ICU discharge than six months; hence, our questionnaire may miss issues that resolve completely within this time frame. Nor was the questionnaire developed for patients with an ICU length-of-stay shorter than 72 h or with neurological/neurosurgical primary diagnoses, and results cannot be generalized to these groups. We cannot exclude that interviewees may have forgotten issues experienced between ICU discharge and the interview, and thus our questionnaire cannot claim to be comprehensive. However, by including all issues appearing in interviews, no matter how uncommon, we have attempted to minimize the impact of potential recall bias. In the second phase, recall bias was accounted for by using ‘the last month’ as time frame in the provisional questionnaire. Although we have shown that a majority of issues differed significantly in magnitude in comparison with a non-ICU-treated control group, we do not know to what extent these differences were already prevalent before intensive care. Regarding internal validity, there was a difference in age between the interviewees and the cohort groups. However, ranges of age, SAPS score etc. did not differ markedly. Finally, we cannot exclude a selection bias with regard to patients who chose not to participate or who we were unable to reach.
This study describes the development of a provisional questionnaire for long-term health-related quality of life and burden of disease after intensive care. This first version, based mainly on issues from interviews with ICU survivors, clearly identified burden of disease affecting multiple domains in a large group of patients. The next steps in order to make this questionnaire a useful tool for follow-up after intensive care include further statistical analyses including psychometric properties and reduction in the number of questions.
Availability of data and materials
The datasets generated during and/or analyzed during the current study are not publicly available due to them containing information that could compromise research participants privacy/consent, but are available from the corresponding author on reasonable request.
Activities of daily living
EuroQoL's five dimensions instrument
Intensive care unit
Health-related quality of life
Post-intensive care syndrome
Post-traumatic stress disorder
- SAPS 3:
Simplified acute physiology score
Short-form (36) health survey
Pandharipande PP, Girard TD, Jackson JC, Morandi A, Thompson JL, Pun BT, et al. Long-term cognitive impairment after critical illness. New Engl J Med. 2013;369(14):1306–16.
Milton A, Schandl A, Soliman I, Joelsson-Alm E, van den Boogaard M, Wallin E, et al. ICU discharge screening for prediction of new-onset physical disability—A multinational cohort study. Acta Anaesthesiol Scand. 2020;64(6):789–97.
Sivanathan L, Wunsch H, Vigod S, Hill A, Pinto R, Scales DC. Mental illness after admission to an intensive care unit. Intensive Care Med. 2019;45(11):1550–8.
Kamdar BB, Huang M, Dinglas VD, Colantuoni E, Von Wachter TM, Hopkins RO, et al. Joblessness and lost earnings after acute respiratory distress syndrome in a 1-year national multicenter study. Am J Respir Crit Care Med. 2017;196(8):1012–20.
Garland A, Olafson K, Ramsey CD, Yogendran M, Fransoo R. A population-based observational study of intensive care unit-related outcomes with emphasis on post-hospital outcomes. Ann Am Thorac Soc. 2015;12(2):202–8.
Needham DM, Davidson J, Cohen H, Hopkins RO, Weinert C, Wunsch H, et al. Improving long-term outcomes after discharge from intensive care unit: Report from a stakeholders’ conference. Crit Care Med. 2012;40(2):502–9.
Angus DC, Carlet J. Surviving intensive care: a report from the 2002 Brussels Roundtable. Intensive Care Med. 2003;29(3):368–77.
Lim WC, Black N, Lamping D, Rowan K, Mays N. Conceptualizing and measuring health-related quality of life in critical care. J Crit Care. 2016;31(1):183–93.
Parry SM, Denehy L, Beach LJ, Berney S, Williamson HC, Granger CL. Functional outcomes in ICU - what should we be using? - an observational study. Crit Care 2015;19(1).
Prince M, Patel V, Saxena S, et al. No health without mental health. The Lancet. 2007;370:859–77.
Van Lente E, Barry MM, Molcho M, et al. Measuring population mental health and social well-being. Int J Public Health. 2012;57:421–30.
Iwashyna TJ. Survivorship will be the defining challenge of critical care in the 21st century. Ann Intern Med. 2010;153(3):204–5.
Steineck G, Bergmark K, Henningsohn L, Al-Abany M, Dickman PW, Helgason A. Symptom documentation in cancer survivors as a basis for therapy modifications. Acta Oncol. 2002;41(3):244–52.
Lind H, Waldenström A-, Dunberger G, Al-Abany M, Alevronta E, Johansson K-, et al. Late symptoms in long-term gynaecological cancer survivors after radiation therapy: A population-based cohort study. Br J Cancer 2011;105(6):737–745.
Johnson C, Aaronson N, Blazeby JM, et al. EORTC quality of life group guidelines for developing questionnaire modules. 4th ed. Brussels: EORTC Quality of Life Group; 2011.
Omerov P, Steineck G, Runeson B, Christensson A, Kreicbergs U, Pettersén R, et al. Preparatory studies to a population-based survey of suicide-bereaved parents in Sweden. Crisis. 2013;34(3):200–10.
Patton MQ. Qualitative Research & Evaluation Methods. 4th ed. Los Angeles: SAGE; 2015.
Rothman M, Burke L, Erickson P, Leidy NK, Patrick DL, Petrie CD. Use of existing patient-reported outcome (PRO) instruments and their modification: The ISPOR good research practices for evaluating and documenting content validity for the use of existing instruments and their modification PRO task force report. Value Health. 2009;12(8):1075–83.
American Educational Research Association. Standards for Educational and Psychological Testing. 2014:230.
Patrick DL, Burke LB, Gwaltney CJ, Leidy NK, Martin ML, Molsen E, et al. Content validity - Establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: Part 2 - Assessing respondent understanding. Value Health. 2011;14(8):978–88.
Bush K, Kivlahan DR, McDonell MB, Fihn SD, Bradley KA. The AUDIT alcohol consumption questions (AUDIT-C): An effective brief screening test for problem drinking. Arch Intern Med. 1998;158(16):1789–95.
Katz S, Ford AB, Moskowitz RW, Jackson BA, Jaffe MW. Studies of Illness in the Aged: The Index of ADL: A Standardized Measure of Biological and Psychosocial Function. JAMA. 1963;185(12):914–9.
Tuomi K, Ilmarinen J, Jahkola A, Katajarinne L, Tulkki A. Work Ability Index. Helsinki: Finnish Institute of Occupational Health; 1998.
Crapo RO, Casaburi R, Coates AL, Enright PL, MacIntyre NR, McKay RT, et al. ATS statement: guidelines for the six-minute walk test. Am J Respir Crit Care Med. 2002;166(1):111–7.
Elliott D, Davidson JE, Harvey MA, Bemis-Dougherty A, Hopkins RO, Iwashyna TJ, et al. Exploring the scope of post-intensive care syndrome therapy and care: engagement of non-critical care providers and survivors in a second stakeholders meeting. Crit Care Med. 2014;42(12):2518–26.
Jeong YJ, Kang J. Development and validation of a questionnaire to measure post-intensive care syndrome. Intensive Crit Care Nurs 2019;55.
Nedergaard HK, Haberlandt T, Reichmann PD, Toft P, Jensen HI. Patients’ opinions on outcomes following critical illness. Acta Anaesthesiol Scand. 2018;62(4):531–9.
Coulman KD, Howes N, Hopkins J, Whale K, Chalmers K, Brookes S, et al. A Comparison of Health Professionals’ and Patients’ Views of the Importance of Outcomes of Bariatric Surgery. Obes Surg. 2016;26(11):2738–46.
Gorst SL, Young B, Williamson PR, Wilding JPH, Harman NL. Incorporating patients' perspectives into the initial stages of core outcome set development: A rapid review of qualitative studies of type 2 diabetes. BMJ Open Diabetes Res Care 2019;7(1).
Wallace SJ, Worrall L, Rose T, Le Dorze G, Cruice M, Isaksen J, et al. Which outcomes are most important to people with aphasia and their families? an international nominal group technique study framed within the ICF. Disabil Rehabil. 2017;39(14):1364–79.
Jensen JF, Egerod I, Bestle MH, Christensen DF, Elklit A, Hansen RL, et al. A recovery program to improve quality of life, sense of coherence and psychological health in ICU survivors: a multicenter randomized controlled trial, the RAPIT study. Intensive Care Med. 2016;42(11):1733–43.
Macht M, Wimbish T, Clark BJ, Benson AB, Burnham EL, Williams A, et al. Postextubation dysphagia is persistent and associated with poor outcomes in survivors of critical illness. Crit Care 2011;15(5).
Clavet H, Doucette S, Trudel G. Joint contractures in the intensive care unit: Quality of life and function 3.3 years after hospital discharge. Disabil Rehabil 2015;37(3):207–213.
Alexopoulou C, Bolaki M, Akoumianaki E, Erimaki S, Kondili E, Mitsias P, et al. Sleep quality in survivors of critical illness. Sleep Breathing. 2019;23(2):463–71.
Khandelwal N, Hough CL, Downey L, Engelberg RA, Carson SS, White DB, et al. Prevalence, risk factors, and outcomes of financial stress in survivors of critical illness. Crit Care Med. 2018;46(6):e530–9.
Basch E, Spertus J, Adams Dudley R, Wu A, Chuahan C, Cohen P, et al. Methods for developing patient-reported outcome-based performance measures (PRO-PMs). Value Health. 2015;18(4):493–504.
van Rijnsoever FJ, Derrick GE. (I Can’t Get No) Saturation: a simulation and guidelines for sample sizes in qualitative research. PLOS One. 2017;12(7):e0181689.
Vasileiou K, Barnett J, Thorpe S, Young T. Characterising and justifying sample size sufficiency in interview-based studies: Systematic analysis of qualitative health research over a 15-year period. BMC Med Res Methodol 2018;18(1).
Esper AM, Martin GS. The impact of cormorbid conditions on critical illness. Crit Care Med. 2011;39(12):2728–35.
Prof. Anders Wallin and PhD Marie Eckerström at the Institute of Neuroscience and Physiology, University of Gothenburg, for valuable advice.
Open access funding provided by University of Gothenburg. This study was financed by departmental resources and grants from the Gothenburg Medical Society, the Swedish Heart–Lung Foundation, and the Swedish State Support for Clinical Research. None of the funding agencies had any input in designing the study, nor in the collection, analysis or interpretation of data or in writing the manuscript.
Ethics approval and consent to participate
The two parts of the study were approved by the Regional Research Ethics Committee of Gothenburg, Sweden, on February 26, 2015 (Ref 017-15) and February 23, 2016 (Ref 011-16), respectively. The second part is registered in Clinical Trials Gov (Ref. NCT 02767180). Interviewed participants were given oral and written information concerning their participation in the study. Consent in the applicatory part was assumed by the return of a completed questionnaire. Data were handled and stored according to the European General Data Protection Regulation (GDPR).
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Table S1: List of scales and questionnaires discussed during interviews with ICU survivors.
Table S2a: Descriptive statistics and response rates for ICU survivors and the control group.
Table S2b: Descriptive statistics and response rates for ICU survivor-specific questions.
Figure S1: Examples of interview questions and verbal prompts and probes used in both original interviews and during cognitive interviews.
Figure S2: Examples on construction of questions.
About this article
Cite this article
Malmgren, J., Waldenström, AC., Rylander, C. et al. Long-term health-related quality of life and burden of disease after intensive care: development of a patient-reported outcome measure. Crit Care 25, 82 (2021). https://doi.org/10.1186/s13054-021-03496-7
- Critical care
- Intensive care unit
- Critical illness
- Quality of life
- Follow-up studies
- Long-term adverse effects
- Patient-reported outcome