- Open Access
Impact of weekly case-based tele-education on quality of care in a limited resource medical intensive care unit
Critical Care volume 23, Article number: 220 (2019)
Limited critical care subspecialty training and experience is available in many low- and middle-income countries, creating barriers to the delivery of evidence-based critical care. We hypothesized that a structured tele-education critical care program using case-based learning and ICU management principles is an efficient method for knowledge translation and quality improvement in this setting.
Methods and interventions
Weekly 45-min case-based tele-education rounds were conducted in the recently established medical intensive care unit (MICU) in Banja Luka, Bosnia and Herzegovina. The Checklist for Early Recognition and Treatment of Acute Illness (CERTAIN) was used as a platform for structured evaluation of critically ill cases. Two practicing US intensivists fluent in the local language served as preceptors using a secure two-way video communication platform. Intensive care unit structure, processes, and outcomes were evaluated before and after the introduction of the tele-education intervention.
Patient demographics and acuity were similar before (2015) and 2 years after (2016 and 2017) the intervention. Sixteen providers (10 physicians, 4 nurses, and 2 physical therapists) evaluated changes in the ICU structure and processes after the intervention. Structural changes prompted by the intervention included standardized admission and rounding practices, incorporation of a pharmacist and physical therapist into the interprofessional ICU team, development of ICU antibiogram and hand hygiene programs, and ready access to point of care ultrasound. Process changes included daily sedation interruption, protocolized mechanical ventilation management and liberation, documentation of daily fluid balance with restrictive fluid and transfusion strategies, daily device assessment, and increased family presence and participation in care decisions. Less effective (dopamine, thiopental, aminophylline) or expensive (low molecular weight heparin, proton pump inhibitor) medications were replaced with more effective (norepinephrine, propofol) or cheaper (unfractionated heparin, H2 blocker) alternatives. The intervention was associated with reduction in ICU (43% vs 27%) and hospital (51% vs 44%) mortality, length of stay (8.3 vs 3.6 days), cost savings ($400,000 over 2 years), and a high level of staff satisfaction and engagement with the tele-education program.
Weekly, structured case-based tele-education offers an attractive option for knowledge translation and quality improvement in the emerging ICUs in low- and middle-income countries.
The World Health Organization defines telemedicine as “a health service in conditions where the distance is a critical factor, and it involves the use of information and communication technologies for the exchange of useful information about diagnosis, treatment and prevention of diseases and injuries, for research and continuous medical education of health workers, all in order to improve the health of individuals and their communities” . Telemedicine in critical care has largely gone down the advanced E-ICU pathway to provide 24/7 intensivist support in high-income countries . However, there are alternate paths within the scope of telemedicine suitable for wide array of health care settings, including “e-learning” or tele-education [3, 4].
Preliminary studies from Mayo Clinic investigators recently demonstrated that video-enabled remote simulation training based on a structured platform (CERTAIN: Checklist for Early Recognition and Treatment of Acute Illness)  can be a successful and efficient case-based learning method to disseminate clinical skills to critical care practitioners in diverse international settings [6, 7].
In low- and middle-income countries, tele-education may offer an attractive option to accelerate knowledge translation and address infrastructure barriers and limited opportunities for intensive care subspecialty training. Video-enabled, case-based discussions were chosen as a cost-efficient method to provide face-to-face education and reinforcement of the CERTAIN platform, which provides a standardized approach and checklist of common, evidence-based best critical care practices.
Our aim was to provide a longitudinal tele-education program and evaluate the impact of spaced, case-based learning and reinforcement on clinical practice and patient outcomes in a recently established intensive care unit in Bosnia and Herzegovina.
Using a before-after cohort study design, the investigators evaluated the effect of a remote, video-enabled longitudinal case-based tele-education intervention on clinician satisfaction, ICU processes and procedures, clinical outcomes, and cost in the newly established medical intensive care unit (MICU) at the University Clinical Center of Republika Srpska. The MICU serves as a referral center for a region with 500,000 inhabitants and is currently the most advanced multidisciplinary MICU in Bosnia and Herzegovina. It should be noted that Bosnia and Herzegovina is in a post-war period and a country in transition. Currently, there are 8 ICU beds in the closed ICU with 8 specialists (internal medicine, pulmonary, neurology, anesthesiology), 3 physicians in training, 1 head nurse, and 19 bedside nurses. While not formally employed by the ICU, the MICU multidisciplinary team includes a pharmacist and 2 physical therapists available Monday through Friday.
The tele-education intervention was conducted between January 2016 and December 2017 in cooperation with expert intensivists from Mayo Clinic in Rochester, Minnesota (OG), and Jacksonville, Florida (EF), who are both proficient in the local language and served as educators. The intervention involved a 45-min weekly engagement at a time that is convenient for both parties (Tuesday 2:30 pm, Europe, 8.30 am CST and 9.30 EST USA, Fig. 1). Learners included attending physicians, resident trainees, and nurses employed in the MICU. Discussion topics were selected to incorporate the breath of medical critical care, including specific organ failures and ICU syndromes, evidence-based interventions, and end of life care. During each tele-education session, a standardized patient presentation algorithm was used including the following outline.
Phase 1 (preparation)
Before each tele-education session, the host nation MICU physicians decided which active MICU patient would be presented. Selection was based on the severity and complexity of critical illness. Written patient summaries were emailed to the assigned educator prior to each session. The summary contained the reason for ICU admission, initial care, diagnostic and therapeutic procedures, and the patient’s condition in the previous 24 h.
Phase 2 (video session)
Learners presented the patient’s current clinical information followed by a diagnostic and treatment plan based on the CERTAIN Admission Checklist and the Rounding Checklist  to the assigned educator, followed by a group discussion of the clinical issues and diagnostic and management considerations. Learners included all MICU attending and resident physicians, as well the chief nurse and nurses involved in the care of the patient.
Phase 3 (evaluation)
All changes in the diagnostic and/or therapeutic plan were recorded. Electronic communication and expert evaluation continued beyond the tele-rounds ad hoc and as needed via e-mail.
Video sessions were performed via the ZOOM™ platform (www.zoom.us), which has been used as secure two-way video communication platform and is available as an application for PC Windows, Android, and iOS (Mac, Apple) operating systems. It allows for intuitive, easy to use real-time communication and secure sharing of multimedia content. The internet connection was supported by a wireless router in the MICU.
The study included all patients admitted and treated in the MICU from January 1, 2015, to December 31, 2017. Patients admitted and treated in the MICU before the intervention (before 1/1/2016) served as a control group. Patient data were collected from the University Clinical Center information system and patients’ medical records, including baseline demographics, indication and duration of mechanical ventilation, use of vasopressors, and diagnosis-related groups (DRGs). DRGs were used as a measure of the baseline severity of illness. DRGs use a method of classifying acute hospital patients into groups with similar clinical features who require similar consumption of hospital resources.
Study of intervention
Data required for DRG group determination include main diagnosis (ICD 10), additional diagnoses (complications and comorbidities ICD 10), main procedure, other procedures, age, gender, and admission source. Higher DRG values are associated with more severe clinical conditions. Main clinical outcomes included hospital and ICU length of stay, and unadjusted mortality. Data on MICU annual resource expenditures were obtained from the University Clinical Center Financial Office. Annual MICU spending is based on the consumption of pharmaceuticals, medical devices, and the cost of diagnostic and treatment services during a MICU stay. After each tele-education session, we also recorded changes in therapeutic and diagnostic algorithms, ad hoc correspondence, technical problems, and shortcomings. Changes in structure or processes were graded as “fully implemented,” “partially implemented,” or “not implemented.”
Staff satisfaction was measured using a survey instrument using questions based on the 5-point Likert scale that was distributed to all MICU employees at the of end of first year of the intervention. Sixteen providers (10 physicians, 4 charge nurses, and 2 physical therapists) who were employed in the hospital throughout the study period completed a semistructured evaluation of structure and processes in the ICU according to organ systems based on the CERTAIN platform .
Baseline patient characteristics, treatment, and outcomes were compared among 3 years: (1) 2015, (2) 2016, and (3) 2017. Continuous variables were compared among the groups using the Mann–Whitney U test, Student t-test, and ANOVA, and categorical variables using the chi-squared test. Continuous data were summarized as medians with 25th to 75th percentiles (interquartile range, IQR). To evaluate the effect of the intervention on clinical outcomes, we performed a multivariate analyses taking into consideration severity of illness including age, DRG, vasopressor use, and invasive mechanical ventilation. The IBM SPSS 20 statistical program was used for all statistical analyses. The significance of the difference was tested (p < 0.01 and p < 0.05) for the monitored parameters.
The study was approved by the local University Clinical Center Ethics Committee. All patient information was de-identified during tele-education discussions and e-mail correspondence to protect personal privacy during these interactions.
Forty-three tele-education sessions were conducted in 2016, and 39 in 2017 (82 total). Minimal technical issues (WiFi connection disruptions) were observed in 31 out of 82 sessions, and decreased over time.
Demographic and baseline characteristic of the patients included in this study and main outcomes are presented in Table 1. Age, gender distribution, and vasopressor support requirements were similar before and after intervention. DRG coefficients and the number of patients requiring invasive mechanical ventilation decreased in the period following the intervention (Table 1).
Detailed changes in structure and processes according to each organ system (based on CERTAIN checklist) are presented in Table 2. Supporting quantitative data are provided in Additional file 1: Table S1, electronic data supplement. Structural changes among others included standardized admission and rounds, hand-washing dispensers and instructions, in-ICU physical therapy, assessment of antimicrobial sensitivity, point of care ultrasound, assessment and documentation of fluid balance, pharmacist review, closed ventilator suction, default lung-protective ventilator settings, and family presence. The process changes included daily sedation interruption, spontaneous breathing trials, restrictive fluid and transfusion, daily assessment of devices, and the use of prone position and neuromuscular blockade in severe ARDS. Less effective (dopamine, thiopental, aminophylline) or expensive (low molecular weight heparin, proton pump inhibitor) medications were replaced with more effective (norepinephrine, propofol) or cheaper (unfractionated heparin, H2 blocker) alternatives. The vast majority of changes were evaluated as “fully implemented.” Standardized nutrition, avoidance of polypharmacy, the use of beta blockers, and bone marrow biopsy were considered “partially implemented” by a minority of evaluators (25%).
Multiple changes in management were observed in 18 out of 82 sessions and included changes in medications, ventilator mode and settings, and/or diagnostic procedures. Management changes were more likely to occur in more complex patients and were seen in greater frequency early in the study period (total management changes were 118 in 2016 vs 89 in 2017). The same decreasing frequency of ad hoc e-mail correspondence (16 in 2016 vs 7 in 2017) was also seen over time.
ICU and hospital mortality, length of stay, and per-patient costs markedly decreased after the intervention (Additional file 1: Table S2, electronic data supplement). When adjusted for baseline DRG coefficients and the use of mechanical ventilation, the difference in mortality, length of stay, and cost remained significant. When adjusted for age, sex, mechanical ventilation, and vasopressors, tele-education remained associated with a significant decrease in hospital mortality (Additional file 1: Table S3) and shorter ICU and hospital length of stay (Additional file 1: Table S4, electronic data supplement).
A staff satisfaction survey was completed in 2016 and 2017 by all 10 participating physicians and nurses showing a universally high satisfaction with their interactions during the tele-education program.
The implementation of structured, weekly tele-education sessions in a MICU with limited resources was feasible, well received, and associated with a significant improvement in clinical outcomes and associated costs of care. The results demonstrate that the essential elements of high-quality critical care are not expensive and can be implemented outside high-resource settings.
Although methodologically limited and inconsistent, most telemedicine studies in the ICU have reported improved clinical outcomes including mortality, reduction of the average length of hospitalization, and overall treatment costs [8,9,10,11,12]. These observed benefits are likely at least in part explained by education of bedside providers and reinforcement of best evidence-based practices that was the key focus of our intervention (i.e., tele-education). While our intervention is clearly not comparable to a comprehensive telemedicine program, we feel that the results of our study suggest tele-education may offer a favorable alternative in resource-limited settings. The cost of advanced tele-ICU systems is substantial, due to the installation, staffing, and maintenance of the command center and complicated software platforms. Yet, the evidence for benefit is limited. In contrast, our intervention used a secure but affordable video-conference platform and focused on an efficient expert knowledge transfer in the form of a case-based, concise tele-education session. We have shown that it is possible to make significant improvements with modest investments in time and technology. These results are encouraging and provide proof of concept to expand this program to other emerging ICUs in our region and in other low- and middle-income countries using a train-the-trainer model.
Our results are clearly influenced by the consistent use of a standardized approach to the evaluation and treatment of critically ill patients using the CERTAIN Admission Checklist and the CERTAIN Rounding Checklist, which both reduced practice variation and the risk of medical error. This standardized approach has become part of the practice in our ICU, leading to the reduction of unnecessary diagnostic, therapeutic procedures, and treatments in accordance with evidence-based recommendations for critical care practice. Physician and nursing staff in our MICU were very accepting and satisfied with this tele-education program, due in no small part to the interprofessional engagement and leadership support in our MICU that are commonly recognized to be essential components of a successful change management. The hospital leadership support was essential and was reinforced by the CERTAIN program influence in obtaining certification (ISO9001:2015) with our ICU becoming a referral medical ICU for an area covering 500,000 people.
To analyze the actual effect of this tele-education program on mortality and average hospitalization, we compared processes, treatments, and patient outcomes in 2015, 2016, and 2017. Although the general demographics of our population remained unchanged, we found the number of mechanically ventilated patients decreased sequentially. This could be explained by increased use of noninvasive ventilation in our setting, including its application in the step down unit of our hospital. We also recorded a decrease in the DRG coefficient during all 3 years, which may also be a reflection of decreased mechanical ventilated days or a reduction in complications due to adoption of evidence-based critical care practices (i.e., restrictive fluid resuscitation strategies).
Our study results do include several limitations that potentially impact the generalizability of these findings. We used a DRG coefficient and the use of vasopressors and invasive mechanical ventilation as markers of severity of illness. While more complex severity of illness scores such as Acute Physiology and Chronic Health Evaluation (APACHE) have not been validated in low resource settings , the absence of a standardized case-mixed adjustment is a principal methodologic limitation. In the before-after design, it is impossible to attribute the observed changes in practice and outcomes to the tele-education as opposed to natural trends. While no other quality improvement, educational interventions, or staffing changes occurred during the study period, multiple additional unmeasured factors could have influenced the observed results. The decrease in the use and duration of invasive mechanical ventilation can in part be explained by the emphasis on weaning and increased use of noninvasive ventilation after the intervention.
Although the evidence for the efficacy of the CERTAIN checklist per se is limited at this time, CERTAIN content of the CERTAIN content is non-controversial and is based on current recommendation by practice guidelines from critical care societies .
Unfortunately, formal educational outcomes were not collected. However, indirect proof of knowledge gained is the fact that five physicians subsequently passed independent critical care subspecialty exam. With regard to cost analysis, our hospital database does not keep an accurate account of per-patient spending. Therefore, we calculated the total spending based on the consumption of pharmaceuticals and medical devices and the cost of diagnostic and other services for the entire ICU before and after the intervention. Finally, the study was conducted in a single center with motivated staff and leadership support, which may not be present in all limited resource settings.
In conclusion, weekly 45-min, tele-education case-based learning sessions utilizing the CERTAIN platform were effectively adopted into the medical ICU practice in a limited resource setting and were associated with the improvement of clinical outcomes and reduced costs. Low-cost tele-education interventions can serve as a good model for educating health workers and improving care and outcomes of critically ill patients in low- and middle-income countries.
Availability of data and materials
The data that support the findings of this study are available from the corresponding author upon reasonable request.
Checklist for Early Recognition and Treatment of Acute Illness
Intensive care unit
Medical intensive care unit
World Health Organization. Telemedicine: opportunities and developments in member states: report on the second global survey on eHealth. Global Observatory for eHealth Series, 2. Geneva, World Health Organization; 2009. https://www.who.int/goe/publications/goe_telemedicine_2010.pdf.
Pannu J, Devang S, Sheley T, Schroeder DR, Kashyap R, Marquez A, et al. Impact of telemedicine monitoring of community ICUs on interhospital transfers. Crit Care Med. 2017;45(8):1344–51.
Combi C, Pozzani G, Pozzi G. Telemedicine for developing countries. A survey and some design issues. Appl Clin Inform. 2016;7(4):1025–50.
Wootton R, Geissbuhler A, Jethwani K, Kovarik C, Person DA, Vladzymyrskyy A, et al. Comparative performance of seven long-running telemedicine networks delivering humanitarian services. J Telemed Telecare. 2012;18(6):305–11.
Vukoja M, Kashyap R, Gavrilovic S, Dong Y, Kilickaya O, Gajic O. Checklist for early recognition and treatment of acute illness: international collaboration to improve critical care practice. World J Crit Care Med. 2015;4(1):55–61.
Shao M, Kashyap R, Niven A, Barwise A, Garcia-Arguello L, Suzuki R, et al. Feasibility of an international remote simulation training program in critical care delivery: a pilot study. Mayo Clin Proc Innov Qual Outcomes. 2018;2(3):229–33.
Sevilla-Berrios R, O'Horo JC, Schmickl CN, Erdogan A, Chen X, Garcia Arguello LY, et al. Prompting with electronic checklist improves clinician performance in medical emergencies: a high-fidelity simulation study. Int J Emerg Med. 2018;11(1):26.
Morrison JL, Cai Q, Davis N, Yan Y, Berbaum ML, Ries M, et al. Clinical and economic outcomes of the electronic intensive care unit: results from two community hospitals. Crit Care Med. 2010;38(1):2–8.
Lilly CM, Cody S, Zhao H, Landry K, Baker SP, McIlwaine J, et al. Hospital mortality, length of stay and preventable complications among critically ill patients before and after tele-ICU reengineering of critical care processes. JAMA. 2011;305(21):2175–83.
Young LB, Chan PS, Lu X, Nallamothu BK, Sasson C, Cram PM. Impact of telemedicine intensive care unit coverage on patient outcomes: a systematic review and meta-analysis. Arch Intern Med. 2011;171(6):498–506.
Breslow MJ, Rosenfeld BA, Doerfler M, Burke G, Yates G, Stone DJ, et al. Effect of a multiple-site intensive care unit telemedicine program on clinical and economic outcomes: an alternative paradigm for intensivist staffing. Crit Care Med. 2004;32(1):31–8.
Rosenfeld BA, Dorman T, Breslow MJ, Pronovost P, Jenckes M, Zhang N, et al. Intensive care unit telemedicine: alternate paradigm for providing continuous intensivist care. Crit Care Med. 2000;28(12):3925–31.
Vukoja M, Riviello ED, Schultz MJ. Critical care outcomes in resource-limited settings. Curr Opin Crit Care. 2018;24(5):421–7.
Barwise A, Garcia-Arguello L, Dong Y, Hulyalkar M, Vukoja M, Schultz MJ, et al. Checklist for Early Recognition and Treatment of Acute Illness (CERTAIN): evolution of a content management system for point-of-care clinical decision support. BMC Med Inform Decis Mak. 2016;16(1):12.
The study was funded in part by the grants in support of CERTAIN project from Chest Foundation and Laerdal Foundation.
Ethics approval and consent to participate
The study was approved by the local ethics committee of the University Clinical Centre of Republika Srpska, Banja Luka, Bosnia and Herzegovina.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Additional file 1:
Table S1. Granular data pertinent to changes in practice before and after tele-education intervention. Table S2. Multivariate linear regression for length of stay in the ICU. Table S3. Multivariate linear regression for length of stay in the hospital. Table S4. Multivariate linear regression for length of stay in the hospital. (DOCX 18 kb)
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
About this article
Cite this article
Kovacevic, P., Dragic, S., Kovacevic, T. et al. Impact of weekly case-based tele-education on quality of care in a limited resource medical intensive care unit. Crit Care 23, 220 (2019). https://doi.org/10.1186/s13054-019-2494-6
- Case-based learning
- Low resource
- Intensive care