The comforting companion: using AI to bring loved one's voices to newborns, infants, and unconscious patients in ICU
Critical Care volume 27, Article number: 135 (2023)
Emotional support from loved ones is of the utmost importance for a patient's recovery in the challenging environment of the intensive care unit (ICU). This environment can be particularly difficult for vulnerable groups, such as newborns, infants, and minimally conscious patients. Numerous studies have proved that the presence of family members can significantly enhance a patient's physical and mental well-being, by reducing anxiety and stress [1, 2]. Additionally, emotional support has been identified as crucial for arousal in comatose patients [3,4,5]. However, providing reassurance can be challenging, especially when patients cannot communicate effectively. The absence of family members due to hospital regulations exacerbates the issue.
To address this issue, we introduce the reassurance voice mimic system which utilizes artificial intelligence (AI) technology to generate personalized soothing voice messages for each patient based on their medical condition and psychological needs. These messages can be delivered via various terminal devices, such as smart speakers and interactive toys. By providing personalized emotional support, the reassurance voice mimic system has the potential to improve the mental well-being of patients and enhance the quality of care provided by healthcare professionals.
In the intensive care unit, providing personalized care that considers the unique needs of each patient is crucial for successful treatment and recovery. To this end, we developed a prototype system that generates soothing voice messages for newborns, infants, and comatose patients by mimicking the voices of their family members (Fig. 1). The generation of individualized reassurance messages involves a complex process, which includes generating reassurance texts, reviewing and approval of the text, voice imitation, and audio message synthesis. To achieve this, the reassurance voice mimic system uses a pipeline architecture that combines various AI technologies, such as natural language processing, voice cloning, and audio synthesis. The system utilizes a language model to analyze patients' medical records, such as their medical history, progress notes, and treatment plans. By extracting relevant information about the patients, the system will then generate soothing messages that are tailored to their specific needs. These reassuring messages include phrases such as “Honey, honey, I'm your mother”, “Don't be afraid now, I will be by your side.”, “I'm so proud of you for staying strong.”, “Every day you are making progress, keep it up.”, etc. The reassurance texts are then reviewed by a healthcare provider to ensure they are appropriate and safe for the patient. The healthcare provider can either approve the text as is or revise it as needed to ensure that it does not conflict with any medical instructions or treatment plans. By providing a one-minute sound recording of the family member, a voice-cloning model will generate a voice message from the reassurance text that imitates the voice of a loved one. Next, the system will synthesize the audio message by adding personalized background music that will make the patient feel calm or joyful. Finally, the reassurance audio message can be delivered to the patient through a variety of terminal devices, such as smart speakers for adults or interactive toys for infants. Patients can interact with the system using voice commands, or other interfaces, depending on their physical and cognitive abilities.
A typical scenario for using our system is as follows: When it is necessary to communicate with an infant or minimally conscious patient, the nursing staff will initiate the system by pressing a button on the terminal device. The bedside device will then start to comfort the patient in the voice of the patient's loved one. The AI system will make adjustments to the patient's reassurance plan based on their progress notes at any given time.
One of the key benefits of our system is its ability to deliver reassurance messages that are specifically tailored to each patient's emotional needs. This personalized approach helps establish a stronger sense of emotional connection between patients and their loved ones, even in situations where physical presence is not possible due to ICU regulations. The system's employment of voice-cloning technology to replicate the familiar and soothing voice of loved ones is especially comforting and reassuring for infants and minimally conscious patients. By serving as a supportive measure for patients, the reassurance voice mimic system can effectively complement the care provided by healthcare providers in the ICU, potentially reducing their workload and stress levels, and enabling them to focus on other critical aspects of patient care. Moreover, the system has the potential to positively impact patient outcomes. Extensive research has suggested that emotional stress and anxiety can adversely affect patient outcomes while providing emotional support can help diminish the risk of complications and hasten recovery times [6, 7].
In summary, the reassurance voice mimic system offers a promising solution for delivering individualized emotional support to infants and minimally conscious patients in the ICU. Its use of artificial intelligence technology to deliver personalized messages can improve emotional well-being, alleviate stress and anxiety, and enhance the quality of care provided by healthcare providers. Although certain challenges and considerations need to be addressed, the potential benefits of this system make it a valuable and promising tool for critical care settings.
Availability of data and materials
Wong P, et al. Barriers to regaining control within a constructivist grounded theory of family resilience in ICU: Living with uncertainty. J Clin Nurs. 2017;26(23–24):4390–403.
Kotfis K, et al. The future of intensive care: delirium should no longer be an issue. Crit Care. 2022;26(1):200.
Mohsen S, et al. Impact of family presence on delirium in critically Ill patients: a retrospective cohort study. Crit Care Med. 2022;50(11):1628–37.
Zuo J, et al. The effect of family-centered sensory and affective stimulation on comatose patients with traumatic brain injury: a systematic review and meta-analysis. Int J Nurs Stud. 2021;115: 103846.
Ahmed FR, et al. Outcomes of family-centred auditory and tactile stimulation implementation on traumatic brain injured patients. Nurs Open. 2023;10(3):1601–10.
Pineda R, et al. Parent participation in the neonatal intensive care unit: predictors and relationships to neurobehavior and developmental outcomes. Early Hum Dev. 2018;117:32–8.
Fisk AC, et al. Parent perception of their role in the pediatric cardiac intensive care unit. Dimens Crit Care Nurs. 2022;41(1):2–9.
Supported by grants from the Medical health Project of Zhejiang Province (2022KY1238), Key project of the Affiliated Hospital of Jiaxing University(2022-ZD-003), Key medical discipline in Jiaxing–Hepatopancreatobiliary Surgery (2023-ZC-005), and the First Hospital of Jiaxing /Affiliated Hospital of Jiaxing University Joint Research Fund (2021LHJJ004).
Ethics approval and consent to participate
The authors declare no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Zhou, H., Wu, X. & Yu, L. The comforting companion: using AI to bring loved one's voices to newborns, infants, and unconscious patients in ICU. Crit Care 27, 135 (2023). https://doi.org/10.1186/s13054-023-04418-5