8 April 2022
Funding from Federal MinistryMedical appointments with a 3D avatar
Photo: Adobe Stock/Gorodenkoff
The corona pandemic has highlighted the importance of well-functioning, remote medical diagnosis in emergency situations without the possibility of contact between medical personnel and patients. The goal is to avoid infection while providing care during quarantine. This type of medical consultation is also a useful alternative to on-site consultation in cases of limited mobility, long waiting periods, dearth of specialists, complaints and symptoms that cause feelings of shame, or psychological illnesses.
But how can telemedicine be made more attractive to increase acceptance? Researchers in the interdisciplinary research project Hybride Interaktionssysteme zur Aufrechterhaltung der Gesundheit auch in Ausnahmesituationen (HIVAM) want to investigate this very question in the fields of informatics, medicine, and social sciences.
Previous applications are strongly based on traditional forms of telecommunication and are, despite their advantages, scarce in the German healthcare system. Above all, critics cite a lack of natural communication, insufficient usability, and difficulties adapting telemedicine to existing systems and processes. The goal of HIVAM is to change this situation and to develop new approaches by unifying the human intelligence of health experts, artificial intelligence (AI), and mixed reality (MR).
To do this, AI-based virtual 3D avatars for medical experts will be created. Mobile end devices, 3D glasses, and other MR applications can depict these avatars while recording the environment and patient data (e.g., blood pressure, blood sugar levels, or weight) in real time via various sensors. Unlike current speech assistants, AI-controlled 3D avatars not only understand language input; they also master facial expressions and gestures and can express emotions such as empathy using non-verbal communication—for example, by modulating facial expression, vocal range, and posture. This gives everyone involved the feeling of social presence and familiarity.
The goal is an AI assistant that can independently perform simple standard and routine tasks. Subsequently, medical experts would go through the online appointments and review the accuracy of the AI assistants. If the patients wish to speak directly to the medical experts, they can make a real or virtual appointment via the assistant.
“This way, the patients can interact seamlessly with the doctors’, therapists’, and nurses’ intelligent virtual assistants,” explains informatics specialist Prof. Dr. Frank Steinicke Universität Hamburg, spokesperson for the 3-year collaborative project. According to Steinicke, an AI-based assistant system could be developed that helps in the decision process, improves health and nursing care, addresses the consequences of social isolation and psychological stress, and optimizes on-site consultation, visits, and treatment. The moral and ethical principles of the medical profession will also be taken into consideration within the scope of the project.
In addition to the Department of Informatics at Universität Hamburg and the University Medical Center Hamburg-Eppendorf, the Hochschule Düsseldorf University of Applied Sciences and the Hamburger firms Sympatient and apoQlar are taking part in the project. The project is being funded in the amount of €1.8 million by the Federal Ministry of Education and Research as part of their call for hybrid interactive systems in healthcare in emergency situations.
More information about the project:
https://www.interaktive-technologien.de/projekte/hivam