A future role for health applications of large language models depends on regulators enforcing safety standards

Among the rapid integration of artificial intelligence in clinical settings, large language models (LLMs), such as Generative Pre-trained Transformer-4, have emerged as multifaceted tools that have potential for health-care delivery, diagnosis, and patient care. However, deployment of LLMs raises substantial regulatory and safety concerns. Due to their high output variability, poor inherent explainability, and the risk of so-called AI hallucinations, LLM-based health-care applications that serve a medical purpose face regulatory challenges for approval as medical devices under US and EU laws, including the recently passed EU Artificial Intelligence Act. Despite unaddressed risks for patients, including…

• BF
  Bundesministerium für Bildung und Forschung

  Award: 16KISA100K
• BF
  Bundesministerium für Bildung, Wissenschaft, Forschung und Technologie