Physician- and Large Language Model–Generated Hospital Discharge Summaries

High-quality discharge summaries are associated with improved patient outcomes, but contribute to clinical documentation burden. Large language models (LLMs) provide an opportunity to support physicians by drafting discharge summary narratives.

To determine whether LLM-generated discharge summary narratives are of comparable quality and safety to those of physicians. Design, Setting, and Participants: This cross-sectional study conducted at the University of California, San Francisco included 100 randomly selected inpatient hospital medicine encounters of 3 to 6 days' duration between 2019 and 2022. The analysis took place in July 2024. Exposure: A blinded evaluation of physician- and LLM-generated narratives was performed in duplicate by 22 attending physician reviewers. Main Outcomes and Measures: Narratives were reviewed for overall quality, reviewer preference, comprehensiveness, concision, coherence, and 3 error types (inaccuracies, omissions, and hallucinations). Each error individually, and each narrative overall, were assigned potential harmfulness scores ranging from 0 to 7 on an adapted Agency for Healthcare Research and Quality scale.