Toward expert-level medical question answering with large language models

Large language models (LLMs) have shown promise in medical question answering, with Med-PaLM being the first to exceed a 'passing' score in United States Medical Licensing Examination style questions. However, challenges remain in long-form medical question answering and handling real-world workflows. Here, we present Med-PaLM 2, which bridges these gaps with a combination of base LLM improvements, medical domain fine-tuning and new strategies for improving reasoning and grounding through ensemble refinement and chain of retrieval. Med-PaLM 2 scores up to 86.5% on the MedQA dataset, improving upon Med-PaLM by over 19%, and demonstrates dramatic performance increases across MedMCQA, PubMedQA and MMLU clinical…

• GA
  Gordon and Betty Moore Foundation

  Award: 12409
• AH
  American Heart Association
• NI
  National Institutes of Health

  Award: UG1DA015815
• SB
  Stanford Bio-X
• NI
  National Institute of Allergy and Infectious Diseases

  Award: 1R01AI17812101
• NC
  National Center for Advancing Translational Sciences