Diagnostic reasoning prompts reveal the potential for large language model interpretability in medicine

One of the major barriers to using large language models (LLMs) in medicine is the perception they use uninterpretable methods to make clinical decisions that are inherently different from the cognitive processes of clinicians. In this manuscript we develop diagnostic reasoning prompts to study whether LLMs can imitate clinical reasoning while accurately forming a diagnosis. We find that GPT-4 can be prompted to mimic the common clinical reasoning processes of clinicians without sacrificing diagnostic accuracy. This is significant because an LLM that can imitate clinical reasoning to provide an interpretable rationale offers physicians a means to evaluate whether an LLMs response is likely correct and can be…

• UD
  U.S. Department of Veterans Affairs
• GA
  Gordon and Betty Moore Foundation

  Award: 12409
• AH
  American Heart Association
• GC
  Georgia Clinical and Translational Science Alliance

  Award: UL1TR003142
• NI
  National Institutes of Health

  Awards: UG1DA015815, UL1TR003142
• NI
  National Institute of Allergy and Infectious Diseases

  Award: 1R01AI17812101
• NC
  National Center for Advancing Translational Sciences

  Award: UL1TR003142