How Does ChatGPT Perform on the United States Medical Licensing Examination (USMLE)? The Implications of Large Language Models for Medical Education and Knowledge Assessment

Gilson, Aidan; Safranek, Conrad; Huang, Thomas; Socrates, Vimig; Chi, Ling; Taylor, Richard A.; Chartash, David

doi:10.2196/45312

articleJMIR Medical EducationFeb 8, 2023GOLD OA

How Does ChatGPT Perform on the United States Medical Licensing Examination (USMLE)? The Implications of Large Language Models for Medical Education and Knowledge Assessment

AGAidan Gilson CSConrad Safranek THThomas Huang VSVimig Socrates LCLing Chi

Yale University · University College Dublin · +1 more institution

PubMed

Indexed incrossrefdoajpubmed

Abstract

Background

Chat Generative Pre-trained Transformer (ChatGPT) is a 175-billion-parameter natural language processing model that can generate conversation-style responses to user input.

Objective

This study aimed to evaluate the performance of ChatGPT on questions within the scope of the United States Medical Licensing Examination (USMLE) Step 1 and Step 2 exams, as well as to analyze responses for user interpretability.

Citation impact

1,997

total citations

FWCI: 72.09
Percentile: 100%
References: 21

Citations per year

Authors

7

Topics & keywords

Topics

Keywords

Interpretability
United States Medical Licensing Examination
Computer science
Set (abstract data type)
Medical education
Artificial intelligence
Medical school
Medicine

UN Sustainable Development Goals

Quality Education

No related works found for this paper.