Performance of an Artificial Intelligence Chatbot in Ophthalmic Knowledge Assessment

ChatGPT is an artificial intelligence (AI) chatbot that has significant societal implications. Training curricula using AI are being developed in medicine, and the performance of chatbots in ophthalmology has not been characterized.

To assess the performance of ChatGPT in answering practice questions for board certification in ophthalmology. Design, Setting, and Participants: This cross-sectional study used a consecutive sample of text-based multiple-choice questions provided by the OphthoQuestions practice question bank for board certification examination preparation. Of 166 available multiple-choice questions, 125 (75%) were text-based. Exposures: ChatGPT answered questions from January 9 to 16, 2023, and on February 17, 2023. Main Outcomes and Measures: Our primary outcome was the number of board certification examination practice questions that ChatGPT answered correctly. Our secondary outcomes were the proportion of questions for which ChatGPT provided additional explanations, the mean length of questions and responses provided by ChatGPT, the performance of ChatGPT in answering questions without multiple-choice options, and changes in performance over time.