articleResearch Methods in Applied LinguisticsApr 19, 2023HYBRID OA

Exploring the potential of using an AI language model for automated essay scoring

AMAtsushi MizumotoMEMasaki Eguchi

Kansai University · University of Oregon

Indexed incrossref

Abstract

The widespread adoption of ChatGPT, an AI language model, has the potential to bring about significant changes to the research, teaching, and learning of foreign languages. The present study aims to leverage this technology to perform automated essay scoring (AES) and evaluate its reliability and accuracy. Specifically, we utilized the GPT-3 text-davinci-003 model to automatically score all 12,100 essays contained in the ETS Corpus of Non-Native Written English (TOEFL11) and compared these scores to benchmark levels. The study also explored the extent to which linguistic features influence AES with GPT. The results showed that AES using GPT has a certain level of accuracy and reliability and could provide…

Citation impact

518
total citations
FWCI
85.82
Percentile
100%
References
70
Citations per year

Authors

2

Topics & keywords

Topics
Keywords
  • Leverage (statistics)
  • Computer science
  • Reliability (semiconductor)
  • Benchmark (surveying)
  • Natural language processing
  • Artificial intelligence
  • Machine learning
  • Data science
UN Sustainable Development Goals
  • Quality Education
No related works found for this paper.

Funding