GPT is an effective tool for multilingual psychological text analysis

The social and behavioral sciences have been increasingly using automated text analysis to measure psychological constructs in text. We explore whether GPT, the large-language model (LLM) underlying the AI chatbot ChatGPT, can be used as a tool for automated psychological text analysis in several languages. Across 15 datasets ( n = 47,925 manually annotated tweets and news headlines), we tested whether different versions of GPT (3.5 Turbo, 4, and 4 Turbo) can accurately detect psychological constructs (sentiment, discrete emotions, offensiveness, and moral foundations) across 12 languages. We found that GPT ( r = 0.59 to 0.77) performed much better than English-language dictionary analysis ( r = 0.20 to 0.30)…

• RS
  Russell Sage Foundation

  Award: G-2110-33990
• SF
  Sage Foundation
• TW
  Templeton World Charity Foundation

  Award: TWCF-31570
• NS
  Natural Sciences and Engineering Research Council of Canada

  Award: 567554-2022
• NI
  National Institute of Mental Health

  Award: R01MH119511