Testing theory of mind in large language models and humans
Universität Hamburg · University Medical Center Hamburg-Eppendorf · +4 more institutions
Abstract
At the core of what defines us as humans is the concept of theory of mind: the ability to track other people's mental states. The recent development of large language models (LLMs) such as ChatGPT has led to intense debate about the possibility that these models exhibit behaviour that is indistinguishable from human behaviour in theory of mind tasks. Here we compare human and LLM performance on a comprehensive battery of measurements that aim to measure different theory of mind abilities, from understanding false beliefs to interpreting indirect requests and recognizing irony and faux pas. We tested two families of LLMs (GPT and LLaMA2) repeatedly against these measures and compared their performance with…
Citation impact
- FWCI
- 150.46
- Percentile
- 100%
- References
- 57
Authors
12- JWJames W. A. StrachanCorresponding
Universität Hamburg, University Medical Center Hamburg-Eppendorf
- DADalila Albergo
University of Trento, Italian Institute of Technology
- GBGiulia Borghini
Italian Institute of Technology
- OPOriana Pansardi
Universität Hamburg, Italian Institute of Technology, University Medical Center Hamburg-Eppendorf, University of Turin
- ESEugenio Scaliti
Universität Hamburg, Italian Institute of Technology, University Medical Center Hamburg-Eppendorf, University of Turin
Topics & keywords
- Psychology
- Cognitive science
- Cognitive psychology
- Computer science
- No poverty