Why human–AI relationships need socioaffective alignment
University of Oxford · Research Institute in Science of Cyber Security · +5 more institutions
Abstract
Humans strive to design safe AI systems that align with our goals and remain under our control. However, as AI capabilities advance, we face a new challenge: the emergence of deeper, more persistent relationships between humans and AI systems. We explore how increasingly capable AI agents may generate the perception of deeper relationships with users, especially as AI becomes more personalised and agentic. This shift, from transactional interaction to ongoing sustained social engagement with AI, necessitates a new focus on socioaffective alignment—how an AI system behaves within the social and psychological ecosystem co-created with its user, where preferences and perceptions evolve through mutual influence.…
Citation impact
- FWCI
- 99.22
- Percentile
- 100%
- References
- 97
Authors
5- HRHannah Rose KirkCorresponding
University of Oxford, Research Institute in Science of Cyber Security
- IGIason Gabriel
Google DeepMind (United Kingdom), Google (United Kingdom)
- CSChris Summerfield
University of Oxford, Research Institute in Science of Cyber Security
- BVBertie Vidgen
Contextual Change (United States)
- SAScott A. Hale
University of Oxford, Aedas (United Kingdom), Takeda (United Kingdom)
Topics & keywords
- Artificial intelligence
- Computer science
- Natural language processing
- Reduced inequalities