Rolling Memory: A New Approach to Annotation with Generative LLMs in Social and Political Research

Abstract Generative Large Language Models (LLMs) have shown promising results in text annotation tasks, which is of interest to social scientists. The most commonly used approaches, zero-shot and few-shot learning, do not sufficiently exploit the in-context learning capabilities of these models. Extant work demonstrates that allowing these models to retain memory throughout the annotation task increases performance considerably. In this article, we propose a refinement to the memory approach from Timoneda et al. (Memory Is All You Need: Testing How Model Memory Affects LLM Performance in Annotation Tasks. arXiv preprint: 2503.04874, 2025) that leads to significant performance gains over the original version.…

• PU
  Purdue University