articleMay 1, 2023Closed access

An Analysis of the Automatic Bug Fixing Performance of ChatGPT

DSDominik SobaniaMBMartin BrieschCHCarol HannaJPJustyna Petke

Johannes Gutenberg University Mainz · University College London

Indexed incrossref

Abstract

To support software developers in finding and fixing software bugs, several automated program repair techniques have been introduced. Given a test suite, standard methods usually either synthesize a repair, or navigate a search space of software edits to find test-suite passing variants. Recent program repair methods are based on deep learning approaches. One of these novel methods, which is not primarily intended for automated program repair, but is still suitable for it, is ChatGPT. The bug fixing performance of ChatGPT, however, is so far unclear. Therefore, in this paper we evaluate ChatGPT on the standard bug fixing benchmark set, QuixBugs, and compare the performance with the results of several other…

Citation impact

294
total citations
FWCI
61.60
Percentile
100%
References
26
Citations per year

Authors

4

Topics & keywords

Topics
Keywords
  • Computer science
  • Test suite
  • Suite
  • Benchmark (surveying)
  • Software bug
  • Software
  • Set (abstract data type)
  • Deep learning
No related works found for this paper.

Funding