The DaCapo benchmarks

Since benchmarks drive computer science research and industry product development, which ones we use and how we evaluate them are key questions for the community. Despite complex runtime tradeoffs due to dynamic compilation and garbage collection required for Java programs, many evaluations still use methodologies developed for C, C++, and Fortran. SPEC, the dominant purveyor of benchmarks, compounded this problem by institutionalizing these methodologies for their Java benchmark suite. This paper recommends benchmarking selection and evaluation methodologies, and introduces the DaCapo benchmarks, a set of open source, client-side Java benchmarks. We demonstrate that the complex interactions of (1)…

• NS
  National Science Foundation

  Awards: EIA-0303609, 0303609, 0085792, 0311829
• DF
  Directorate for Computer and Information Science and Engineering
• DA
  Defense Advanced Research Projects Agency

  Award: NBCH30390004