articleMaynooth University ePrints and eTheses Archive (Maynooth University)Jan 1, 2015GREEN OA

Automatic differentiation in machine learning: a survey

AGAtılım Güneş BaydinBABarak A. PearlmutterARAlexey RadulJMJeffrey Mark Siskind

University of Oxford · Science Oxford · +3 more institutions

Abstract

Derivatives, mostly in the form of gradients and Hessians, are ubiquitous in machine learning. Automatic differentiation (AD) is a technique for calculating derivatives of numeric functions expressed as computer programs efficiently and accurately, used in fields such as computational fluid dynamics, nuclear engineering, and atmospheric sciences. Despite its advantages and use in other fields, machine learning practitioners have been little influenced by AD and make scant use of available tools. We survey the intersection of AD and machine learning, cover applications where AD has the potential to make a big impact, and report on some recent developments in the adoption of this technique. We aim to dispel some…

Citation impact

2,097
total citations
FWCI
5.88
Percentile
100%
References
165
Citations per year

Authors

4

Topics & keywords

Topics
Keywords
  • Computer science
  • Artificial intelligence
  • Relevance (law)
  • Machine learning
  • Automatic differentiation
  • Differentiable function
  • CLARITY
  • Toolbox
No related works found for this paper.

Funding