Automatic Differentiation Just Works

Automatic differentiation is really pretty fantastic. There are so many things where I would think “In principle that is differentiable, but there is no way in hell I am going to work it out, so I’ll do something else instead”, but it Just Works with AD.

John Carmack praises automatic differentiation, explaining that it makes difficult mathematical derivatives work automatically when calculating them by hand would be impractical. The joke is a relatable reaction from programmers and mathematicians: a powerful tool somehow solves problems that seem impossible to work out manually.

• @ithueti / 3591 2021-01-24