|
1 | | -# Using ChainRules in your AutoDiff system |
| 1 | +# Using ChainRules in your AD system |
2 | 2 |
|
3 | 3 | This section is for authors of AD systems. |
4 | | -It assumes a pretty solid understanding of Julia, and of automatic differentiation. |
5 | | -It explains how to make use of ChainRule's rule sets, |
| 4 | +It assumes a pretty solid understanding of both Julia and automatic differentiation. |
| 5 | +It explains how to make use of ChainRule's "rulesets" ([`frule`](@ref)s, [`rrule`](@ref)s,) |
6 | 6 | to avoid having to code all your own AD primitives / custom sensitives. |
7 | 7 |
|
8 | 8 | There are 3 main ways to access ChainRules rule sets in your AutoDiff system. |
9 | 9 |
|
10 | 10 | 1. [Operation Overloading Generation](operator_overloading.html) |
11 | | - - This is primarily intended for operator overloading based AD systems which will generate overloads for primal function based for their overloaded types based on the existance of an `rrule`/`frule`. |
| 11 | + - This is primarily intended for operator overloading based AD systems which will generate overloads for primal functions based for their overloaded types based on the existence of an `rrule`/`frule`. |
12 | 12 | - A source code generation based AD can also use this by overloading their transform generating function directly so as not to recursively generate a transform but to just return the rule. |
13 | 13 | - This does not play nice with Revise.jl, adding or modifying rules in loaded files will not be reflected until a manual refresh, and deleting rules will not be reflected at all. |
14 | 14 | 2. Source code tranform based on inserting branches that check of `rrule`/`frule` return `nothing` |
15 | | - - if the `rrule`/`frule` returns a rule result then use it, if it return `nothing` then do normal AD path |
| 15 | + - If the `rrule`/`frule` returns a rule result then use it, if it returns `nothing` then do normal AD path. |
16 | 16 | - In theory type inference optimizes these branchs out; in practice it may not. |
17 | 17 | - This is a fairly simple Cassette overdub (or similar) of all calls, and is suitable for overloading based AD or source code transformation. |
18 | 18 | 3. Source code transform based on `rrule`/`frule` method-table |
19 | | - - Always use `rrule`/`frule` iff and only if use the rules that exist, else generate normal AD path. |
| 19 | + - If an applicable `rrule`/`frule` exists in the method table then use it, else generate normal AD path. |
20 | 20 | - This avoids having branches in your generated code. |
21 | | - - This requires maintaining your own back-edges |
| 21 | + - This requires maintaining your own back-edges. |
22 | 22 | - This is pretty hard-code even by the standard of source code tranformations |
0 commit comments