Non-differentiable operations are left off the tape, so replays return stale values and gradients #313

floor, ceil, trunc, round (also with a type argument), fld, cld and div of TrackedReals return untracked values (src/tracked.jl), so nothing is recorded for them. forward_pass! only replays recorded instructions, so downstream instructions keep the value captured at recording time. A replayed GradientTape or compiled tape then returns a wrong gradient, and gradient! with a DiffResult also a wrong value:

using ReverseDiff, DiffResults

f(x) = floor(x[1]) * x[2]
tape = ReverseDiff.GradientTape(f, [1.5, 2.0])
x = [2.5, 2.0]
r = ReverseDiff.gradient!(DiffResults.GradientResult(x), tape, x)

DiffResults.value(r)                          # 2.0, but f(x) == 4.0
DiffResults.gradient(r)                       # [0.0, 1.0], but ReverseDiff.gradient(f, x) == [0.0, 2.0]
ReverseDiff.gradient!(ReverseDiff.compile(tape), x)  # [0.0, 1.0]

This is not the documented limitation on value-dependent branching: there is no branch, the operation is just not recorded. Recording these operations (with zero derivatives) would let the forward pass recompute them while the reverse pass skips them.

#301 extends the same behaviour to broadcasts whose function returns no Dual (e.g. (t -> t > 0 ? 1.0 : -1.0).(x)), which master records.