Repository navigation
Value is sometimes not set when using DiffResults #269
Description
Here's an MWE I'm struggling to understand. My best guess sofar is that this is hitting a Julia compiler optimization which somehow eludes the value computation. Unless I'm just holding the thing wrong?
using ReverseDiff: ReverseDiff
using DiffResults: DiffResults
f(x) = sum(abs2, x)
x = [3.0, 5.0]
compiled_tape = ReverseDiff.compile(ReverseDiff.GradientTape(f, x))
function value_and_gradient1!(grad, tape, x)
result = DiffResults.MutableDiffResult(zero(eltype(x)), (grad,))
result = ReverseDiff.gradient!(result, tape, x)
return DiffResults.value(result), DiffResults.gradient(result)
end
function value_and_gradient2!(grad, tape, x)
result = DiffResults.MutableDiffResult(zero(eltype(x)), (grad,))
result = ReverseDiff.gradient!(result, tape, x)
return DiffResults.value(result), grad
end
function value_and_gradient_nested1!(grad, tape, x)
y, _ = value_and_gradient1!(grad, tape, x)
return y, grad
end
function value_and_gradient_nested2!(grad, tape, x)
y, _ = value_and_gradient2!(grad, tape, x)
return y, grad
endjulia> value_and_gradient1!(similar(x), compiled_tape, x)
(34.0, [6.0, 10.0])
julia> value_and_gradient_nested1!(similar(x), compiled_tape, x) # wrong value
(0.0, [6.0, 10.0])
julia> value_and_gradient2!(similar(x), compiled_tape, x)
(34.0, [6.0, 10.0])
julia> value_and_gradient_nested2!(similar(x), compiled_tape, x)
(34.0, [6.0, 10.0])Related:
Details
(jl_gKdWaj) pkg> st
Status `/private/var/folders/pt/myrxq2gs5hzcp79v5475_dh40000gp/T/jl_gKdWaj/Project.toml`
[163ba53b] DiffResults v1.1.0
[37e2e3b7] ReverseDiff v1.15.3
julia> versioninfo()
Julia Version 1.11.3
Commit d63adeda50d (2025-01-21 19:42 UTC)
Build Info:
Official https://julialang.org/ release
Platform Info:
OS: macOS (arm64-apple-darwin24.0.0)
CPU: 11 × Apple M3 Pro
WORD_SIZE: 64
LLVM: libLLVM-16.0.6 (ORCJIT, apple-m2)
Threads: 5 default, 0 interactive, 2 GC (on 5 virtual cores)
Environment:
JULIA_EDITOR = code
Activity
nsajko commentedon Feb 16, 2025
This doesn't reproduce on nightly Julia.
nsajko commentedon Feb 16, 2025
BTW, as mentioned on Slack,
--compile=minis a good way to check if a bug is a miscompilation.gdalle commentedon Feb 16, 2025
You're right, with
--compile=minit disappears as well. Not sure what to do about it then.nsajko commentedon Feb 16, 2025
Create a MWE and report an issue to JuliaLang/julia.
Create the MWE by:
Another thing to do is bisect the fix (given that the bug is perhaps fixed on nightly), and see if it's backportable.
gdalle commentedon Feb 16, 2025
Unfortunately I wouldn't even know where to begin to inline ReverseDiff. I tried removing it entirely and the effect disappeared.
devmotion commentedon Sep 16, 2026
This still reproduces, and it isn't a ReverseDiff bug — it's the Julia optimizer bug JuliaLang/julia#62812, fixed by #57201 in Julia 1.11.5 and never backported to the 1.10 LTS. Wrong on Julia 1.8 – 1.11.4, correct before and after, regardless of the ReverseDiff version. That's why it looked fixed on nightly in February.
Two corrections to the original report: it hits a plain
GradientTapetoo, not just a compiled one, and only the gradient path is affected —jacobian!,hessian!andgradient!(result, f, x)are fine. A package-free reproducer is in the upstream issue.Fix: no compiler barrier and no version check needed — just re-alias
resultthe way the DiffResults docs ask for (#251), on the whole chain:src/api/gradients.jlandseeded_reverse_pass!(result, ::AbstractTape)insrc/api/tape.jl. Still 0 allocations, still type stable.One note for #261: re-aliasing only helps if every level does it. On 1.10.12, #261 alone is still wrong,
tape.jlalone is still wrong, both together are correct. #261 doesn't touchtape.jl, and since that method gets inlined, re-aliasing only ingradients.jlrebinds the same value and changes nothing — so #261 needs one more line to close this.Realias `result` on the gradient path (fixes #269)