Skip to content

Fix argument handling in @grad_from_chainrules - #328

Merged
devmotion merged 10 commits into
masterfrom
dmw/grad-from-chainrules-fixes
Nov 9, 2026
Merged

devmotion merged 10 commits into
masterfrom
dmw/grad-from-chainrules-fixes

Conversation

@devmotion

@devmotion devmotion commented Nov 4, 2026 •

Copy link
Copy Markdown
Member

Fixes #314, fixes #315, fixes #319, fixes #320.

@grad_from_chainrules reuses the imported signature's argument list in value positions — both in the forwarding call to ReverseDiff.track and in the args = (...) tuple. A type annotation means something different there, so several signature shapes were broken:

Shape Symptom
f(x::TrackedArray, ys::T...) TypeError: expected T, got a value of type Tuple{T, T} — the collected varargs are asserted against the element type. Also hit with zero varargs (Tuple{}).
f(x::TrackedArray, ys::T...) T was never escaped, so it resolved in ReverseDiff's scope: UndefVarError for any user-defined type (#319).
f(x::TrackedArray, y::Int = 2) The default made it into call position, where track(f, x, y = 2) passes y as a keyword argument.
f(x::TrackedArray; k = 1) _make_fwd_args rewrote the :parameters expression in place, and all three argument lists share it, so the declared defaults were dropped.

Only untyped args... — the one form the tests covered — was unaffected.

Approach

The macro now builds the generated argument lists directly from MacroTools.splitdef/splitarg/combinearg, and _make_fwd_args is removed. The forwarding call and the args tuple use argument names only.

Everything user-supplied in the signature is escaped — the function, where parameters, argument names, types and defaults — while the macro's own locals rely on standard hygiene. So a user argument named kwargs, args or back cannot clash with them. Because escaped argument names could shadow an escaped reference to the function in a method body, both generated definitions bind the function through its type instead ((func::_selftype(f))(...), track(func::_selftype(f), ...)). _selftype returns Type{T} for a type T and typeof(f) otherwise, like ChainRulesCore's helper of the same name: typeof would give DataType for constructors, and Core.Typeof is renamed in Julia 1.14.

Also in here

  • where clauses are supported (@grad_from_chainrules does not support where clauses #320). The escaped static parameters are attached to the generated definitions. Works whether the parameter appears in the tracked argument (x::TrackedArray{V}), only in an untracked one (y::AbstractVector{V}), only in an argument with a default (y::V = 2), only in a keyword argument (k::V = 1), carries a bound (V <: Real), or is used only in the bound of another parameter (where {T, S <: TrackedArray{T}}). track and the special_*_exec! methods only see positional types, so they get only the parameters that positional types refer to, directly or through bounds; otherwise Julia warns about unused type variables.
  • A pullback returning the wrong number of tangents now throws (@grad/@grad_from_chainrules silently reuse a tangent if a pullback returns too few tangents #315). The old @assert input_derivs isa Tuple only checked the type, and tuple broadcasting silently expands a length-1 tuple across every argument — so a pullback returning too few tangents assigned its first tangent to every argument. The same check now applies to @grad. For rrule pullbacks the expected length includes the tangent of the function itself.
  • Zero tangents for tracked arguments no longer error (Zero tangents (NoTangent, ZeroTangent, nothing) for tracked arguments throw a MethodError in @grad/@grad_from_chainrules #314). NoTangent()/ZeroTangent() from an rrule and nothing from a @grad pullback reached increment_deriv!, which has no method for them. They are now no-ops.
  • Constructors get their own special instructions (related to Custom rules collide with ReverseDiff's own SpecialInstructions for the same function #316). For a type, SpecialInstruction's function parameter was DataType, so the special_*_exec! methods of two constructors with the same positional signature overwrote each other — an error during package precompilation. _SpecialInstruction now uses _selftype(func), and both macros dispatch on SpecialInstruction{_selftype(f)}. @grad also uses _selftype for its track method, so it works for constructors too. No built-in instruction records a type, so only user rules are affected.
  • Signatures without positional arguments are rejected. Keyword arguments do not take part in dispatch, so @grad_from_chainrules f(; k::TrackedArray) replaced the original f(; k), and the rrule then recursed into it.
  • The docstring documents that overlapping imported signatures need the fully tracked combination imported too, or the generated special_*_exec! methods are ambiguous.

Note for reviewers

The f_vararg and f_kw rrules in test/ChainRulesTests.jl returned two tangents for four arguments and passed only because the varargs were untracked Ints that hit the _add_to_deriv!(d1, d2) = nothing fallback. They now return the missing NoTangent()s. That was a latent wrong-gradient bug, not a regression from this PR.

The stricter length check is a behaviour change for rules that return short tangent tuples: they now error instead of being silently broadcast. @grad is not public API, but it is used downstream, so I checked the @grad rules in the registered packages GitHub code search finds using it (Tullio, DiffEqBase, NonlinearSolveBase, SimpleNonlinearSolve, Bijectors, SphericalFunctions, ADEV): all of them return one tangent per argument.

@grad with qualified names (#318) is out of scope here and left for a follow-up that moves @grad to the same approach.

Tests

New cases in test/ChainRulesTests.jl:

  • typed varargs with a user type (zero and two varargs)
  • pullbacks returning too few tangents or a non-tuple, for both macros
  • keyword defaults (the rrule uses a different default, so a dropped signature default is visible)
  • positional defaults
  • zero tangents for tracked arrays and scalars, including nothing from @grad
  • where clauses in six shapes, with @test_nowarn for the unused type variable warnings
  • constructors (Doubled(x::TrackedArray)), including a second constructor with the same signature (no method overwrite), a @grad constructor, and that constructors of other types are unaffected
  • qualified function names
  • argument names that coincide with the function or with internal names
  • invalid signatures, including keyword-only ones, with their error messages

🤖 Generated with Claude Code

devmotion and others added 5 commits November 5, 2026 00:27
The macro reused the signature's argument list in value positions, which
broke several signature shapes:

- typed varargs (`ys::T...`) were re-asserted against their element type,
  and the vararg type was never escaped, so it resolved in ReverseDiff's
  scope instead of the caller's
- positional defaults (`y::Int = 2`) were forwarded as keyword arguments
- `_make_fwd_args` rewrote the `:parameters` expression in place, and it is
  shared by all three argument lists, so the declared kwarg defaults were
  dropped

The forwarding call now passes argument names only, through `getargs_expr`,
and per-argument escaping goes through `MacroTools.splitarg`/`combinearg`.

Beyond that:

- `where` clauses are supported. `MacroTools.splitdef` unwraps the
  signature and the escaped static parameters are attached to all four
  generated definitions.
- A pullback that returns the wrong number of tangents now throws instead
  of silently broadcasting the first tangent over every argument. The same
  check applies to `@grad`. The `f_vararg` and `f_kw` rrules in the tests
  relied on the old behaviour and have been corrected.
- `NoTangent()`/`ZeroTangent()` for a tracked argument no longer reaches
  `increment_deriv!`, which has no method for `AbstractZero`.
- The docstring documents that overlapping imported signatures need the
  fully tracked combination imported too, or dispatch is ambiguous.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Build the generated argument lists directly from `MacroTools.splitdef` and
`splitarg` and remove `_make_fwd_args`. Escape all user-supplied parts of the
signature, including argument names, and bind the function through its type
so an argument named like the function cannot shadow it. Take argument types
from `splitarg`, which fixes unused type variable warnings for `where`
parameters in arguments with defaults. Count the function's own tangent in
the length check for rrule pullbacks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…/grad-from-chainrules-fixes

# Conflicts:
#	src/macros.jl
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Nov 7, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.40%. Comparing base (608569b) to head (561dd59).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #328      +/-   ##
==========================================
+ Coverage   94.32%   94.40%   +0.08%     
==========================================
  Files          19       19              
  Lines        1710     1735      +25     
==========================================
+ Hits         1613     1638      +25     
  Misses         97       97              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

devmotion and others added 5 commits November 8, 2026 00:51
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bind the function via `Type{T}` for constructors instead of `DataType`, which
made the generated method ambiguous for all types. `Core.Typeof` is renamed in
Julia 1.14, so use an internal helper as ChainRulesCore does.

Methods that only see positional types no longer declare `where` parameters
that are used only in keyword types, which caused warnings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Keyword arguments do not dispatch, so a keyword-only signature replaced the
original method and recursed. Also escape untyped arguments like all other
types, cover the `L <: T <: U` form of `where` parameters, and check the
messages of the signature errors.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`MacroTools.combinearg` drops the annotation only for an unescaped `:Any`.
Escaping it turned an untyped keyword splat into `kwargs::Any...`, which
Julia rejects.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
For a type, the function parameter of `SpecialInstruction` was `DataType`, so
the methods generated for constructors with the same positional signature
overwrote each other. `@grad` now uses `_selftype` as well, which makes it
work for constructors.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@devmotion
devmotion marked this pull request as ready for review November 9, 2026 15:28
@devmotion
devmotion merged commit 74fe920 into master Nov 9, 2026
9 checks passed
devmotion added a commit that referenced this pull request Nov 10, 2026
* Fix argument handling in `@grad`

Rework `@grad` along the lines of #328: escape only the user-supplied
parts and let macro hygiene rename the internals instead of escaping the
whole output and using `@gensym`.

This fixes unnamed arguments such as `::Val{N}`, arguments named like
the function, and signatures without positional arguments (now an error
at expansion). Keyword arguments are forwarded to the user's definition
unchanged, which also drops the `getargs_expr`, `getkwargs_expr` and
`remove_tp` helpers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Add regression tests for qualified names and keyword replay in `@grad`

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Fix keyword replay test and formatting

The pullback recorded on the tape is reused when replaying, so the test
rule now reads the input array that the replay updates, and also checks
the value computed in the forward pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Reject unnamed and non-definition input in `@grad`

Like `@grad_from_chainrules`, `@grad` now checks its input up front
instead of failing in MacroTools or with a `KeyError(:name)`. The tests
match the full error messages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Unify input check in `@grad` and reference #187 in replay test

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
yebai added a commit to TuringLang/Bijectors.jl that referenced this pull request Nov 11, 2026
* Fix loading the ReverseDiff extension with ReverseDiff 1.18.8

The three removed rules name functions the extension never imports, so
they only defined unused local functions. ReverseDiff 1.18.8
(JuliaDiff/ReverseDiff.jl#328) adds the method to `typeof(f)`, which
requires `f` to exist, so loading fails with `UndefVarError`.

Assisted-by: Claude Code <noreply@anthropic.com>

* Make doctests independent of distribution printing

Distributions now prints `LogNormal()` as a multi-line block, so the
doctests suppress that output instead of matching it.

Assisted-by: Claude Code <noreply@anthropic.com>

* Compute `find_alpha` partials in the ReverseDiff pullback

ReverseDiff re-runs the `@grad` body on each forward pass of a
recorded tape but keeps the pullback from recording time. Partials
computed from tracked inputs in the body therefore raise `tape has
finished recording` on ReverseDiff 1.18.4+, and partials computed in
the body from any values are stale when a tape is replayed at a new
input, as they already were before 1.18.4.

Assisted-by: Claude Code <noreply@anthropic.com>

* Accept untracked arguments in the ReverseDiff `find_alpha` rule

Reusing a tape through `find_alpha` at a new input raised a
`MethodError` when some of its arguments were untracked, for example
the inverse `PlanarLayer` with plain `Float64` parameters. ReverseDiff
stores untracked inputs of a recorded call as plain values, but the
`@grad` signature required `TrackedReal` for every argument.

The tracking method needs no change: mixed arguments reach it through
the promotion in `find_alpha(::Real, ::Real, ::Real)`.

Assisted-by: Claude Code <noreply@anthropic.com>
shravanngoswamii pushed a commit to TuringLang/Bijectors.jl that referenced this pull request Nov 11, 2026
* Fix loading the ReverseDiff extension with ReverseDiff 1.18.8

The three removed rules name functions the extension never imports, so
they only defined unused local functions. ReverseDiff 1.18.8
(JuliaDiff/ReverseDiff.jl#328) adds the method to `typeof(f)`, which
requires `f` to exist, so loading fails with `UndefVarError`.

Assisted-by: Claude Code <noreply@anthropic.com>

* Make doctests independent of distribution printing

Distributions now prints `LogNormal()` as a multi-line block, so the
doctests suppress that output instead of matching it.

Assisted-by: Claude Code <noreply@anthropic.com>

* Compute `find_alpha` partials in the ReverseDiff pullback

ReverseDiff re-runs the `@grad` body on each forward pass of a
recorded tape but keeps the pullback from recording time. Partials
computed from tracked inputs in the body therefore raise `tape has
finished recording` on ReverseDiff 1.18.4+, and partials computed in
the body from any values are stale when a tape is replayed at a new
input, as they already were before 1.18.4.

Assisted-by: Claude Code <noreply@anthropic.com>

* Accept untracked arguments in the ReverseDiff `find_alpha` rule

Reusing a tape through `find_alpha` at a new input raised a
`MethodError` when some of its arguments were untracked, for example
the inverse `PlanarLayer` with plain `Float64` parameters. ReverseDiff
stores untracked inputs of a recorded call as plain values, but the
`@grad` signature required `TrackedReal` for every argument.

The tracking method needs no change: mixed arguments reach it through
the promotion in `find_alpha(::Real, ::Real, ::Real)`.

Assisted-by: Claude Code <noreply@anthropic.com>

* Fix triangular helpers for adjoint Cholesky factors on Julia 1.13

On Julia 1.13, `Cholesky.U` of a factorisation stored as 'L' wraps
`factors'` without copying, so `parent(C.U)` is an `Adjoint`.
`upper_triangular` returned `convert(typeof(A), UpperTriangular(A))`,
but Julia cannot `convert` a triangular wrapper to an `Adjoint`, so it
threw a `MethodError`:

    upper_triangular(parent(cholesky(Hermitian([2.0 1; 1 2], :L)).U))

`cholesky_upper` requests 'U' storage, so this surfaces only where
`cholesky` stores 'L' regardless: Mooncake forward mode does, so
`CorrBijector` and `VecCorrBijector` failed under it.
`lower_triangular` has the same latent defect for 'U'-stored factors.

Use `triu`/`tril` instead, which return a dense array for wrapper
types. A regression test covers both helpers.

Assisted-by: Claude Code <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant