Repository navigation
Fix argument handling in @grad_from_chainrules - #328
Merged
Merged
Conversation
The macro reused the signature's argument list in value positions, which broke several signature shapes: - typed varargs (`ys::T...`) were re-asserted against their element type, and the vararg type was never escaped, so it resolved in ReverseDiff's scope instead of the caller's - positional defaults (`y::Int = 2`) were forwarded as keyword arguments - `_make_fwd_args` rewrote the `:parameters` expression in place, and it is shared by all three argument lists, so the declared kwarg defaults were dropped The forwarding call now passes argument names only, through `getargs_expr`, and per-argument escaping goes through `MacroTools.splitarg`/`combinearg`. Beyond that: - `where` clauses are supported. `MacroTools.splitdef` unwraps the signature and the escaped static parameters are attached to all four generated definitions. - A pullback that returns the wrong number of tangents now throws instead of silently broadcasting the first tangent over every argument. The same check applies to `@grad`. The `f_vararg` and `f_kw` rrules in the tests relied on the old behaviour and have been corrected. - `NoTangent()`/`ZeroTangent()` for a tracked argument no longer reaches `increment_deriv!`, which has no method for `AbstractZero`. - The docstring documents that overlapping imported signatures need the fully tracked combination imported too, or dispatch is ambiguous. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Build the generated argument lists directly from `MacroTools.splitdef` and `splitarg` and remove `_make_fwd_args`. Escape all user-supplied parts of the signature, including argument names, and bind the function through its type so an argument named like the function cannot shadow it. Take argument types from `splitarg`, which fixes unused type variable warnings for `where` parameters in arguments with defaults. Count the function's own tangent in the length check for rrule pullbacks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…/grad-from-chainrules-fixes # Conflicts: # src/macros.jl
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #328 +/- ##
==========================================
+ Coverage 94.32% 94.40% +0.08%
==========================================
Files 19 19
Lines 1710 1735 +25
==========================================
+ Hits 1613 1638 +25
Misses 97 97 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bind the function via `Type{T}` for constructors instead of `DataType`, which
made the generated method ambiguous for all types. `Core.Typeof` is renamed in
Julia 1.14, so use an internal helper as ChainRulesCore does.
Methods that only see positional types no longer declare `where` parameters
that are used only in keyword types, which caused warnings.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Keyword arguments do not dispatch, so a keyword-only signature replaced the original method and recursed. Also escape untyped arguments like all other types, cover the `L <: T <: U` form of `where` parameters, and check the messages of the signature errors. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`MacroTools.combinearg` drops the annotation only for an unescaped `:Any`. Escaping it turned an untyped keyword splat into `kwargs::Any...`, which Julia rejects. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
For a type, the function parameter of `SpecialInstruction` was `DataType`, so the methods generated for constructors with the same positional signature overwrote each other. `@grad` now uses `_selftype` as well, which makes it work for constructors. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
devmotion
marked this pull request as ready for review
November 9, 2026 15:28
devmotion
added a commit
that referenced
this pull request
Nov 10, 2026
* Fix argument handling in `@grad` Rework `@grad` along the lines of #328: escape only the user-supplied parts and let macro hygiene rename the internals instead of escaping the whole output and using `@gensym`. This fixes unnamed arguments such as `::Val{N}`, arguments named like the function, and signatures without positional arguments (now an error at expansion). Keyword arguments are forwarded to the user's definition unchanged, which also drops the `getargs_expr`, `getkwargs_expr` and `remove_tp` helpers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Add regression tests for qualified names and keyword replay in `@grad` Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Fix keyword replay test and formatting The pullback recorded on the tape is reused when replaying, so the test rule now reads the input array that the replay updates, and also checks the value computed in the forward pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Reject unnamed and non-definition input in `@grad` Like `@grad_from_chainrules`, `@grad` now checks its input up front instead of failing in MacroTools or with a `KeyError(:name)`. The tests match the full error messages. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Unify input check in `@grad` and reference #187 in replay test Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
yebai
added a commit
to TuringLang/Bijectors.jl
that referenced
this pull request
Nov 11, 2026
* Fix loading the ReverseDiff extension with ReverseDiff 1.18.8 The three removed rules name functions the extension never imports, so they only defined unused local functions. ReverseDiff 1.18.8 (JuliaDiff/ReverseDiff.jl#328) adds the method to `typeof(f)`, which requires `f` to exist, so loading fails with `UndefVarError`. Assisted-by: Claude Code <noreply@anthropic.com> * Make doctests independent of distribution printing Distributions now prints `LogNormal()` as a multi-line block, so the doctests suppress that output instead of matching it. Assisted-by: Claude Code <noreply@anthropic.com> * Compute `find_alpha` partials in the ReverseDiff pullback ReverseDiff re-runs the `@grad` body on each forward pass of a recorded tape but keeps the pullback from recording time. Partials computed from tracked inputs in the body therefore raise `tape has finished recording` on ReverseDiff 1.18.4+, and partials computed in the body from any values are stale when a tape is replayed at a new input, as they already were before 1.18.4. Assisted-by: Claude Code <noreply@anthropic.com> * Accept untracked arguments in the ReverseDiff `find_alpha` rule Reusing a tape through `find_alpha` at a new input raised a `MethodError` when some of its arguments were untracked, for example the inverse `PlanarLayer` with plain `Float64` parameters. ReverseDiff stores untracked inputs of a recorded call as plain values, but the `@grad` signature required `TrackedReal` for every argument. The tracking method needs no change: mixed arguments reach it through the promotion in `find_alpha(::Real, ::Real, ::Real)`. Assisted-by: Claude Code <noreply@anthropic.com>
shravanngoswamii
pushed a commit
to TuringLang/Bijectors.jl
that referenced
this pull request
Nov 11, 2026
* Fix loading the ReverseDiff extension with ReverseDiff 1.18.8 The three removed rules name functions the extension never imports, so they only defined unused local functions. ReverseDiff 1.18.8 (JuliaDiff/ReverseDiff.jl#328) adds the method to `typeof(f)`, which requires `f` to exist, so loading fails with `UndefVarError`. Assisted-by: Claude Code <noreply@anthropic.com> * Make doctests independent of distribution printing Distributions now prints `LogNormal()` as a multi-line block, so the doctests suppress that output instead of matching it. Assisted-by: Claude Code <noreply@anthropic.com> * Compute `find_alpha` partials in the ReverseDiff pullback ReverseDiff re-runs the `@grad` body on each forward pass of a recorded tape but keeps the pullback from recording time. Partials computed from tracked inputs in the body therefore raise `tape has finished recording` on ReverseDiff 1.18.4+, and partials computed in the body from any values are stale when a tape is replayed at a new input, as they already were before 1.18.4. Assisted-by: Claude Code <noreply@anthropic.com> * Accept untracked arguments in the ReverseDiff `find_alpha` rule Reusing a tape through `find_alpha` at a new input raised a `MethodError` when some of its arguments were untracked, for example the inverse `PlanarLayer` with plain `Float64` parameters. ReverseDiff stores untracked inputs of a recorded call as plain values, but the `@grad` signature required `TrackedReal` for every argument. The tracking method needs no change: mixed arguments reach it through the promotion in `find_alpha(::Real, ::Real, ::Real)`. Assisted-by: Claude Code <noreply@anthropic.com> * Fix triangular helpers for adjoint Cholesky factors on Julia 1.13 On Julia 1.13, `Cholesky.U` of a factorisation stored as 'L' wraps `factors'` without copying, so `parent(C.U)` is an `Adjoint`. `upper_triangular` returned `convert(typeof(A), UpperTriangular(A))`, but Julia cannot `convert` a triangular wrapper to an `Adjoint`, so it threw a `MethodError`: upper_triangular(parent(cholesky(Hermitian([2.0 1; 1 2], :L)).U)) `cholesky_upper` requests 'U' storage, so this surfaces only where `cholesky` stores 'L' regardless: Mooncake forward mode does, so `CorrBijector` and `VecCorrBijector` failed under it. `lower_triangular` has the same latent defect for 'U'-stored factors. Use `triu`/`tril` instead, which return a dense array for wrapper types. A regression test covers both helpers. Assisted-by: Claude Code <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #314, fixes #315, fixes #319, fixes #320.
@grad_from_chainrulesreuses the imported signature's argument list in value positions — both in the forwarding call toReverseDiff.trackand in theargs = (...)tuple. A type annotation means something different there, so several signature shapes were broken:f(x::TrackedArray, ys::T...)TypeError: expected T, got a value of type Tuple{T, T}— the collected varargs are asserted against the element type. Also hit with zero varargs (Tuple{}).f(x::TrackedArray, ys::T...)Twas never escaped, so it resolved in ReverseDiff's scope:UndefVarErrorfor any user-defined type (#319).f(x::TrackedArray, y::Int = 2)track(f, x, y = 2)passesyas a keyword argument.f(x::TrackedArray; k = 1)_make_fwd_argsrewrote the:parametersexpression in place, and all three argument lists share it, so the declared defaults were dropped.Only untyped
args...— the one form the tests covered — was unaffected.Approach
The macro now builds the generated argument lists directly from
MacroTools.splitdef/splitarg/combinearg, and_make_fwd_argsis removed. The forwarding call and theargstuple use argument names only.Everything user-supplied in the signature is escaped — the function,
whereparameters, argument names, types and defaults — while the macro's own locals rely on standard hygiene. So a user argument namedkwargs,argsorbackcannot clash with them. Because escaped argument names could shadow an escaped reference to the function in a method body, both generated definitions bind the function through its type instead ((func::_selftype(f))(...),track(func::_selftype(f), ...))._selftypereturnsType{T}for a typeTandtypeof(f)otherwise, like ChainRulesCore's helper of the same name:typeofwould giveDataTypefor constructors, andCore.Typeofis renamed in Julia 1.14.Also in here
whereclauses are supported (@grad_from_chainrulesdoes not supportwhereclauses #320). The escaped static parameters are attached to the generated definitions. Works whether the parameter appears in the tracked argument (x::TrackedArray{V}), only in an untracked one (y::AbstractVector{V}), only in an argument with a default (y::V = 2), only in a keyword argument (k::V = 1), carries a bound (V <: Real), or is used only in the bound of another parameter (where {T, S <: TrackedArray{T}}).trackand thespecial_*_exec!methods only see positional types, so they get only the parameters that positional types refer to, directly or through bounds; otherwise Julia warns about unused type variables.@grad/@grad_from_chainrulessilently reuse a tangent if a pullback returns too few tangents #315). The old@assert input_derivs isa Tupleonly checked the type, and tuple broadcasting silently expands a length-1 tuple across every argument — so a pullback returning too few tangents assigned its first tangent to every argument. The same check now applies to@grad. For rrule pullbacks the expected length includes the tangent of the function itself.NoTangent,ZeroTangent,nothing) for tracked arguments throw aMethodErrorin@grad/@grad_from_chainrules#314).NoTangent()/ZeroTangent()from an rrule andnothingfrom a@gradpullback reachedincrement_deriv!, which has no method for them. They are now no-ops.SpecialInstructions for the same function #316). For a type,SpecialInstruction's function parameter wasDataType, so thespecial_*_exec!methods of two constructors with the same positional signature overwrote each other — an error during package precompilation._SpecialInstructionnow uses_selftype(func), and both macros dispatch onSpecialInstruction{_selftype(f)}.@gradalso uses_selftypefor itstrackmethod, so it works for constructors too. No built-in instruction records a type, so only user rules are affected.@grad_from_chainrules f(; k::TrackedArray)replaced the originalf(; k), and the rrule then recursed into it.special_*_exec!methods are ambiguous.Note for reviewers
The
f_varargandf_kwrrules intest/ChainRulesTests.jlreturned two tangents for four arguments and passed only because the varargs were untrackedInts that hit the_add_to_deriv!(d1, d2) = nothingfallback. They now return the missingNoTangent()s. That was a latent wrong-gradient bug, not a regression from this PR.The stricter length check is a behaviour change for rules that return short tangent tuples: they now error instead of being silently broadcast.
@gradis not public API, but it is used downstream, so I checked the@gradrules in the registered packages GitHub code search finds using it (Tullio, DiffEqBase, NonlinearSolveBase, SimpleNonlinearSolve, Bijectors, SphericalFunctions, ADEV): all of them return one tangent per argument.@gradwith qualified names (#318) is out of scope here and left for a follow-up that moves@gradto the same approach.Tests
New cases in
test/ChainRulesTests.jl:nothingfrom@gradwhereclauses in six shapes, with@test_nowarnfor the unused type variable warningsDoubled(x::TrackedArray)), including a second constructor with the same signature (no method overwrite), a@gradconstructor, and that constructors of other types are unaffected🤖 Generated with Claude Code