Skip to content

Ignore an undeclared extra, as pip and uv do - #63

Merged
jonyoder merged 2 commits into
mainfrom
fix/20988-undeclared-extra
Sep 25, 2026
Merged

jonyoder merged 2 commits into
mainfrom
fix/20988-undeclared-extra

Conversation

@jonyoder

@jonyoder jonyoder commented Sep 25, 2026 •

Copy link
Copy Markdown
Collaborator

An extra a version does not declare is now ignored, as pip and uv do, instead of excluding that version. It's reported as structured data (Resolution.MissingExtras) so each caller decides how to show it.

Refs rstudio/package-manager#20988


Decided by Jon 2026-09-25: one solving rule for every caller, no strict/lenient option. This reverses go-pyresolver's deliberate "an undeclared extra must fail loudly" rule from dc2dae5 (#30): a misspelled pkg[tests] no longer fails; the warning is what keeps it visible.

What changed

  • provider/dependencies.go: the extra branch of dependenciesFrom keeps the version selectable when the extra is undeclared, adds no extra-only requirements, and records the requester → package[extra] edge plus the undeclared fact. The requester is recorded as-is (not reduced to its base name), so a request made by base[extraA] is never mistaken for one made by plain base.
  • resolver/resolver.go: Resolve filters those recordings against the final solution (not everything the search merely visited and later backtracked past) to build Resolution.MissingExtras, and drops an undeclared extra from Resolution.Extras (which promises "what a caller needs to reproduce the same install"). When the requester is itself an extra, its own extra must also still be active in the final solution — a name+version match on the base alone would let a phantom warning through if base[extraA] were backtracked away while plain base stayed pinned at the same version.
  • resolver/satoracle_test.go: the SAT oracle's requirementFormula now treats an undeclared extra as satisfied by any admissible base version, with no extra variable and no extra-only clauses.
  • resolver/packse_test.go: extras/missing-extra and extras/extra-does-not-exist-backtrack are removed from knownFail and now pass as regular scenarios; missing-extra additionally asserts MissingExtras names a[extra], requested by the root.

API addition

New exported types in resolver: MissingExtra (Package, Version, Extra, RequestedBy) and Requester (Root, Package, Version), plus the field Resolution.MissingExtras []MissingExtra. CLAUDE.md's exported-surface line is updated to name them.

What pip and uv do (verified against upstream source)

  • pip (pypa/pip main a7002c9, src/pip/_internal/resolution/resolvelib/candidates.py:535-549): logs a warning ("%s %s does not provide the extra '%s'") and yields dependencies for valid_extras only. Does not backtrack.
  • uv (astral-sh/uv main 30de9e2): the MissingExtra diagnostic (crates/uv-distribution-types/src/resolution.rs:145) is shown as a warning. crates/uv/tests/pip/pip_install_scenarios.rs:621-626 installs a==3.0.0 for the backtrack scenario, :822-827 installs a==1.0.0 for missing-extra.

Review follow-up: an extra-requests-extra chain could leak a phantom warning

A reviewer flagged that recordExtraRequests(Project(pkg.Name), v, expanded) dropped pkg's own extra when pkg was itself an extra node, so a request from base[extraA] to other[extraB] was attributed only to plain base. missingExtras only checked that base was pinned at that version, not that extraA survived into res.Extras[base]. If base stayed pinned while base[extraA] was backtracked away, the warning could leak.

Reproduced it with TestResolveDoesNotReportAMissingExtraFromAnAbandonedExtraOfAPinnedBase: x 2.0 asks for z[declared] (a real extra of z), which asks for other[missing] (undeclared); a diamond conflict introduced by a low-priority y backjumps past z[declared] after other[missing] has already been decided, so x reverts to 1.0 while z and other stay pinned at the same versions via their own separate root requirements. Before the fix this reports a phantom MissingExtra{other, missing, RequestedBy: z}. Fixed by recording the requester as-is (pkg instead of Project(pkg.Name)) and requiring the requester's own extra, when it has one, to still be in res.Extras for the final solution.

Mutation proof: reverting just this change (provider/dependencies.go's recordExtraRequests call and resolver/resolver.go's extra check) turns the new test RED; restoring it turns it GREEN.

Verification

  • TestPackse: 147 scenarios = 94 pass (+2 from 92) + 2 known-fail (-2 from 4) + 6 divergence + 4 unsupported + 41 out-of-scope.
  • TestSATOracle: 102 scenarios cross-checked, 8 disagreements, down from 9 before this PR. extras/missing-extra was one of the 9 (packse said sat, resolver and oracle said unsat) and now agrees across packse, resolver, and oracle. The remaining 8 are unrelated (prereleases, requires_python, yanked).
  • TestResolutionTranscriptMatchesGolden: unaffected — no pinned version in the corpus moved.
  • 5 mutation proofs for the base fix (each: RED before restore, GREEN after) covering: the old early-return, activating the undeclared extra's marker-matched requirements, building MissingExtras from every recorded edge without final-solution filtering, a nil MissingExtras, and reverting the SAT oracle's extras model. Plus 1 more for the review follow-up above.
  • go test -count=1 ./... and go test -race -count=1 ./... both green. golangci-lint run ./... (v2.11.2, config verify) 0 issues. gofmt -l . clean.

PPM follow-up (separate work, tracked by #20988's tracking issue)

  • Bump the go-pyresolver pin in go.mod (currently v0.11.0).
  • Add a get pypi test with a transitive extra that the newest version dropped.
  • Show Resolution.MissingExtras to users in the offline downloader output.

🤖 Generated with Claude Code

jonyoder and others added 2 commits September 25, 2026 12:00
Refs rstudio/package-manager#20988. An extra a version does not declare
no longer excludes that version (dc2dae5 / #30's rule) -- it is ignored,
like pip and uv, and reported as data in the new Resolution.MissingExtras
(MissingExtra, Requester) so each caller decides how to show it.

- provider: the extra branch of dependenciesFrom keeps the version
  selectable and adds no extra-only requirements; records the
  requester -> package[extra] edge and the undeclared fact.
- resolver: Resolve filters those against the final solution (not what
  the search merely encountered) to build MissingExtras, and drops an
  undeclared extra from Resolution.Extras.
- SAT oracle and packse: extras/missing-extra and
  extras/extra-does-not-exist-backtrack now pass as regular scenarios.
- CLAUDE.md: MissingExtra and Requester join the package's exported
  surface.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
base[extraA] asking for other[extraB] was recorded as a request from
plain base, dropping extraA. If base stayed pinned while base[extraA]
was abandoned, missingExtras could report a phantom MissingExtra for
other[extraB]. Record the requester as-is and require its own extra to
still be active in the final solution.

Refs rstudio/package-manager#20988

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@jonyoder
jonyoder merged commit e73b0a9 into main Sep 25, 2026
2 checks passed
@jonyoder
jonyoder deleted the fix/20988-undeclared-extra branch September 25, 2026 16:58
@jonyoder jonyoder mentioned this pull request Sep 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant