Skip to content

Fall back to a pre-release when no final release is in range - #62

Merged
jonyoder merged 1 commit into
mainfrom
fix/20987-prerelease-fallback
Sep 25, 2026
Merged

jonyoder merged 1 commit into
mainfrom
fix/20987-prerelease-fallback

Conversation

@jonyoder

@jonyoder jonyoder commented Sep 25, 2026 •

Copy link
Copy Markdown
Collaborator

When no final release satisfies a requirement's range, offer a pre-release instead of failing — matching pip and current uv, not packse's older per-package rule. Six packse scenarios are reclassified as intentional divergence rather than fixed to match packse.


Decision (Jon, 2026-09-25)

Use the pip / current-uv rule: in-range fallback, not packse's per-package rule ("only when the package publishes no final release at all"). Where packse's expected outcome follows the old rule, this matches pip and records the scenario as an intentional divergence instead.

Reversal: to go back to packse's per-package rule, recompute admission per-package from the full version list, and move the five pre-release divergence entries back onto the passing set. requires_python/python-less-than-current stays on the divergence list either way.

Shape chosen, and why it's safe

Ranking, not admission. Pre-releases stay in the candidate universe for every package. provider.rankedVersions stably moves a non-enabled package's pre-releases after its finals (finalsFirst); Candidates no longer skips them. The enabled set (candidate.PrereleaseSet) is still computed once, before solving, and never moves — so go-pubgrub's cached derivations stay valid. The alternative (recomputing admission from the solver's current range) was rejected for exactly this reason: it would make admission depend on search state, which the EnabledPrereleases/prerelease.rs comments both forbid.

Rank rule. rank counts finals in range when any exist, and pre-releases only when none do — never their sum. Otherwise every package with pre-releases would report a larger rank for a reason unrelated to this fix, and could shift the solver's package order. TestResolutionTranscriptMatchesGolden's golden did not move for that reason (see below for why it moved at all).

Exported symbols: unchanged. candidate.PrereleaseSet.Admits and EnabledPrereleases keep their signatures; only their doc comments changed (they now decide ranking, not existence). Callers: provider/provider.go, resolver/resolver.go, plus test callers resolver/satoracle_test.go (oracleAdmissible, which now drops the pre-release check entirely), candidate/prerelease_test.go, provider/provider_test.go. PPM has zero callers of go-pyresolver/candidate. Not Breaking.

Conflict case (Jon, 2026-09-25: follow uv, not pip)

b has 1.0 (depends on a missing package) and 2.0rc1; root is plain b. pip's specifier-only admission sees a final in range and fails outright. uv, and this resolver, admit the pre-release once the conflict rules 1.0 out of the derived range. Test 7 pins b==2.0rc1. No packse scenario is affected — the 20 prereleases/ scenarios all exclude their final by a specifier, never by a conflict.

The same fallback fires when every final in range is unusable (no matching wheel, Requires-Python mismatch, marker exclusion), not only on a conflict. uv's PreferStable does the same; pip fails there.

Known remaining gap, not implemented: a transitive specifier that itself names a pre-release (e.g. >=1.0rc1) turns on pre-releases in pip via a proxy package (uv #19993); this resolver only enables a package from a root specifier. No packse scenario covers it.

Classification after the change

147 = 92 pass + 4 known-fail + 6 divergence + 4 unsupported + 41 out-of-scope (measured; was 94 pass + 8 known-fail + 4 unsupported + 41 out-of-scope).

Scenario packse expects This change List
package-only-prereleases a=1.0.0a1 a=1.0.0a1 knownFail → pass
package-only-prereleases-boundary a=0.1.0a1 a=0.1.0a1 knownFail → pass
transitive-package-only-prereleases a=0.1.0, b=1.0.0a1 same knownFail → pass
package-only-prereleases-in-range unsatisfiable a=1.0.0a1 pass → divergence
transitive-package-only-prereleases-in-range unsatisfiable a=0.1.0, b=1.0.0a1 pass → divergence
transitive-prerelease-and-stable-dependency unsatisfiable a=1.0.0, b=1.0.0, c=2.0.0b1 pass → divergence
transitive-...-many-versions unsatisfiable c=2.0.0b9 pass → divergence
transitive-...-many-versions-holes unsatisfiable c=2.0.0b4 pass → divergence
requires_python/python-less-than-current sat unsat knownFail → divergence

Confirmed with packaging 26.3's SpecifierSet(...).filter(versions, prereleases=None) for each divergent package's exact range:

package-only-prereleases-in-range: a>0.1.0 over [0.1.0, 1.0.0a1] -> ['1.0.0a1']
transitive-package-only-prereleases-in-range: b>0.1 over [0.1.0, 1.0.0a1] -> ['1.0.0a1']
transitive-prerelease-and-stable-dependency: c==2.0.0b1,>=1.0.0,<=3.0.0 -> ['2.0.0b1']
transitive-...-many-versions: c>=2.0.0b1,>=1.0.0,<=3.0.0 -> [...,'2.0.0b9'] highest 2.0.0b9
transitive-...-many-versions-holes: c>1.0.0,!=2.0.0a5,!=2.0.0a6,!=2.0.0a7,!=2.0.0b1,<2.0.0b5,>=1.0.0,<=3.0.0 -> [...,'2.0.0b4'] highest 2.0.0b4

resolver/packse_test.go's new intentionalDivergence map asserts pip's exact outcome (full pin map, or nil for *resolver.ResolutionError) per scenario, not merely "disagrees with packse" — an entry that starts matching packse fails loudly. resolver/testdata/packse/README.md documents the list.

Test changes

  • TestResolveHonoursAllowPrerelease rewritten. It used to assert that an unopted-in pre-release-only package fails to resolve — that half of the old assertion is exactly what this change removes. flask now has both a final (1.0) and a pre-release (2.0rc1); without opt-in the final wins, with opt-in the pre-release does.
  • 7 new unit tests in resolver/resolver_test.go, each asserting the full pin set or a *resolver.ResolutionError, covering: pre-release-only packages, dev releases, the in-range fallback itself, a final winning over an in-range pre-release, a post-release counting as final, a backtracking case where a newer version now resolves via a transitive pre-release fallback, and the conflict case above.
  • TestResolutionTranscriptMatchesGolden's golden changed and was regenerated. Confirmed by hand: sphinx-basic-ng, black, and pytest-black in the excerpt fixture publish only pre-release versions matching their constraints, so they used to fail outright ("has no versions" / backtrack) and now resolve via the fallback (e.g. sphinx-basic-ng 1.0.0b2), which in turn lets furo resolve to a newer version. pandas/anpy/nbformat still fail overall but with a longer derivation, because the solver now tries a pre-release fallback deeper in their graphs (e.g. python-dateutil) before giving up. This is the same class of change the trap in the brief anticipated ("some resolutions that used to fail or backtrack now pick a pre-release or newer version"), not a regression.

CHANGELOG

Added one ### Fixed entry under [Unreleased]. No PPM NEWS.md entry (sibling repo); the PPM-side entry belongs to the later pin-bump unit.

Follow-ups

  • #20988 (undeclared extras) edits the same knownFail map and must rebase onto this PR's merge commit.
  • PPM #20885 part 2 (separate, later unit): bump the go-pyresolver pin and add GetPyPISuite tests for a pre-release-only dependency, once this ships in a tagged release.

Refs rstudio/package-manager#20987

🤖 Generated with Claude Code

pip and current uv (astral-sh/uv#19993) offer a package's pre-release
only when no final release satisfies the range, not merely when the
package publishes no final at all. Implement this as ranking rather
than admission: rankedVersions moves a non-enabled package's
pre-releases after its finals, so the enabled set stays fixed for
go-pubgrub's cached derivations. Candidates no longer skips
pre-releases; rank counts finals when any are in range, else
pre-releases, to keep today's numbers where a final exists.

Reclassify requires_python/python-less-than-current and the five
prereleases/ scenarios where packse still encodes uv's pre-#19993
rule onto a new intentionalDivergence list, asserted against pip's
exact outcome. The SAT oracle drops the pre-release check from its
own admission, since every version is now admissible for existence.

Refs rstudio/package-manager#20987

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@jonyoder
jonyoder merged commit 91e3620 into main Sep 25, 2026
2 checks passed
@jonyoder
jonyoder deleted the fix/20987-prerelease-fallback branch September 25, 2026 15:34
@jonyoder jonyoder mentioned this pull request Sep 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant