diff --git a/CHANGELOG.md b/CHANGELOG.md index 4efd84c..a8eace4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,12 @@ served it. ## [Unreleased] +### Fixed + +- **When no final release satisfies a requirement's range, a pre-release is now chosen, as pip + and uv do.** A final release in range is still preferred. Some resolutions that used to fail or + backtrack now pick a pre-release or a newer version instead. + ## [0.11.0] - 2026-09-18 ### Breaking diff --git a/candidate/doc.go b/candidate/doc.go index 852f2db..ee9e9b9 100644 --- a/candidate/doc.go +++ b/candidate/doc.go @@ -6,12 +6,16 @@ // Two separable things live here, and keeping them separate is the point: // // - Admission is a yes/no about a single version, decided once before -// solving so it cannot move while the solver backtracks. Today that is the -// pre-release rule: see PrereleaseSet. +// solving so it cannot move while the solver backtracks. // - Ranking is a total order over the admissible versions, supplied by the // caller through Policy so that an embedder -- Package Manager, say -- can // demote versions it would rather not install without making them -// unavailable. Newest is the default. +// unavailable. Newest is the default. The pre-release rule lives here too: +// see PrereleaseSet. A package's pre-releases rank alongside its finals if +// the caller enabled it, and after every final release otherwise, so a +// pre-release is chosen only when nothing final in range is usable -- the +// pip/uv in-range fallback. Nothing is ever inadmissible for being a +// pre-release. // // # Ranking must never remove a version // diff --git a/candidate/prerelease.go b/candidate/prerelease.go index a601eef..67866af 100644 --- a/candidate/prerelease.go +++ b/candidate/prerelease.go @@ -8,10 +8,11 @@ import ( "github.com/posit-dev/go-python-packaging/version" ) -// PrereleaseSet records which packages may have pre-release versions offered. +// PrereleaseSet records which packages rank their pre-releases alongside +// their final releases, rather than after all of them. // // Keys are PEP 503-canonical names. A package absent from the set, or present -// with a false value, gets final releases only. +// with a false value, still offers its pre-releases -- only after its finals. type PrereleaseSet map[index.PackageName]bool // EnabledPrereleases derives the set from the requirements a resolution starts @@ -23,11 +24,18 @@ type PrereleaseSet map[index.PackageName]bool // canonicalized here; allow is assumed to hold values already built with // index.NewPackageName. // +// For an enabled package, its pre-releases are ranked right alongside its +// final releases by Policy. For a package that is not enabled, Admits still +// says every version exists (see below); the caller ranks its pre-releases +// after every final release instead, so a pre-release is only ever chosen +// there when nothing final is usable. That in-range fallback matches pip and +// current uv. +// // The set is computed ONCE, before solving, and must not be recomputed as the -// solver narrows a package's allowed range. That is what makes pre-release -// admission a fact about a version rather than a fact about the current search -// state, and go-pubgrub caches derivations on the assumption that the facts -// behind them do not move. +// solver narrows a package's allowed range. That is what keeps a version's +// pre-release status a fact about the version rather than a fact about the +// current search state, and go-pubgrub caches derivations on the assumption +// that the facts behind them do not move. // // ⚠️ Detection uses Specifiers.PreReleases, NOT Specifiers.FilterVersions, and // the difference is not stylistic. FilterVersions implements pip's fuller rule, @@ -37,6 +45,8 @@ type PrereleaseSet map[index.PackageName]bool // admit a pre-release under one range and reject it under a wider one, and a // cached incompatibility derived from the earlier answer would then be wrong. // Do not call FilterVersions here, and do not call it from Candidates either. +// The pip/uv in-range fallback described above is implemented as ranking +// instead, precisely so it can react to the range without moving admission. // // Note that "!=1.0a1" does not enable pre-releases even though it names one, // and neither does "==1.*"; both match pypa/packaging's own derivation, which @@ -59,17 +69,15 @@ func EnabledPrereleases(reqs []requirement.Requirement, allow []index.PackageNam return set } -// Admits reports whether v may be offered for pkg. -// -// Every final release is admitted; a pre-release is admitted only for a -// package the set enables. "Pre-release" here is version.IsPreRelease, so a -// development release (2.0.dev1) counts and a post-release (2.0.post1) does -// not -- confirmed against go-python-packaging v0.5.0 rather than assumed. -// -// This is admission, not ranking: a version this method rejects is one the -// resolution genuinely may not use, and the reason does not change while the -// solver runs. Anything that merely makes a version less desirable belongs in -// a Policy, where it cannot make the version look nonexistent. +// Admits now decides ORDERING, not admission: every version of pkg is +// admissible for existence, whatever this returns. What it decides is +// whether v should be ranked no worse than pkg's final releases. A final +// release always is; a pre-release is only for a package the set enables -- +// everyone else's pre-releases get ranked after every final release, so one +// is picked only when nothing final is usable (see EnabledPrereleases). +// "Pre-release" here is version.IsPreRelease, so a development release +// (2.0.dev1) counts and a post-release (2.0.post1) does not -- confirmed +// against go-python-packaging v0.5.0 rather than assumed. func (s PrereleaseSet) Admits(pkg index.PackageName, v version.Version) bool { if !v.IsPreRelease() { return true diff --git a/provider/export_test.go b/provider/export_test.go index 1aa53a1..553ad84 100644 --- a/provider/export_test.go +++ b/provider/export_test.go @@ -73,25 +73,58 @@ func (p *Provider) ExactCandidates(pkg Package, allowed pep440set.Set) (pep440se return pep440set.Empty(), false, 0, err } - admissible := make([]version.Version, 0, len(all)) + // Admission is range alone now -- nothing is excluded for being a + // pre-release, so every in-range version is tested for usability. + // finalsInRange is the SAME raw count Candidates uses for rank, kept + // separately so the count returned below can mirror it exactly rather + // than merely bound it. + inRange := make([]version.Version, 0, len(all)) + finalsInRange := 0 for _, v := range all { - if !allowed.Contains(v) || !p.opts.Prereleases.Admits(pkg.Name, v) { + if !allowed.Contains(v) { continue } + inRange = append(inRange, v) + if !v.IsPreRelease() { + finalsInRange++ + } + } + + ranked := finalsFirst(pkg.Name, rankBySortRef(pkg.Name, inRange, p.opts.Policy), p.opts.Prereleases) + + var ( + best version.Version + found bool + finalsUsable, preUsable int + ) + for _, v := range ranked { ok, err := p.usable(pkg, v) if err != nil { return pep440set.Empty(), false, 0, err } - if ok { - admissible = append(admissible, v) + if !ok { + continue + } + if v.IsPreRelease() { + preUsable++ + } else { + finalsUsable++ + } + if !found { + best, found = v, true } } - if len(admissible) == 0 { + if !found { return pep440set.Empty(), false, 0, nil } - - ranked := rankBySortRef(pkg.Name, admissible, p.opts.Policy) - return pep440set.Exactly(ranked[0]), true, len(ranked), nil + // Mirrors Candidates' own rank rule (provider.go) exactly, not merely an + // upper bound of it: finalsInRange (raw) here equals the count Candidates + // itself computes, so finalsUsable/preUsable can never exceed it. + count := finalsUsable + if finalsInRange == 0 { + count = preUsable + } + return pep440set.Exactly(best), true, count, nil } // InRangeRanked is the ranked in-range version list Candidates walks, before any @@ -109,10 +142,10 @@ func (p *Provider) InRangeRanked(pkg Package, allowed pep440set.Set) ([]version. inRange := make([]version.Version, 0, len(all)) for _, v := range all { - if !allowed.Contains(v) || !p.opts.Prereleases.Admits(pkg.Name, v) { - continue + if allowed.Contains(v) { + inRange = append(inRange, v) } - inRange = append(inRange, v) } - return rankBySortRef(pkg.Name, inRange, p.opts.Policy), nil + ranked := rankBySortRef(pkg.Name, inRange, p.opts.Policy) + return finalsFirst(pkg.Name, ranked, p.opts.Prereleases), nil } diff --git a/provider/provider.go b/provider/provider.go index b981752..41cf846 100644 --- a/provider/provider.go +++ b/provider/provider.go @@ -251,20 +251,23 @@ func (p *Provider) Candidates(pkg Package, allowed pep440set.Set) (pep440set.Set // a weakening of the rule the error path exists for: nothing is being reported as // unavailable on the strength of an outage. It is simply not being looked at. var ( - best version.Version - found bool - inRange int + best version.Version + found bool + finalsInRange, prereleaseInRange int ) for _, v := range ranked { - // The cheap half of admission: range and pre-release policy, no metadata - // and no I/O. What passes both is what rank counts. + // The cheap half of admission is range alone now: rankedVersions already + // ordered pre-releases after finals for a package that is not enabled, so + // nothing needs skipping here -- the fallback is a ranking effect, not an + // admission one. See candidate.PrereleaseSet. if !allowed.Contains(v) { continue } - if !p.opts.Prereleases.Admits(pkg.Name, v) { - continue + if v.IsPreRelease() { + prereleaseInRange++ + } else { + finalsInRange++ } - inRange++ if found { // best is settled; the rest of the walk only counts, which costs no // metadata read. @@ -285,7 +288,18 @@ func (p *Provider) Candidates(pkg Package, allowed pep440set.Set) (pep440set.Set // everything. return pep440set.Empty(), false, 0, nil } - return pep440set.Exactly(best), true, inRange, nil + // rank counts finals only when there are any, and pre-releases otherwise -- + // NOT their sum -- so a package's rank does not grow just because it also + // publishes pre-releases the walk above would never prefer while a final is + // usable. See the package's pre-release fallback in rankedVersions; without + // this every package with pre-releases would report a larger rank than + // before this change, for a reason unrelated to it, and could shift the + // solver's package order (TestResolutionTranscriptMatchesGolden). + rank := finalsInRange + if rank == 0 { + rank = prereleaseInRange + } + return pep440set.Exactly(best), true, rank, nil } // rankedVersions returns pkg's full published version list in Policy order, @@ -415,10 +429,32 @@ func (p *Provider) rankedVersions(pkg Package) ([]version.Version, error) { // is unchanged. provider/differential_test.go checks that against the exact // reference rather than leaving it as an argument. r := candidate.Rank(pkg.Name, all, p.opts.Policy) + r = finalsFirst(pkg.Name, r, p.opts.Prereleases) p.ranked[pkg.Name] = r return r, nil } +// finalsFirst stably moves the pre-releases of a package that is not enabled +// to the end of an already-ranked list, leaving everything else untouched. +// +// This is the pip/uv in-range fallback (candidate.PrereleaseSet's doc +// comment), implemented as ranking rather than admission so the enabled set +// can stay fixed for the whole resolution: candidate.Rank is a stable sort, +// so this partition preserves Policy order within each half, and for an +// enabled package (Admits true for every version) it is a no-op. +func finalsFirst(pkg index.PackageName, ranked []version.Version, prereleases candidate.PrereleaseSet) []version.Version { + out := make([]version.Version, 0, len(ranked)) + var after []version.Version + for _, v := range ranked { + if prereleases.Admits(pkg, v) { + out = append(out, v) + } else { + after = append(after, v) + } + } + return append(out, after...) +} + // singleVersion answers for a package with exactly one version and no index // behind it: the root and the interpreter. func singleVersion(v version.Version, allowed pep440set.Set) (pep440set.Set, bool, int, error) { diff --git a/provider/provider_test.go b/provider/provider_test.go index 432209c..a3ade72 100644 --- a/provider/provider_test.go +++ b/provider/provider_test.go @@ -161,8 +161,8 @@ func TestCandidatesPrereleaseAdmission(t *testing.T) { t.Fatal("found = false, want true") } if rank != 1 { - t.Errorf("rank = %d, want 1 (the release candidate is not admissible, and "+ - "pre-release admission is part of the in-range filter rank counts)", rank) + t.Errorf("rank = %d, want 1 (rank counts finals only when any are in range, "+ + "even though the release candidate is still in range as a fallback)", rank) } if got := bestVersion(t, best); got.String() != "1.0" { t.Errorf("best = %s, want 1.0", got) @@ -181,8 +181,9 @@ func TestCandidatesPrereleaseAdmission(t *testing.T) { if !found { t.Fatal("found = false, want true") } - if rank != 2 { - t.Errorf("rank = %d, want 2", rank) + if rank != 1 { + t.Errorf("rank = %d, want 1 (finals-in-range count, per the rank rule -- "+ + "the pre-release still wins the ranking, via Policy, not via rank)", rank) } if got := bestVersion(t, best); got.String() != "2.0rc1" { t.Errorf("best = %s, want 2.0rc1", got) diff --git a/resolver/packse_scenario_test.go b/resolver/packse_scenario_test.go index f355404..dec053a 100644 --- a/resolver/packse_scenario_test.go +++ b/resolver/packse_scenario_test.go @@ -89,8 +89,8 @@ type tomlResolverOptions struct { // packseScenario is one loaded scenario, named by its path relative to // testdata/packse (e.g. "requires_python/python-less-than-current"), which is -// what the classification lists (outOfScope, knownFail, unsupportedOption) key -// on. +// what the classification lists (outOfScope, knownFail, intentionalDivergence, +// unsupportedOption) key on. type packseScenario struct { relName string // category/scenario-name, no extension path string diff --git a/resolver/packse_test.go b/resolver/packse_test.go index 370c37f..23dd5af 100644 --- a/resolver/packse_test.go +++ b/resolver/packse_test.go @@ -94,21 +94,66 @@ var knownFail = map[string]string{ "uv ignores an extra no candidate provides", "extras/extra-does-not-exist-backtrack": "same gap as extras/missing-extra: the newest version (3.0.0) does not " + "provide the extra, so go-pyresolver backtracks to the one that does (1.0.0) instead of dropping the extra", +} - "prereleases/package-only-prereleases": "candidate.PrereleaseSet admits a pre-release only when a specifier " + - "names one or the caller opts in; it does not implement pip/uv's further fallback of admitting one when a " + - "package publishes no final release at all (see PrereleaseSet.Admits's doc comment)", - "prereleases/package-only-prereleases-boundary": "same gap as prereleases/package-only-prereleases", - "prereleases/transitive-package-only-prereleases": "same gap as prereleases/package-only-prereleases, one level " + - "down the dependency graph", +// divergence is one scenario's entry on intentionalDivergence: the reason and +// pip's outcome, asserted exactly rather than merely "not packse". +type divergence struct { + reason string + // pins is the full pin set pip is asserted to reach. Nil means pip fails: + // Resolve must return a *resolver.ResolutionError. + pins map[string]string +} - "requires_python/python-less-than-current": "go-pyresolver's SupportsPython enforces the full Requires-Python " + - "specifier per PEP 440, including an upper bound; uv deliberately ignores an upper bound on Requires-Python", +// intentionalDivergence lists scenarios where go-pyresolver deliberately +// differs from packse's (uv's) expectation because it matches pip. +// +// Ruling 2026-09-24: requires_python/python-less-than-current. pip enforces +// the whole Requires-Python specifier, upper bound included; uv ignores the +// upper bound. Reclassified rather than fixed. +// +// Ruling 2026-09-25: the five prereleases/ scenarios. packse encodes uv's +// admission rule from before astral-sh/uv#19993 ("no final release at all"). +// pip (packaging's SpecifierSet.filter with prereleases=None, per PEP 440) +// and current uv (crates/uv-resolver/src/prerelease.rs, PreferStable) instead +// fall back to a pre-release when nothing final satisfies the RANGE, which is +// what this change implements. Confirmed against packaging 26.3's +// SpecifierSet.filter for each scenario's package (PR description). +var intentionalDivergence = map[string]divergence{ + "requires_python/python-less-than-current": { + reason: "pip enforces the whole Requires-Python specifier, upper bound included, and uv ignores the " + + "upper bound", + pins: nil, // pip: unsatisfiable + }, + "prereleases/package-only-prereleases-in-range": { + reason: "no final release of a is in range (only 0.1.0, excluded by a>0.1.0), so pip and current uv fall " + + "back to the pre-release", + pins: map[string]string{"a": "1.0.0a1"}, + }, + "prereleases/transitive-package-only-prereleases-in-range": { + reason: "same fallback one level down: no final release of b is in range", + pins: map[string]string{"a": "0.1.0", "b": "1.0.0a1"}, + }, + "prereleases/transitive-prerelease-and-stable-dependency": { + reason: "c's range (==2.0.0b1 intersected with >=1.0.0,<=3.0.0) admits no final release at all", + pins: map[string]string{"a": "1.0.0", "b": "1.0.0", "c": "2.0.0b1"}, + }, + "prereleases/transitive-prerelease-and-stable-dependency-many-versions": { + reason: "c's range (>=2.0.0b1) excludes every final and every alpha, so the fallback picks the highest " + + "beta in range", + pins: map[string]string{"a": "1.0.0", "b": "1.0.0", "c": "2.0.0b9"}, + }, + "prereleases/transitive-prerelease-and-stable-dependency-many-versions-holes": { + reason: "c's range excludes the only final and several pre-releases by name; the fallback picks the " + + "highest surviving one", + pins: map[string]string{"a": "1.0.0", "b": "1.0.0", "c": "2.0.0b4"}, + }, } -// runPackseScenario resolves s and reports whether the result matches -// expected. -func runPackseScenario(t *testing.T, s tomlScenario) (matched bool, detail string) { +// resolvePackseScenario runs s against the real resolver and returns the raw +// result, shared by runPackseScenario and the intentionalDivergence check so +// both assert against the SAME resolve rather than running it twice. +func resolvePackseScenario(t *testing.T, s tomlScenario) (*resolver.Resolution, error) { t.Helper() py, err := scenarioPython(s) @@ -140,10 +185,18 @@ func runPackseScenario(t *testing.T, s tomlScenario) (matched bool, detail strin } reqs := mustRequirements(t, s.Root.Requires...) - res, resolveErr := resolver.Resolve(context.Background(), reqs, idx, opts) + return resolver.Resolve(context.Background(), reqs, idx, opts) +} + +// matchOutcome reports whether resolving s produced wantSatisfiable and, when +// satisfiable, exactly wantPackages. +func matchOutcome( + t *testing.T, res *resolver.Resolution, resolveErr error, wantSatisfiable bool, wantPackages map[string]string, +) (matched bool, detail string) { + t.Helper() switch { - case !s.Expected.Satisfiable: + case !wantSatisfiable: if resolveErr == nil { return false, fmt.Sprintf("expected unsatisfiable, but resolved to %v", pins(t, res)) } @@ -155,13 +208,13 @@ func runPackseScenario(t *testing.T, s tomlScenario) (matched bool, detail strin } return true, "" - case len(s.Expected.Packages) > 0: + case len(wantPackages) > 0: if resolveErr != nil { - return false, fmt.Sprintf("expected packages %v, but Resolve failed: %v", s.Expected.Packages, resolveErr) + return false, fmt.Sprintf("expected packages %v, but Resolve failed: %v", wantPackages, resolveErr) } got := pins(t, res) - if !reflect.DeepEqual(got, s.Expected.Packages) { - return false, fmt.Sprintf("Pinned = %v, want %v", got, s.Expected.Packages) + if !reflect.DeepEqual(got, wantPackages) { + return false, fmt.Sprintf("Pinned = %v, want %v", got, wantPackages) } return true, "" @@ -173,6 +226,14 @@ func runPackseScenario(t *testing.T, s tomlScenario) (matched bool, detail strin } } +// runPackseScenario resolves s and reports whether the result matches +// packse's own expected outcome. +func runPackseScenario(t *testing.T, s tomlScenario) (matched bool, detail string) { + t.Helper() + res, resolveErr := resolvePackseScenario(t, s) + return matchOutcome(t, res, resolveErr, s.Expected.Satisfiable, s.Expected.Packages) +} + // TestPackse runs every non-universal vendored packse scenario against the // real resolver.Resolve, classified per testdata/packse/README.md and the // three lists above. See the package doc comment on this file's neighbors for @@ -184,7 +245,7 @@ func TestPackse(t *testing.T) { } seen := make(map[string]bool, len(scenarios)) - var pass, known, unsupported, outScope int + var pass, known, diverged, unsupported, outScope int for _, ps := range scenarios { name := ps.relName @@ -220,7 +281,29 @@ func TestPackse(t *testing.T) { } reason, isKnownFail := knownFail[name] + div, isDivergence := intentionalDivergence[name] + if isKnownFail && isDivergence { + t.Errorf("%s: listed on both knownFail and intentionalDivergence", name) + } + t.Run(name, func(t *testing.T) { + if isDivergence { + // Reuse runPackseScenario's resolve, but assert pip's outcome + // exactly rather than merely "disagrees with packse" -- the latter + // would also pass if the harness itself broke. + res, resolveErr := resolvePackseScenario(t, s) + matched, detail := matchOutcome(t, res, resolveErr, div.pins != nil, div.pins) + if !matched { + t.Errorf("%s: intentional divergence from packse did not match pip's outcome (%s): %s", + name, div.reason, detail) + } + matchedPackse, _ := matchOutcome(t, res, resolveErr, s.Expected.Satisfiable, s.Expected.Packages) + if matchedPackse { + t.Errorf("%s unexpectedly matched packse, remove it from intentionalDivergence", name) + } + return + } + matched, detail := runPackseScenario(t, s) switch { case isKnownFail && matched: @@ -232,9 +315,12 @@ func TestPackse(t *testing.T) { } }) - if isKnownFail { + switch { + case isDivergence: + diverged++ + case isKnownFail: known++ - } else { + default: pass++ } } @@ -244,6 +330,11 @@ func TestPackse(t *testing.T) { t.Errorf("knownFail names %q, which does not exist in testdata/packse", name) } } + for name := range intentionalDivergence { + if !seen[name] { + t.Errorf("intentionalDivergence names %q, which does not exist in testdata/packse", name) + } + } for name := range unsupportedOption { if !seen[name] { t.Errorf("unsupportedOption names %q, which does not exist in testdata/packse", name) @@ -255,6 +346,6 @@ func TestPackse(t *testing.T) { } } - t.Logf("packse: %d scenarios = %d pass + %d known-fail + %d unsupported + %d out-of-scope", - len(scenarios), pass, known, unsupported, outScope) + t.Logf("packse: %d scenarios = %d pass + %d known-fail + %d divergence + %d unsupported + %d out-of-scope", + len(scenarios), pass, known, diverged, unsupported, outScope) } diff --git a/resolver/resolver.go b/resolver/resolver.go index b5f0dca..a89ec27 100644 --- a/resolver/resolver.go +++ b/resolver/resolver.go @@ -59,9 +59,16 @@ type Options struct { // See candidate.Rank. Policy candidate.Policy - // AllowPrerelease names packages whose pre-release versions may be offered - // even though no requirement asked for one. Names must already be - // canonical (build them with index.NewPackageName). + // AllowPrerelease names packages whose pre-releases are ranked alongside + // their final releases, rather than only after every final release in + // range. Names must already be canonical (build them with + // index.NewPackageName). + // + // Every package's pre-releases are still offered as a fallback when no + // final release in range is usable, whether or not it is named here -- + // see candidate.PrereleaseSet. This only affects a package where a final + // IS usable: named here, its newest pre-release can still beat an older + // final; not named, the final wins regardless of version order. // // A package whose own requirement names a pre-release -- ">=2.0rc1" -- is // enabled without being listed here. diff --git a/resolver/resolver_test.go b/resolver/resolver_test.go index dbc424e..5950397 100644 --- a/resolver/resolver_test.go +++ b/resolver/resolver_test.go @@ -301,17 +301,28 @@ func TestResolveHonoursThePolicy(t *testing.T) { } } +// AllowPrerelease no longer decides whether a pre-release exists in range -- +// every package's pre-releases are offered as a fallback when nothing final +// is usable (see the field's doc comment and TestResolveFallsBackTo*). What +// it decides is a pre-release winning a range where a final ALSO usable, so +// flask needs a final release too for the two halves to differ. func TestResolveHonoursAllowPrerelease(t *testing.T) { - idx := index.NewMockIndex("test").AddVersion("flask", "2.0rc1") + idx := index.NewMockIndex("test"). + AddVersion("flask", "1.0"). + AddVersion("flask", "2.0rc1") - // A pre-release nobody asked for is not offered, so this cannot resolve. - if _, err := resolve(t, idx, "flask"); err == nil { - t.Fatal("a pre-release was offered without being asked for") + // Not asked for: the final release wins even though 2.0rc1 is newer. + res, err := resolve(t, idx, "flask") + if err != nil { + t.Fatalf("Resolve: %v", err) + } + if got := pins(t, res)["flask"]; got != "1.0" { + t.Errorf("flask = %q, want 1.0 (not asked for a pre-release, and a final is usable)", got) } opts := testOptions(t) opts.AllowPrerelease = []index.PackageName{index.NewPackageName("flask")} - res, err := resolver.Resolve(context.Background(), mustRequirements(t, "flask"), idx, opts) + res, err = resolver.Resolve(context.Background(), mustRequirements(t, "flask"), idx, opts) if err != nil { t.Fatalf("Resolve with AllowPrerelease: %v", err) } @@ -320,6 +331,132 @@ func TestResolveHonoursAllowPrerelease(t *testing.T) { } } +// TestResolveAdmitsPrereleaseWhenNoFinalExists is unit test 1 of the +// verification bar: the package has never published a final release. +func TestResolveAdmitsPrereleaseWhenNoFinalExists(t *testing.T) { + idx := index.NewMockIndex("test").AddVersion("a", "1.0a1") + + res, err := resolve(t, idx, "a") + if err != nil { + t.Fatalf("Resolve: %v", err) + } + want := map[string]string{"a": "1.0a1"} + if got := pins(t, res); !reflect.DeepEqual(got, want) { + t.Errorf("Pinned = %v, want %v", got, want) + } +} + +// TestResolveAdmitsDevReleaseAsPrerelease is unit test 2: a dev release is a +// pre-release for this purpose, and the same in-range fallback applies to it. +func TestResolveAdmitsDevReleaseAsPrerelease(t *testing.T) { + idx := index.NewMockIndex("test").AddVersion("a", "1.0.dev1") + + res, err := resolve(t, idx, "a") + if err != nil { + t.Fatalf("Resolve: %v", err) + } + want := map[string]string{"a": "1.0.dev1"} + if got := pins(t, res); !reflect.DeepEqual(got, want) { + t.Errorf("Pinned = %v, want %v", got, want) + } +} + +// TestResolveFallsBackToPrereleaseWhenNoFinalInRange is unit test 3: the +// in-range fallback itself. a's only final (0.1.0) is excluded by the root +// specifier, so the pre-release is what pip and current uv would pick. +func TestResolveFallsBackToPrereleaseWhenNoFinalInRange(t *testing.T) { + idx := index.NewMockIndex("test"). + AddVersion("a", "0.1.0"). + AddVersion("a", "1.0a1") + + res, err := resolve(t, idx, "a>0.1.0") + if err != nil { + t.Fatalf("Resolve: %v", err) + } + want := map[string]string{"a": "1.0a1"} + if got := pins(t, res); !reflect.DeepEqual(got, want) { + t.Errorf("Pinned = %v, want %v", got, want) + } +} + +// TestResolveResolvePrefersFinalOverPrereleaseInRange is unit test 4: the same +// two versions, but with a final in range, so it wins over the newer +// pre-release. +func TestResolveResolvePrefersFinalOverPrereleaseInRange(t *testing.T) { + idx := index.NewMockIndex("test"). + AddVersion("a", "0.1.0"). + AddVersion("a", "1.0a1") + + res, err := resolve(t, idx, "a") + if err != nil { + t.Fatalf("Resolve: %v", err) + } + want := map[string]string{"a": "0.1.0"} + if got := pins(t, res); !reflect.DeepEqual(got, want) { + t.Errorf("Pinned = %v, want %v", got, want) + } +} + +// TestResolveTreatsPostReleaseAsFinal is unit test 5: a post-release is not a +// pre-release, so it is preferred even over a newer pre-release. +func TestResolveTreatsPostReleaseAsFinal(t *testing.T) { + idx := index.NewMockIndex("test"). + AddVersion("a", "1.0.post1"). + AddVersion("a", "2.0a1") + + res, err := resolve(t, idx, "a") + if err != nil { + t.Fatalf("Resolve: %v", err) + } + want := map[string]string{"a": "1.0.post1"} + if got := pins(t, res); !reflect.DeepEqual(got, want) { + t.Errorf("Pinned = %v, want %v", got, want) + } +} + +// TestResolvePicksNewerVersionViaPrereleaseFallbackAfterBacktracking is unit +// test 6: the backtracking trap. Today x=2.0's dependency y>1.0 has nothing +// final in range, so the resolver used to reject x=2.0 outright and settle +// for the older x=1.0. Under the fallback, y's pre-release is usable, so x=2.0 +// resolves and is preferred as the newer version. +func TestResolvePicksNewerVersionViaPrereleaseFallbackAfterBacktracking(t *testing.T) { + idx := index.NewMockIndex("test"). + AddVersion("x", "1.0"). + AddVersion("x", "2.0", "y>1.0"). + AddVersion("y", "1.0"). + AddVersion("y", "2.0a1") + + res, err := resolve(t, idx, "x") + if err != nil { + t.Fatalf("Resolve: %v", err) + } + want := map[string]string{"x": "2.0", "y": "2.0a1"} + if got := pins(t, res); !reflect.DeepEqual(got, want) { + t.Errorf("Pinned = %v, want %v", got, want) + } +} + +// TestResolveFallsBackToPrereleaseWhenFinalRuledOutByConflict is unit test 7: +// the conflict case (Jon 2026-09-25: follow uv, not pip). b's only final +// (1.0) depends on a package that does not exist, so once the solver rules it +// out, b's range holds only the pre-release. pip would fail here because its +// admission is decided by the specifier alone, before the conflict is known; +// this resolver's admission is range-based throughout, so it does not. +func TestResolveFallsBackToPrereleaseWhenFinalRuledOutByConflict(t *testing.T) { + idx := index.NewMockIndex("test"). + AddVersion("b", "1.0", "missing"). + AddVersion("b", "2.0rc1") + + res, err := resolve(t, idx, "b") + if err != nil { + t.Fatalf("Resolve: %v", err) + } + want := map[string]string{"b": "2.0rc1"} + if got := pins(t, res); !reflect.DeepEqual(got, want) { + t.Errorf("Pinned = %v, want %v", got, want) + } +} + // MaxRounds is a safety valve, not a tuning knob: go-pubgrub documents that // termination of the outer loop is asserted rather than derived, and // requires_dist is untrusted third-party text. Hitting the bound must fail diff --git a/resolver/satoracle_test.go b/resolver/satoracle_test.go index b0f7310..98fa0c0 100644 --- a/resolver/satoracle_test.go +++ b/resolver/satoracle_test.go @@ -11,7 +11,6 @@ import ( "testing" "github.com/crillab/gophersat/bf" - "github.com/posit-dev/go-pyresolver/candidate" "github.com/posit-dev/go-pyresolver/index" "github.com/posit-dev/go-pyresolver/resolver" "github.com/posit-dev/go-python-packaging/extras" @@ -70,12 +69,13 @@ func exactPinAdmitsYanked(root tomlRoot, pkgName string, v version.Version) bool } // oracleAdmissible computes, independently of provider/candidate, whether v -// may be offered at all: Requires-Python, pre-release admission (sharing -// candidate.PrereleaseSet -- the one deliberate exception named in this -// package's mutation-proof and PR-body notes), and wheel/sdist availability. -func oracleAdmissible( - v tomlVersion, parsed version.Version, py pythonSpec, prereleases candidate.PrereleaseSet, pkgName string, matcher *tags.Matcher, -) bool { +// may be offered at all: Requires-Python and wheel/sdist availability. Since +// the pip/uv in-range fallback made pre-release status a ranking concern +// rather than an admission one, the oracle shares no admission input with the +// solver at all -- every version in range genuinely exists, and pin CHOICE +// among pre-releases is covered by packse's expected.packages and the unit +// tests instead. +func oracleAdmissible(v tomlVersion, py pythonSpec, matcher *tags.Matcher) bool { reqPy := requiresPythonOf(v) if reqPy != "" { specs, err := version.NewSpecifiers(reqPy) @@ -83,9 +83,6 @@ func oracleAdmissible( return false } } - if !prereleases.Admits(index.NewPackageName(pkgName), parsed) { - return false - } facts := versionDistFacts(v) hasCompatibleWheel := false @@ -153,8 +150,6 @@ func extraVar(pkg, extra, ver string) string { return pkg + "[" + extra + "]@" + func buildOracleModel(t *testing.T, s tomlScenario, py pythonSpec, env marker.Environment, matcher *tags.Matcher) oracleModel { t.Helper() - prereleases := candidate.EnabledPrereleases(mustRequirements(t, s.Root.Requires...), oraclePrereleaseAllow(s)) - versions := make(map[string][]oracleVersion, len(s.Packages)) for pkgName, pkg := range s.Packages { name := index.NewPackageName(pkgName).String() @@ -163,7 +158,7 @@ func buildOracleModel(t *testing.T, s tomlScenario, py pythonSpec, env marker.En if err != nil { t.Fatalf("oracle %s: %s %s: bad version: %v", s.Name, pkgName, verStr, err) } - admissible := oracleAdmissible(v, parsed, py, prereleases, name, matcher) && + admissible := oracleAdmissible(v, py, matcher) && (!v.Yanked || exactPinAdmitsYanked(s.Root, pkgName, parsed)) provides := make(map[string]bool, len(v.Extras)) for e := range v.Extras { @@ -307,15 +302,6 @@ func originalPackageName(pkgs map[string]tomlPackage, canonical string) string { return canonical } -// oraclePrereleaseAllow mirrors runPackseScenario's resolver_options.prereleases -// emulation, so the oracle and the resolver share the same admission input. -func oraclePrereleaseAllow(s tomlScenario) []index.PackageName { - if !s.ResolverOptions.Prereleases { - return nil - } - return allPackageNames(s.Packages) -} - // resolverModel builds the variable assignment the CNF sees for a resolved // Resolution: true for each pinned (package, version) and each active // (package, extra) at that version, false for every other variable the model diff --git a/resolver/testdata/excerpt-transcript.txt b/resolver/testdata/excerpt-transcript.txt index 249a55e..bc7f446 100644 --- a/resolver/testdata/excerpt-transcript.txt +++ b/resolver/testdata/excerpt-transcript.txt @@ -26,7 +26,7 @@ FAILED lines=1 bytes=138 sha256=bdb2a7f303e226247fad9359fc58833fd06c1e4e9c1c558b FAILED lines=1 bytes=132 sha256=68808372122aea8b86fc4dbe781ff0d1b07007341cdd62df072e19864f7e1290 So, because boto3 has no versions and the root project 0 depends on boto3, the requirements of the root project cannot be satisfied. === pandas numpy<1.26 -FAILED lines=64 bytes=70868 sha256=21fce5c55ba1edc1d186a76b0c42ebcf92630272f4d66ce15bd05b9190b27c75 +FAILED lines=67 bytes=77610 sha256=6ee4b8ac9b2aabf3debc45a58fe4a88ecee5694e3c7100a2cb6950d55271b304 So, because the root project 0 depends on pandas, the requirements of the root project cannot be satisfied. === flask==3.0.0 werkzeug<2 FAILED lines=1 bytes=199 sha256=a44ce1c47cbfec0b638b6a931965d36e95f471701f1beeece7db31df56997725 @@ -45,7 +45,7 @@ FAILED lines=3 bytes=491 sha256=e9701a4005e555d9913c48f7d744b2acd80dec5b3c287713 === alabaster alabaster 1.0.0 === anpy -FAILED lines=16 bytes=4779 sha256=84c1c632100cffc0efd34165d5f208e8ebe345f3f8391acc18f1097f675e51ce +FAILED lines=21 bytes=7598 sha256=65ee7b2150b764a53126d382504f4840730dc179c4315a18a64af64c31169edc So, because the root project 0 depends on anpy, the requirements of the root project cannot be satisfied. === argcomplete argcomplete 3.7.0 @@ -69,8 +69,9 @@ FAILED lines=16 bytes=4779 sha256=84c1c632100cffc0efd34165d5f208e8ebe345f3f8391a FAILED lines=1 bytes=132 sha256=6f4f5d4870177817817afad0af4dbf5ea73d726cba32f4693b5ac37c6a18c959 So, because big-o has no versions and the root project 0 depends on big-o, the requirements of the root project cannot be satisfied. === black -FAILED lines=38 bytes=25043 sha256=b0c9dbc88bc1b51ec1067e435f3d652e775d065d36bdd53624f98d08d1cf0f6d - So, because the root project 0 depends on black, the requirements of the root project cannot be satisfied. + black 18.4a2 + attrs 26.1.0 + click 8.4.2 === blinker blinker 1.9.0 === build @@ -106,24 +107,25 @@ FAILED lines=1 bytes=132 sha256=f3d616b64744cbd97e2ee8c30a0baba977c83de8bf604e38 FAILED lines=32 bytes=18398 sha256=7367984a4818bbf6f732559501d430b8f009f826afd979b9a1352eefc8e52f27 So, because the root project 0 depends on freezegun, the requirements of the root project cannot be satisfied. === furo - furo 2022.4.7 + furo 2023.3.27 beautifulsoup4 4.6.3 - sphinx 4.5.0 + sphinx 5.3.0 sphinxcontrib-jsmath 1.0.1 + imagesize 2.0.0 sphinxcontrib-serializinghtml 2.0.0 - docutils 0.17.1 sphinxcontrib-htmlhelp 2.1.0 sphinxcontrib-devhelp 2.0.0 snowballstemmer 3.1.1 sphinxcontrib-qthelp 2.0.0 sphinxcontrib-applehelp 2.0.0 - imagesize 2.0.0 - alabaster 0.7.16 - pygments 2.20.0 + docutils 0.19 jinja2 3.1.6 markupsafe 3.0.3 - babel 2.18.0 + alabaster 0.7.16 + sphinx-basic-ng 1.0.0b2 + pygments 2.20.0 packaging 26.2 + babel 2.18.0 requests 2.15.1 === guessproj FAILED lines=5 bytes=665 sha256=f0d7da273a9d74e0c5176a7fcd0e399ac6758d572e4494ef3c0d9d8b4d4e51f0 @@ -204,7 +206,7 @@ FAILED lines=13 bytes=3282 sha256=1f62b44e2912d2e38246af87304e9ba4be2975d1ac3b44 === mypy-extensions mypy-extensions 1.1.0 === nbformat -FAILED lines=38 bytes=25829 sha256=e974aeaa8b903e9d1f698f68b0fd9f8c9ea16b74881cd6bde36b9c9d15245263 +FAILED lines=42 bytes=31449 sha256=d834cdb84840a5f6c9e86cb98f72818f6741eb677de80531c95c6568bd3362ee So, because the root project 0 depends on nbformat, the requirements of the root project cannot be satisfied. === numpy numpy 2.4.6 @@ -258,8 +260,16 @@ FAILED lines=12 bytes=3118 sha256=9bc4fbce121329031943edcd4ac1f7f5aa0a6ed6101de5 packaging 26.2 pygments 2.20.0 === pytest-black -FAILED lines=41 bytes=24791 sha256=7cc48491e26c34c81d9b63e97cc72037f97da9311d8432c9ebd9fefaf9541997 - So, because the root project 0 depends on pytest-black, the requirements of the root project cannot be satisfied. + pytest-black 0.6.0 + toml 0.10.2 + black 18.4a2 + attrs 26.1.0 + pytest 9.1.1 + pluggy 1.6.0 + iniconfig 2.3.0 + packaging 26.2 + pygments 2.20.0 + click 8.4.2 === pytest-checkdocs pytest-checkdocs 2.3.0 docutils 0.23 @@ -351,8 +361,24 @@ FAILED lines=41 bytes=24791 sha256=7cc48491e26c34c81d9b63e97cc72037f97da9311d843 === snowballstemmer snowballstemmer 3.1.1 === sphinx-basic-ng -FAILED lines=1 bytes=152 sha256=d7ea3171df0fd329806d92545e4ccff06f70b2613d3c48d1ad79006cf1826dfd - So, because sphinx-basic-ng has no versions and the root project 0 depends on sphinx-basic-ng, the requirements of the root project cannot be satisfied. + sphinx-basic-ng 1.0.0b2 + sphinx 5.3.0 + sphinxcontrib-jsmath 1.0.1 + imagesize 2.0.0 + sphinxcontrib-serializinghtml 2.0.0 + sphinxcontrib-htmlhelp 2.1.0 + sphinxcontrib-devhelp 2.0.0 + snowballstemmer 3.1.1 + sphinxcontrib-qthelp 2.0.0 + sphinxcontrib-applehelp 2.0.0 + docutils 0.19 + jinja2 3.1.6 + markupsafe 3.0.3 + alabaster 0.7.16 + pygments 2.20.0 + packaging 26.2 + babel 2.18.0 + requests 2.15.1 === sphinx-lint sphinx-lint 1.0.2 polib 1.2.0 diff --git a/resolver/testdata/packse/README.md b/resolver/testdata/packse/README.md index 0980563..8e467a4 100644 --- a/resolver/testdata/packse/README.md +++ b/resolver/testdata/packse/README.md @@ -30,6 +30,21 @@ resolves one concrete marker environment at a time (see resolution is deferred by RFD 0001. `resolver/packse_test.go`'s `outOfScope` list names them. +## Intentional divergence + +Six scenarios run against the real resolver but are asserted against pip's outcome instead of +packse's `expected` block, because go-pyresolver deliberately matches pip over packse (uv). +`resolver/packse_test.go`'s `intentionalDivergence` list names them and the pip outcome each one +must produce. + +- `requires_python/python-less-than-current`: pip enforces the whole `Requires-Python` specifier, + upper bound included; uv ignores the upper bound. +- The five `prereleases/*` scenarios where packse expects "unsatisfiable" because no final release + is in range: pip (`packaging.specifiers.SpecifierSet.filter`, PEP 440's own recommendation) and + current uv (astral-sh/uv#19993, "Support transitive pre-release dependencies") both fall back to + a pre-release instead of failing. packse's `prereleases/` scenarios were last touched before that + uv change, so they still encode the older, per-package rule. + ## How to refresh Bump the pinned commit above, re-run the copy command (all 147 files, so a