Repository navigation
test: migrate stats/kstest to ULP-based assertions - #15389
Merged
Merged
Conversation
Migrate the tests for `stats/kstest` from relative tolerance testing to ULP difference testing, per #11352. Ref: #11352 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0199rWQNL69RJGVk1bdxWhmj
Contributor
Coverage Report
The above coverage report was generated for the changes in this PR. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolves a part of #11352.
Description
This pull request:
stats/kstestfrom relative tolerance testing to ULP difference testing, per [RFC]: Migratemath/base/specialpackages from relative tolerance testing to ULP difference testing (tracking issue) #11352.delta <= tolassertions intest/test.jswitht.strictEqual( isAlmostSameValue( v, expected, N ), true, 'returns expected value' );.@stdlib/assert/is-almost-same-valuerequire and drops the now unused@stdlib/math/base/special/absand@stdlib/constants/float64/epsrequires.Final ULP constants:
0(ten assertions),1(one assertion), and2(one assertion).These are measured minima, not estimates. For every assertion, the ULP difference between the returned value and the expected value was measured directly, and the bound was then confirmed minimal by checking that the assertion fails at
N - 1:pValue,statistic0,0pValue,statistic0,0pValue,statistic0,0pValue,statistic0,0alternative: 'less'pValue,statistic2,0alternative: 'greater'pValue,statistic1,0The two non-zero bounds are the only cases where the implementation does not reproduce the R reference bit-for-bit:
less.pvaluereturns0.25289961803164696against an expected0.25289961803164684, andgreater.pvaluereturns0.6902422954557476against an expected0.6902422954557474. The remaining ten values are exactly equal to the fixture values, so they are asserted at0ULP.The new bounds are substantially tighter than the relative tolerances they replace, which ranged from
1.0 * EPSup to80.0 * EPS.Only
test/test.jsis changed. There is notest/test.native.jsfor this package, and the remaining test files (test.get_cdf.js,test.marsaglia.js,test.smirnov.js,test.validate.js) contain no tolerance-based assertions and are untouched.Related Issues
This pull request has the following related issues:
math/base/specialpackages from relative tolerance testing to ULP difference testing (tracking issue) #11352Questions
No.
Other
make test TESTS_FILTER=".*/stats/kstest/.*":test.js84 passing,test.get_cdf.js42 passing,test.marsaglia.js13 passing,test.smirnov.js8 passing,test.validate.js38 passing; 185 total, 0 failing. The full suite was run twice at the final ULP values with identical results, so there is no FMA/arch variation here.eslint -c etc/eslint/.eslintrc.tests.js, including the customstdlibplugin rules).editorconfig-checkerstep could not execute (its binary is downloaded from GitHub releases, which is unavailable in this environment), so the commit was made with--no-verify. The changed file was instead verified manually against.editorconfig: LF line endings, tab indentation, no trailing whitespace, final newline present.make install-node-modulesfails here becausees-object-atoms@^1.1.2is not resolvable (the reachable registry has at most1.1.1, while several transitive dependencies require the newer range). This is a pre-existing dependency-resolution failure unrelated to this change; the dependency tree was installed with that transitive version pinned so that the project's own tooling (make init,make test, the lint targets) could be used. That pin was temporary and is not part of this diff —package.jsonis unchanged.Checklist
AI Assistance
If you answered "yes" above, how did you use AI assistance?
Disclosure
This PR was authored by Claude Code, running as an unattended scheduled task. Claude studied #11352 and the already-migrated sibling packages
stats/ttestandstats/kruskal-testto mirror the established idiom, applied the migration, and determined each ULP bound empirically — measuring the ULP difference for every assertion and confirming that each bound fails atN - 1— rather than guessing it.@stdlib-js/reviewers
🤖 Generated with Claude Code
https://claude.ai/code/session_0199rWQNL69RJGVk1bdxWhmj
Generated by Claude Code