Skip to content

fix: avoid cancellation in math/base/special/versin and haversin - #15609

Open
Abhist17 wants to merge 3 commits into
stdlib-js:developfrom
Abhist17:fix/versin-haversin-accuracy
Open

Abhist17 wants to merge 3 commits into
stdlib-js:developfrom
Abhist17:fix/versin-haversin-accuracy

Conversation

@Abhist17

@Abhist17 Abhist17 commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Description

What is the purpose of this pull request?

This pull request:

  • fixes the loss of precision in math/base/special/versin and math/base/special/haversin (JS and C) for arguments near multiples of 2*pi, including small |x|.
var versin = require( '@stdlib/math/base/special/versin' );

versin( 1.0e-4 );  // 4.999999969612645e-9   (exact: 4.9999999958333e-9)
versin( 1.0e-6 );  // 5.000444502911705e-13  (exact: 4.9999999999996e-13)
versin( 1.0e-7 );  // 4.9960036108132044e-15 (exact: 5e-15)
versin( 1.0e-10 ); // 0                      (exact: 5e-21)

Both functions evaluate 1 - cos(x). When cos(x) is close to one, the subtraction cancels, so the relative error is about 2e-16 / x^2, and below |x| ~ 1.5e-8 the result is exactly 0. This is the region the functions exist for: the haversine formula is used in place of the law of cosines precisely because sin^2(x/2) stays accurate for small angles.

The fix uses the exact half-angle identities:

  • versin(x) = 2*sin(x/2)^2
  • haversin(x) = sin(x/2)^2

x/2 is exact, and sin is accurate over the whole range (including huge x, where it does its own argument reduction), so both are within 2 ULP of a 256-bit reference everywhere I tested, from |x| = 1e-300 to 2^1000*pi/2. On develop, the error near the zeros is unbounded.

Related Issues

Does this pull request have any related issues?

No.

Questions

Any questions for reviewers of this pull request?

No.

Other

Any other information relevant to this pull request? This may include screenshots, references, and/or implementation notes.

  • Fixtures: the Julia runners generated the expected values with the same 1.0 .- cos.( x ) formula, so they recorded the cancellation and the tests compared them with strictEqual. The runners now evaluate the same textbook formula, 1 - cos(x), at 4096 bits and round to Float64, so the fixtures are independent of the identity the implementation uses. The existing ranges are unchanged, and there are new small_negative.json and small_positive.json fixtures (|x| log-spaced over [1e-300, 1]), which the old fixtures didn't cover at all. The fixture tests now allow 2 ULP, and the worst case is 2 ULP.
  • All tests pass in test.js and test.native.js for both packages (32,005 assertions each).
  • vercos and havercos have the same problem near odd multiples of pi. 1 + cos(x) is off by up to 2.4 million ULP there, and 2*cos(x/2)^2 fixes it the same way. The single-precision variants (versinf, haversinf, ...) do too. I'll send those separately to keep this PR small.

Checklist

Please ensure the following tasks are completed before submitting this pull request.

AI Assistance

When authoring the changes proposed in this PR, did you use any kind of AI assistance?

  • Yes
  • No

If you answered "yes" above, how did you use AI assistance?

  • Code generation (e.g., when writing an implementation or fixing a bug)
  • Test/benchmark generation
  • Documentation (including examples)
  • Research and understanding

Disclosure

If you answered "yes" to using AI assistance, please provide a short disclosure indicating how you used AI assistance. This helps reviewers determine how much scrutiny to apply when reviewing your contribution. Example disclosures: "This PR was written primarily by Claude Code." or "I consulted ChatGPT to understand the codebase, but the proposed changes were fully authored manually by myself.".

This PR was written primarily by Claude Code. The bug was found by comparing every stats/base/dists function against SciPy on random parameters.


@stdlib-js/reviewers

`1 - cos(x)` loses significant digits as `cos(x)` approaches one: the
relative error is about `2e-16/x^2`, so `versin(1e-6)` has 4 correct
digits, `versin(1e-7)` has 3, and below `|x| ~ 1.5e-8` the result is
exactly `0` (e.g., `versin(1e-10)` returns `0` instead of `5e-21`). Use
the half-angle identities `versin(x) = 2*sin(x/2)^2` and
`haversin(x) = sin(x/2)^2`, which are within 2 ULP everywhere.
Regenerate the fixtures in extended precision and add fixtures for
small `|x|`.
@Abhist17
Abhist17 requested a review from a team September 27, 2026 23:17
@stdlib-bot stdlib-bot added Math Issue or pull request specific to math functionality. Needs Review A pull request which needs code review. labels Sep 27, 2026
@stdlib-bot

stdlib-bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Coverage Report

Package Statements Branches Functions Lines
math/base/special/haversin $\\color{green}181/181$
$\\color{green}+0.00\\%$
$\\color{green}5/5$
$\\color{green}+0.00\\%$
$\\color{green}2/2$
$\\color{green}+0.00\\%$
$\\color{green}181/181$
$\\color{green}+0.00\\%$
math/base/special/versin $\\color{green}181/181$
$\\color{green}+0.00\\%$
$\\color{green}5/5$
$\\color{green}+0.00\\%$
$\\color{green}2/2$
$\\color{green}+0.00\\%$
$\\color{green}181/181$
$\\color{green}+0.00\\%$

The above coverage report was generated for the changes in this PR.

@kgryte

kgryte commented Sep 28, 2026

Copy link
Copy Markdown
Member

@anandkaranubc Mind taking a look at this one, as well? Same comments as #15611 (comment).

@kgryte kgryte added the Needs Changes Pull request which needs changes before being merged. label Sep 28, 2026
- document the half-angle method in the JSDoc and C doc comments instead of an inline comment
- use `abs2` to square
- generate fixtures from the textbook formula `1 - cos(x)` evaluated at 4096 bits, rather than from the identity used by the implementation, and separate comments from expressions in the Julia runners
@Abhist17

Copy link
Copy Markdown
Contributor Author

Same changes as in #15611, in 814681b: a ## Method section in the JSDoc and C docs, abs2, and fixtures generated from the textbook 1 - cos(x) at 4096 bits (enough for |x| down to 1e-300) instead of from the identity. The regenerated fixtures are identical doubles, and the implementation stays within 2 ULP of them on every range.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Math Issue or pull request specific to math functionality. Needs Changes Pull request which needs changes before being merged. Needs Review A pull request which needs code review.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants