Skip to content

fix: use expm1 in stats/base/dists/lognormal/variance - #15774

Open
Abhist17 wants to merge 5 commits into
stdlib-js:developfrom
Abhist17:fix/lognormal-variance-small-sigma
Open

Abhist17 wants to merge 5 commits into
stdlib-js:developfrom
Abhist17:fix/lognormal-variance-small-sigma

Conversation

@Abhist17

@Abhist17 Abhist17 commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Description

What is the purpose of this pull request?

This pull request:

  • evaluates stats/base/dists/lognormal/variance as expm1( sigma^2 ) * exp( 2*mu + sigma^2 ) instead of ( exp( sigma^2 ) - 1 ) * exp( 2*mu + sigma^2 ) (JS and C).

exp(sigma^2) - 1 cancels when sigma is small, and is 0 for sigma^2 < 2^-53:

var variance = require( '@stdlib/stats/base/dists/lognormal/variance' );

variance( 0.0, 1.0e-9 );
// develop: 0.0
// this PR: 1.0e-18

variance( 0.0, 1.0e-5 );
// develop: 1.000000082840371e-10
// this PR: 1.0000000001500002e-10

stdev calls variance, so it's fixed by this change too. Its fixtures had the same Distributions.jl cancellation, so they're regenerated the same way (with a new small_sigma.json). The tolerances are 18 ULP on data and 1 on small_sigma.

Fixtures and tests:

  • runner.jl now evaluates the textbook formula in 2048-bit BigFloat instead of Distributions.var, which uses the same double-precision expression. Distributions is dropped from REQUIRE, and data.json is regenerated with the same input ranges.
  • There's a new small_sigma.json with sigma log-spaced over [1e-150, 0.1]. It passes at 3 ULP in JS and native.
  • On data.json, the max error is 24 ULP (develop: 45). It comes at sigma ≈ 4.2, where the rounding of 2*mu + sigma^2 is multiplied by the size of the exponent, so the tolerance goes from 0 to 24. The old 0 only held because the fixture used the same double-precision formula.
  • On develop, 1,002 of the 1,110 assertions in test.js fail. With this change they all pass.

Related Issues

Does this pull request have any related issues?

None.

Questions

Any questions for reviewers of this pull request?

lognormal/skewness has the same exp(sigma^2) - 1 cancellation, and lognormal/kurtosis has a worse one: e^{4s} + 2e^{3s} + 3e^{2s} - 6 with s = sigma^2. With u = expm1(s) the latter is exactly u*(16 + u*(15 + u*(6 + u))), with no cancellation. I'll send both separately.

Other

Any other information relevant to this pull request? This may include screenshots, references, and/or implementation notes.

No.

Checklist

Please ensure the following tasks are completed before submitting this pull request.

AI Assistance

When authoring the changes proposed in this PR, did you use any kind of AI assistance?

  • Yes
  • No

If you answered "yes" above, how did you use AI assistance?

  • Code generation (e.g., when writing an implementation or fixing a bug)
  • Test/benchmark generation
  • Documentation (including examples)
  • Research and understanding

Disclosure

If you answered "yes" to using AI assistance, please provide a short disclosure indicating how you used AI assistance. This helps reviewers determine how much scrutiny to apply when reviewing your contribution. Example disclosures: "This PR was written primarily by Claude Code." or "I consulted ChatGPT to understand the codebase, but the proposed changes were fully authored manually by myself.".

This PR was written primarily by Claude Code. I found the bug by scanning stats/base/dists for exp(...) - 1.


@stdlib-js/reviewers

`exp(sigma^2) - 1` cancels for small `sigma` and is `0` for
`sigma^2 < 2^-53`. The fixtures are now generated from the textbook
formula in extended precision.
@Abhist17
Abhist17 requested a review from a team October 1, 2026 23:42
@stdlib-bot stdlib-bot added Statistics Issue or pull request related to statistical functionality. Needs Review A pull request which needs code review. labels Oct 1, 2026
@stdlib-bot

stdlib-bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Coverage Report

Package Statements Branches Functions Lines
stats/base/dists/lognormal/stdev $\\color{green}169/169$
$\\color{green}+0.00\\%$
$\\color{green}5/5$
$\\color{green}+0.00\\%$
$\\color{green}2/2$
$\\color{green}+0.00\\%$
$\\color{green}169/169$
$\\color{green}+0.00\\%$
stats/base/dists/lognormal/variance $\\color{green}179/179$
$\\color{green}+0.00\\%$
$\\color{green}9/9$
$\\color{green}+0.00\\%$
$\\color{green}2/2$
$\\color{green}+0.00\\%$
$\\color{green}179/179$
$\\color{green}+0.00\\%$

The above coverage report was generated for the changes in this PR.

Abhist17 and others added 3 commits October 2, 2026 05:19
`stdev` calls `variance`, so its Distributions.jl fixtures, which have
the same `exp(sigma^2) - 1` cancellation, no longer match at 0 ULP.
Signed-off-by: Athan <kgryte@gmail.com>
Signed-off-by: Athan <kgryte@gmail.com>
Signed-off-by: Athan <kgryte@gmail.com>

@kgryte kgryte left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These changes seem reasonable. Thanks for flagging, @Abhist17.

@kgryte kgryte added Tests Pull requests specifically adding tests. review: 3 and removed Needs Review A pull request which needs code review. labels Oct 2, 2026
Planeshifter

This comment was marked as outdated.

@kgryte

This comment was marked as outdated.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review: 3 Statistics Issue or pull request related to statistical functionality. Tests Pull requests specifically adding tests.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants