Skip to content

fix: avoid cancellation for small p in stats/base/dists/geometric/logcdf - #15766

Open
Abhist17 wants to merge 1 commit into
stdlib-js:developfrom
Abhist17:fix/geometric-logcdf-small-p
Open

Abhist17 wants to merge 1 commit into
stdlib-js:developfrom
Abhist17:fix/geometric-logcdf-small-p

Conversation

@Abhist17

@Abhist17 Abhist17 commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

Description

What is the purpose of this pull request?

This pull request:

  • fixes the precision of stats/base/dists/geometric/logcdf for small p (JS, factory and C).

The function computed log1p( -pow( 1.0-p, x+1.0 ) ). Forming 1 - p rounds away the low-order bits of p, and for p < 2^-53 it rounds to 1, so the result is -Infinity:

var logcdf = require( '@stdlib/stats/base/dists/geometric/logcdf' );

logcdf( 0.0, 1.0e-20 );
// develop: -Infinity
// this PR: -46.051701859880914 (= ln(1e-20))

logcdf( 9.0, 1.0e-10 );
// develop: -20.723265754206043
// this PR: -20.72326583739641

For p < 0.5 it now uses log1mexp( (x+1) * log1p(-p) ). For p >= 0.5, 1 - p is exact, so the old form stays. There it is more accurate than going through log1p, because for p near 1 the error in (x+1) * log1p(-p) gets multiplied by its own magnitude.

Fixtures and tests:

  • runner.jl now evaluates the textbook formula ln(1 - (1-p)^(floor(x)+1)) in 2048-bit BigFloat instead of Distributions.logcdf. Distributions computes in double precision with the same loss. Distributions is dropped from REQUIRE. small_p.json and large_p.json are regenerated with the same input ranges.

  • New tiny_p.json: p log-spaced over [1e-300, 0.1], x in [0, 100].

  • Max error against the new fixtures, JS and native:

    fixture develop this PR
    large_p 2 ULP 2 ULP
    small_p 56 ULP 9 ULP
    tiny_p -Infinity for p < 2^-53 3 ULP

    The tolerances are set to those values (were 42 and 15). On develop, 1,006 of the 3,012 assertions in test.logcdf.js fail. With this change they all pass.

Related Issues

Does this pull request have any related issues?

None. Companion to #15765 (geometric/cdf).

Questions

Any questions for reviewers of this pull request?

No.

Other

Any other information relevant to this pull request? This may include screenshots, references, and/or implementation notes.

No.

Checklist

Please ensure the following tasks are completed before submitting this pull request.

AI Assistance

When authoring the changes proposed in this PR, did you use any kind of AI assistance?

  • Yes
  • No

If you answered "yes" above, how did you use AI assistance?

  • Code generation (e.g., when writing an implementation or fixing a bug)
  • Test/benchmark generation
  • Documentation (including examples)
  • Research and understanding

Disclosure

If you answered "yes" to using AI assistance, please provide a short disclosure indicating how you used AI assistance. This helps reviewers determine how much scrutiny to apply when reviewing your contribution. Example disclosures: "This PR was written primarily by Claude Code." or "I consulted ChatGPT to understand the codebase, but the proposed changes were fully authored manually by myself.".

This PR was written primarily by Claude Code. I found the bug by reading the stats/base/dists functions for 1 - p cancellation.


@stdlib-js/reviewers

…logcdf`

`1 - p` rounds away the low-order bits of `p`. For `p < 2^-53` the
function returned `-Infinity`. Below `p = 0.5`, use
`log1mexp( (x+1) * log1p(-p) )`; above it `1-p` is exact and the old
form is kept. The fixtures are generated from the textbook formula in
extended precision.
@Abhist17
Abhist17 requested a review from a team October 1, 2026 23:25
@stdlib-bot stdlib-bot added Statistics Issue or pull request related to statistical functionality. Needs Review A pull request which needs code review. labels Oct 1, 2026
@stdlib-bot

Copy link
Copy Markdown
Contributor

Coverage Report

Package Statements Branches Functions Lines
stats/base/dists/geometric/logcdf $\\color{green}307/307$
$\\color{green}+100.00\\%$
$\\color{green}31/31$
$\\color{green}+100.00\\%$
$\\color{green}4/4$
$\\color{green}+100.00\\%$
$\\color{green}307/307$
$\\color{green}+100.00\\%$

The above coverage report was generated for the changes in this PR.


# Tiny `p`:
x = rand( 1000 ) .* 100.0;
p = exp10.( range( -300.0, stop = -1.0, length = 1000 ) );

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
p = exp10.( range( -300.0, stop = -1.0, length = 1000 ) );
p = exp10.( range( -323.0, stop = -1.0, length = 1000 ) );

Non-blocking: this also covers subnormal p, and the tests still pass. Unlike #15797 (comment), no extra branch is needed here, since x+1 >= 1 keeps (x+1)*log1p(-p) exact.

@Abhist17

Abhist17 commented Oct 8, 2026

Copy link
Copy Markdown
Contributor Author

@0PrashantYadav0 thanks. I tried the -323 range: the subnormal end is fine (e.g. logcdf( 10, 1e-320 ) matches ln(11p)), but with the new grid one point near the top of the range, x = 99.23, p = 0.0476, comes out 5 ULP off against the 3-ULP tolerance. That's the existing formula at ordinary p, not this change, and I didn't want to re-roll x until a draw passed. So I've left the range at -300 for now. Happy to switch it and set that block to 5 ULP if you'd rather have the subnormal coverage.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Needs Review A pull request which needs code review. Statistics Issue or pull request related to statistical functionality.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants