Skip to content

Remove redundant nullptr checks after allocate() in bloom_filter - #578

Merged
proost merged 4 commits into
apache:masterfrom
P92Z:fix/bloom-filter-allocation-check
Nov 19, 2026
Merged

proost merged 4 commits into
apache:masterfrom
P92Z:fix/bloom-filter-allocation-check

Conversation

@P92Z

@P92Z P92Z commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Problem

bloom_filter checks the result of allocate() for nullptr in four places and throws std::bad_alloc if it is null. A standard-conforming allocator never returns nullptr from allocate(n) for n > 0 — it throws std::bad_alloc itself, so these checks can never fire.

The other sketches in the library don't make such checks, and neither do the standard containers that the library uses.

Misplaced check

One of the checks was also in the wrong place. In the main constructor, std::fill_n wrote to the buffer before the check:

bit_array_ = AllocUint8(allocator_).allocate(num_bytes);
std::fill_n(bit_array_, num_bytes, 0);
if (bit_array_ == nullptr) {
  throw std::bad_alloc();
}

With an allocator that returns nullptr, this writes through a null pointer before the check runs. UBSan reports store to null pointer, and ASan reports a SEGV in std::fill_n.

Changes

bloom_filter_impl.hpp: removed the nullptr checks after allocate() in:

  • the constructor from num_bits
  • the copy constructor
  • deserialize() from a stream
  • deserialize() from memory (the copying branch of internal_deserialize_or_wrap(); the wrap() branch never allocated).

Behavioral impact

  • Standard-conforming allocators: behavior is unchanged — a failed allocation still throws std::bad_alloc, now from allocate() itself.
  • Allocators that return nullptr instead of throwing: these do not meet the Allocator requirements and aren't supported elsewhere in the library. With such an allocator, three of these paths used to throw std::bad_alloc; they no longer do.

Alternative

If you'd rather keep the checks, the misplaced one can instead be moved before std::fill_n, as the copy constructor and both deserialize() paths already do.

Tests

bloom_filter_allocation_test.cpp has a new test with an allocator that fails in the standard way — by throwing std::bad_alloc. It checks that the exception reaches the caller from:

  • builder::create_by_size()
  • deserialize() from memory
  • deserialize() from a stream

The Bloom filter tests pass under ASan and UBSan.

Scope

This PR touches only filters/.

Update after review

  • Non-empty images with a zero-length bit array are now rejected as corrupt in both deserialize() paths, so allocate() is never called with 0. The check also covers wrap() and writable_wrap(), so no filter with a capacity of 0 can be created. Tests cover each path, and that the capacity of a new filter is rounded up to a multiple of 64 bits.
  • The test allocator is in an anonymous namespace.

@proost proost left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ignore previous review. I missed allocate(0) case.

@proost proost left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for confusing.

Most cases you are right.

According to reference, allocate(0) is undefined.

So we should check num_bytes first.

@P92Z

P92Z commented Nov 15, 2026

Copy link
Copy Markdown
Contributor Author

Ignore previous review. I missed allocate(0) case.

Thanks for the review and clarification!

I'll check allocate(0) and update the patch accordingly.

@leerho

leerho commented Nov 16, 2026

Copy link
Copy Markdown
Member

@P92Z thanks for taking on the allocate(0) update. Just checking in, since we'd like to get this into the next minor release.

As far as I can see, allocate(0) can only be reached from the two deserialize paths (stream and bytes): a non-empty image with num_longs == 0 goes straight to allocate(num_bytes) with 0. The constructor already rejects 0 bits, and an empty image goes through the constructor too. So rejecting num_longs == 0 for a non-empty image with std::invalid_argument, in both paths, plus a test for each, should cover it. A filter with 0 capacity then can't exist, so the copy constructor is covered as well.

The branch also conflicts with master now, because #580 changed the stream deserialize next to one of the removed checks. Once the update is in, we'll take care of the rebase, as we said. If you're busy, we're also happy to push the fix to your branch ourselves (maintainer edits are enabled) and you can review it. Just let us know which you'd prefer.

Thanks again for this.

@P92Z

P92Z commented Nov 17, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @leerho, I'm still on it. Your analysis matches mine, and I'll push the update shortly: rejecting a non-empty image with num_longs == 0 with std::invalid_argument in both deserialize paths, with tests.

@P92Z

P92Z commented Nov 17, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @leerho, and sorry for the wait. I pushed the update:

  • The second commit rejects a non-empty image with num_longs == 0 with std::invalid_argument in both deserialize paths, so allocate() is never called with 0. It has a test for each path. The same check also covers wrap() and writable_wrap(): they don't allocate, but would otherwise create a filter with a capacity of 0, and update() on it divides by zero. The test also checks that the capacity of a new filter is rounded up to a multiple of 64 bits, so that, as you say, a filter with 0 capacity can't exist.
  • The third commit puts the test allocator in an anonymous namespace, as @proost suggested.

On rebasing onto master after #580: I checked it locally. There are two small conflicts: the removed nullptr check is next to the read() that #580 moved into the try block (keep the read() there and drop the check), and both PRs add tests at the end of the same files (keep both). With those resolved, all tests pass. I'm happy for you to do the rebase as you offered.

@P92Z
P92Z requested a review from proost November 17, 2026 12:06

@proost proost left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you rebase? Conflict happens

P92Z added 3 commits November 17, 2026 23:39
Standard-conforming allocators report failure by throwing from
allocate(), never by returning nullptr, so these checks can never fire.
One of them was also placed after std::fill_n had already written to the
buffer. Other sketches in the library don't perform such checks either.

Add a test that a failing (throwing) allocator propagates std::bad_alloc
from the constructor and both deserialize paths.
The length of the bit array is read from the serialized image, and 0
was rejected only for empty images. For a non-empty image with 0 longs,
deserialize() called allocate(0), and deserialize(), wrap() and
writable_wrap() all created a filter with a capacity of 0. update() on
that filter then divided by zero.

Reject such images as corrupt, as other invalid header fields are.

The tests cover non-empty and empty images with 0 longs, and that the
capacity of a new filter is rounded up to a multiple of 64 bits, so its
bit array is never shorter than 8 bytes.
So that it can't clash with the failing allocators that other tests
define in namespace datasketches.
@P92Z
P92Z force-pushed the fix/bloom-filter-allocation-check branch from e18b781 to 1b1e931 Compare November 17, 2026 18:41
@P92Z

P92Z commented Nov 17, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto current master. The conflicts with #580 were the ones I described: I dropped the nullptr check next to the read() that #580 moved into the try block, and kept the tests from both PRs. All tests pass locally.

@P92Z
P92Z requested a review from proost November 17, 2026 18:43

@proost proost left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not num_longs == 0. We should check num_bytes directly. Because overflow can happens in shifts

num_longs is read from the image as a uint32_t, and num_longs << 3 and
num_longs << 6 were computed in 32 bits before being stored in
uint64_t. They could overflow: a non-empty image with 2^29 longs got a
bit array of 0 bytes and a capacity of 0, so it passed the check for 0
longs, called allocate(0), and update() then divided by zero. Even a
valid filter of 2^32 bits or more was read back with its capacity
modulo 2^32.

Widen num_longs before shifting, and check the size of the bit array
itself, num_bytes, instead of num_longs, so that allocate() is never
called with 0.

In the tests, give the filter type with the throwing allocator a short
name, throwing_filter, and name the parameters of the bad_alloc test.
@P92Z

P92Z commented Nov 18, 2026

Copy link
Copy Markdown
Contributor Author

Not num_longs == 0. We should check num_bytes directly. Because overflow can happens in shifts

Thanks @proost, you're right. I reproduced it: a non-empty image with num_longs = 2^29 passes the num_longs == 0 check, but num_longs << 3 is computed in 32 bits and gives 0, so deserialize() called allocate(0) and created a filter with a capacity of 0.

The same overflow in num_longs << 6 also affects valid filters of 2^32 bits or more: they were read back with their capacity modulo 2^32. For example, a wrapped image of a 2^32-bit filter had a capacity of 0.

The new commit widens num_longs to 64 bits before both shifts in both deserialize paths, and checks num_bytes == 0 instead of num_longs == 0, right before allocate(). A new test uses an image with 2^29 longs: from memory it is now rejected as too short, and from a stream the allocator is asked for 2^32 bytes rather than 0 (the test allocator records the size and refuses, so nothing is allocated). On the previous commit, the test fails in both sections.

One more thing, not in this PR: neither deserialize path checks num_longs against the maximum filter size, so the stream path asks the allocator for whatever size the header says. I'm happy to look at that separately if you think it's worth it.

@P92Z
P92Z requested a review from proost November 18, 2026 15:03

@proost proost left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@proost
proost merged commit fb40733 into apache:master Nov 19, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants