Skip to content

fix: avoid memory leak during bloom deserialization - #580

Merged
leerho merged 2 commits into
apache:masterfrom
proost:fix-bloom-stream-deserialize-leak
Nov 15, 2026
Merged

leerho merged 2 commits into
apache:masterfrom
proost:fix-bloom-stream-deserialize-leak

Conversation

@proost

@proost proost commented Nov 14, 2026

Copy link
Copy Markdown
Member

During the #578 review, I found memory leak can happen in L320 if exception is thrown. Because in this case bit_array never deallocated.

To more safe deserialization, I added flag checks more.

@proost
proost requested a review from leerho November 14, 2026 14:51
@proost proost self-assigned this Nov 14, 2026

@leerho leerho left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, Hyeonho. The leak fix is right: with the bit array read inside the try, a short read on a stream with exceptions enabled no longer leaks bit_array. And the is.good() check after the read catches a truncated image of a dirty filter, which the bit count check skips. The constructor that takes ownership doesn't throw, so once the try has passed, nothing can leak. I merged this into current master locally, and bloom_filter_test passes under ASan and UBSan. Your new test fails against master's code, as it should.

One requested change: the empty path returns before the new is.good() check. read<T>() returns an uninitialized value after a short read, so a truncated empty image builds a filter from garbage num_hashes, seed and num_longs. I serialized an empty filter (24 bytes) and deserialized every truncation from 4 to 22 bytes with this branch: all of them were accepted. Some came out with about 3.1 billion bits, so a 4-byte input makes the reader allocate and zero about 390 MB.

Could you move the check up, to just after the header fields are read and before if (is_empty)?

  const uint32_t num_longs = read<uint32_t>(is); // sized in java longs
  read<uint32_t>(is); // unused
  if (!is.good()) throw std::runtime_error("error reading from std::istream");

  // if empty, stop reading
  if (is_empty) {

The check after raw_num_bits_set is still needed for the non-empty path. A test that truncates an empty image and expects std::runtime_error would cover it.

Not for this PR: neither deserialize() overload checks num_hashes on the non-empty path, so an image with 0 hashes is accepted. The empty path catches it through the constructor. That's worth a separate small fix.

Also a heads-up: this PR and #578 both change bloom_filter_impl.hpp and bloom_filter_allocation_test.cpp, so whichever merges second will need a rebase.

@proost

proost commented Nov 15, 2026

Copy link
Copy Markdown
Member Author

You are right. I moved stream validation.

@proost
proost requested a review from leerho November 15, 2026 14:58

@leerho leerho left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, Hyeonho. The stream check now runs right after the header fields, before the empty-filter return, so a truncated empty image is rejected instead of building a filter from uninitialized num_hashes, seed and num_longs. Moving the check rather than adding a second one is fine: on the non-empty path, a stream that ends inside raw_num_bits_set makes the bit array read fail too, and the check after that read catches it.

I merged this into current master locally and ran bloom_filter_test under ASan and UBSan: all 16 test cases pass. Your new "truncated empty stream is rejected" test fails against the previous commit's code, as it should.

As noted before, this PR and #578 both change bloom_filter_impl.hpp, so whichever merges second will need a rebase.

@leerho
leerho merged commit d6cec15 into apache:master Nov 15, 2026
21 checks passed
@leerho

leerho commented Nov 15, 2026

Copy link
Copy Markdown
Member

@proost, This is your PR and normally I would let you merge it. But because it is intertwined with 578, I want to simplify this connection by merging this one now so our focus is on 578, which has changes due and has to do some rebasing as well. Once the changes have been addressed and approved, I can do the required rebasing and final merge.

@proost
proost deleted the fix-bloom-stream-deserialize-leak branch November 16, 2026 14:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants