Skip to content

[Subscription] Release exhausted TsFile parsers before batch ACK - #18820

Merged
jt2594838 merged 1 commit into
apache:masterfrom
Caideyipi:fix/subscription-release-exhausted-tsfile-parser
Oct 10, 2026
Merged

jt2594838 merged 1 commit into
apache:masterfrom
Caideyipi:fix/subscription-release-exhausted-tsfile-parser

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Description

When a subscription tablet batch contains multiple historical TsFiles from the same pipe region, exhausting one file leaves its parser reservation held until the response is ACKed. With the per-pipe-region parser limit set to 1, opening the next file in that batch repeatedly waits for parser memory and times out, delaying snapshot delivery.

Close the current TsFile event as soon as its tablet iterator is exhausted, including files filtered to zero tablets. Keep the parsed tablet event references in the iteration snapshot until ACK or cleanup, so releasing the parser does not release tablets still needed by the response.

Validation

  • Added table-model regression tests using real TsFiles for consecutive files in one region before ACK, a filtered file producing no tablets, and forced cleanup of a partially parsed file with a detached iteration snapshot. The tests also check tablet contents and tablet-memory release.
  • A local baseline run on the release branch with the unchanged iterator lifecycle reproduced both reservation-release failures: 3 tests, 2 failures, 1 pass.
  • Spotless formatting completed for both changed Java files; git diff --check passed.
  • Post-fix tests have not completed. Maven compilation attempts on the release-based working tree failed from Windows/JVM memory exhaustion before test execution. After transplanting the patch and adapting the test constructor to master, compilation and tests were not rerun locally. CI validation and the repeated snapshot cluster scenario remain pending.

This PR has:

  • been self-reviewed, including the synchronized iterator and parser/tablet cleanup lifetimes.
  • added comments explaining why exhausted parsers must close before batch ACK.
  • added regression tests for the affected resource lifetimes.

Key changed/added classes:

  • SubscriptionPipeTabletEventBatch
  • SubscriptionPipeTabletEventBatchTest

@jt2594838
jt2594838 merged commit abcbf2e into apache:master Oct 10, 2026
39 checks passed
@jt2594838
jt2594838 deleted the fix/subscription-release-exhausted-tsfile-parser branch October 10, 2026 11:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants