Skip to content

[Bugfix] Preserve retained subscription progress after the last consumer closes - #18894

Merged
jt2594838 merged 2 commits into
apache:masterfrom
Caideyipi:fix/subscription-retained-progress-cleanup
Oct 16, 2026
Merged

jt2594838 merged 2 commits into
apache:masterfrom
Caideyipi:fix/subscription-retained-progress-cleanup

Conversation

@Caideyipi

@Caideyipi Caideyipi commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

Closing the last consumer currently deletes its group's checkpoints even when the topic has retain.progress.after.unsubscribe=true. Recreating the group can then start at the WAL tail and skip data written while the group was offline. A delayed asynchronous queue close can also recreate detached WAL retention after retention was disabled, the topic was dropped, or a replacement queue was registered.

This change preserves configured checkpoints and the empty group's incarnation and SQL dialect on ConfigNode and DataNode, so resubscription can resume correctly. Empty table groups encode their dialect using a negative empty-consumer count, preserving the existing serialized field boundaries. Disabling retention or dropping the topic still clears retained progress and prunes empty ConfigNode groups once no checkpoints remain.

WAL retention registration now tracks the owning queue. Delayed closes unregister the old queue but cannot restore a canceled or superseded detached registration.

Validation:

  • 88 targeted unit tests passed, covering group serialization, selective checkpoint cleanup, snapshots/rejoin in both data models, and deterministic asynchronous-close races.
  • Full English and Chinese reactor test-compile -DskipTests -Dxml-format.skip=true passed (Chinese with -P with-zh-locale).
  • The new IoTDBConsensusSubscriptionBasicTableIT#testLastConsumerCloseRetainsProgress passed on 1 ConfigNode + 1 DataNode with IoTConsensus, covering both direct close and unsubscribe-then-close, followed by consumption of offline writes without seek. Subscription was enabled only in temporary test artifacts because the community edition disables it at compile time.
  • Spotless and git diff --check passed.

@jt2594838
jt2594838 merged commit 9b807ab into apache:master Oct 16, 2026
39 checks passed
@jt2594838
jt2594838 deleted the fix/subscription-retained-progress-cleanup branch October 16, 2026 02:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants