Repository navigation
AWS: Don't close analytics accelerator stream factories that are still in use - #18589
Open
binarybrain-009 wants to merge 1 commit into
Open
binarybrain-009 wants to merge 1 commit into
binarybrain-009 wants to merge 1 commit into
Conversation
…l in use The stream factory cache was bounded at 100 entries and closed evicted factories, so with more than 100 live S3FileIO instances in one JVM, reads through an evicted factory hung. Factories are now only closed when their S3FileIO is closed. Closes apache#17485
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The analytics accelerator stream factory cache is bounded at 100 entries and its removal listener closes evicted factories, so once a JVM has more than 100 live
S3FileIOinstances, reads through an evicted factory hang. Every entry already has an owner that removes it: closing theS3FileIOcallscleanupCache. This drops the size bound so a factory is closed only when itsS3FileIOis closed. FileIOs that are never closed keep their factory cached, but they already keep their S3 clients open.Closes #17485
Test plan:
TestAnalyticsAcceleratorUtilopens 150 FileIOs and checks that every factory is still cached after cache maintenance. It fails with the size bound and passes without it.AI Disclosure