[type:fix] release the clients that are replaced or removed (#7163) - #7164
Conversation
- CachePluginDataHandler: the previous cache was closed before the new one was installed, and the Singleton entry keeps pointing at the old instance, so a request landing in between was handed the cache whose connection factory had just been released. Install first, then close. - AiTokenLimiterPluginHandler: add removePlugin, which destroys the cached client and clears both caches, so removing or disabling the plugin no longer leaves its pool and its threads alive - tests: CachePluginDataHandlerReplacementTest observes the replacement order without depending on the embedded redis of the existing test class, and AiTokenLimiterPluginHandlerTest covers removePlugin
Aias00
left a comment
There was a problem hiding this comment.
APPROVE — this closes the two release paths that were still open after #7157.
Thanks for picking these up. Both halves address real gaps:
AiTokenLimiterPluginHandler#removePlugin — without it, removing the plugin left the Lettuce client (connection pool plus its Netty resources) alive until the process exited, and left both REDIS_CACHED_HANDLE and REDIS_PROPERTIES_CACHED_HANDLE entries behind. Destroying before removing the handles is the correct order, and Objects.nonNull guards the case where the plugin was never initialised. The test asserts the thing that actually matters (isRunning() is false and the template is gone), rather than just counting destroy calls.
CachePluginDataHandler — installing the new cache before closing the previous one is the fix I was hoping for. With #7157 landed, a closed cache really is dead (its client is released), so the old order left a window where CacheUtils.getCache() could hand out an already-closed instance. Extracting closeCache(ICache) and reusing it from closeCacheIfNeed() keeps the disable/remove paths unchanged — I checked, closeCacheIfNeed() is still used by the disabled branch and by removePlugin, so it is not dead code.
CachePluginDataHandlerReplacementTest is a neat way to pin the ordering without depending on the embedded Redis: capturing what CacheUtils.getCache() returns inside close() is exactly the assertion needed, and it would fail under the old order.
Two follow-ups, both non-blocking:
RateLimiterPluginDataHandlerstill has noremovePlugin. It is the same client-lifecycle leak through a different door, and now thatdestroyQuietlyexists it is the same two lines. Happy to see it here or in a separate PR.ApplicationConfigCache.getInstance().invalidateAll()is now called before the new cache is installed, butSingleton.INST.single(ICache.class, ...)happens after — fine — though note that a request running concurrently between the two can still observe the old cache instance while its config cache has already been invalidated. That was true before as well; just flagging that this PR narrows the dangerous window but does not close it entirely.
CI note: the four shenyu-integrated-test-k8s-ingress-* jobs are red, but they fail with mvnd StaleAddressException: ... daemon may have crashed / No message received within 3000ms — build-daemon infrastructure, not this change. build, Analyze (java), CodeQL and check-license-header are all green.
Motivation
Follow-up of #7157, tracked by #7163. Two of its three items are fixed here; the third one needs a
decision first, see Not in this PR below.
1.
CachePluginDataHandler: the new cache is installed before the previous one is closedhandlerPlugin(...)closed the previous cache first (closeCacheIfNeed()) and installed the new oneafterwards, while the
Singletonentry keeps pointing at the old instance until that install. Between thetwo statements
CacheUtils.getCache()therefore handed out the cache whose connection factory had justbeen released by #7157, so a request landing in that window failed. The new cache is now built and
installed first, and the previous one is closed after it — the order the three handlers changed by #7157
already use.
closeCacheIfNeed()keeps its meaning for the disabled and removed paths, where no requestcan reach the cache any more.
Evicting
ICache.classinstead (the other option raised in the review) is not possible today:Singletonhas no removal API, and the readers of
CacheUtils.getCache()do not null-check it.2.
AiTokenLimiterPluginHandlerreleases its client inremovePluginThe handler keeps its client in a
CommonHandleCache, so removing or disabling the plugin left the client,its connection pool and its Netty event loops alive for the rest of the process.
removePluginnowdestroys it and clears both caches, like
SensitiveWordPluginDataHandlerdoes since #7157.Not in this PR
RateLimiterPluginDataHandlerhas the same gap, but its client lives inSingleton.INSTandSingletonhas no removal API. Destroying it without evicting the entry would make the next
handlerPlugintreat thedestroyed client as valid — it rebuilds only when
RedisConfigPropertiesdiffers — and hand it toRedisRateLimiter/ConcurrentRateLimiterAlgorithm, which do not null-checkSingleton.INST.get(ReactiveRedisTemplate.class). The eviction point has to be decided first, so I asked in#7163 whether
Singletonshould grow aremove(Class)or this handler should keep its client in aCommonHandleCachelike the others.Testing
BUILD SUCCESS, checkstyle clean:CachePluginDataHandlerReplacementTest1/1 — a new class that captures, from aclose()callback, whichcache
CacheUtils.getCache()hands out while the previous one is being closed, and asserts that it isalready the new one. It is a separate class on purpose:
CachePluginDataHandlerTeststarts an embeddedredis in
@BeforeAlland reportsTests run: 0on arm64 macOS (also without this change), so a testadded there would not have run.
AiTokenLimiterPluginHandlerTest5/5, including the newremovePlugincase: the released client stopsrunning and both caches are cleared.