fix: unsubscribe shadow topics so reconnects release the connection graph - #76
Merged
Merged
Conversation
…leak get_rest_devices built its own EventLoopGroup, DefaultHostResolver and ClientBootstrap on every call. awscrt starts a native event loop thread per EventLoopGroup and only stops it when the native resource is destroyed, which never happens here: the per-device shadow subscriptions hold callbacks that reference the device, which references the connection, which owns the bootstrap, so the graph stays reachable even after disconnect(). Callers rebuild the connection whenever the AWS credentials expire, and Hatch issues those hourly, so every refresh stranded another thread along with its pipe pair and CRT buffers. On a long-running Home Assistant instance this showed up as ~24 leaked AwsEventLoop threads per day and steady RSS growth; one install reached 213 leaked threads over 8 days of uptime. Use awscrt's process-wide static default bootstrap instead, which keeps the cost flat at a single event loop thread no matter how often we reconnect. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…raph ShadowClientSubscriberMixin subscribed to update/accepted and get/accepted in __init__ and threw the returned topics away, logging them and nothing more. There was no unsubscribe path anywhere in the library. awscrt holds a reference from native code to each subscription callback, and both callbacks close over self. Python's garbage collector cannot see those references, so the device, its shadow client, and the TLS context and MQTT buffers under them stayed alive even after disconnect() closed the socket and every Python caller had dropped its reference. Callers that rebuild the connection therefore stranded one full connection graph per rebuild. ha_hatch rebuilds whenever the AWS credentials expire, which Hatch issues hourly, so a long-running Home Assistant accumulated one per hour for the life of the process -- measured at 140 live RestIot objects after 137 hours of uptime, an exact 1:1 with the reconnect count, worth ~28 MB/day. Record the topics and add unsubscribe() to release them. Tolerates an already dead connection: the UNSUBACK never arrives, but tearing the connection down releases the same native references, so the topics are cleared either way. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016q4bTcKK8Peja9RBC3jmKd
No behavior change; the rest of this file predates black adoption and is left as-is to keep the diff focused, but new code should match project style.
mabrews
added a commit
to mabrews/ha_hatch
that referenced
this pull request
Aug 26, 2026
The coordinator's teardown now calls RestDevice.unsubscribe(), which only exists as of hatch_rest_api 1.34.4 (dahlb/hatch_rest_api#76).
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stacked on #75 — GitHub can only diff against
main, so this PR's diff includesthat commit until #75 merges. The only new commits here are
a284bf7and051854a; once #75 lands this will shrink to just those.ShadowClientSubscriberMixin.__init__subscribes to two AWS IoT shadow topics perdevice and only logged the returned unsubscribe handles, never storing or calling
them. There was no
unsubscribe()anywhere in the library.awscrt holds a reference from native code to each subscription callback, and both
callbacks close over
self. Python's garbage collector can't see thosereferences, so the device, its shadow client, and the TLS context and MQTT
buffers underneath stayed alive even after
disconnect()closed the socket andevery Python caller had dropped its reference.
Callers that rebuild the connection therefore strand one full connection graph per
rebuild.
ha_hatchrebuilds hourly, on AWS credential expiry, so a long-runningHome Assistant instance accumulates one per hour for the life of the process —
confirmed via HA's built-in memory profiler: 140 live
RestIotobjects after 137hours of uptime, an exact 1:1 with the reconnect count, costing roughly 28 MB/day
on top of the thread leak fixed in #75.
Fix: record the topics returned from each subscribe call, and add
unsubscribe()to release them. Tolerant of an already-dead connection — the UNSUBACK never
arrives, but tearing the connection down releases the same native references, so
the topics are cleared either way.
A companion
ha_hatchPR calls this from the coordinator's teardown path, beforedisconnecting.
Test plan
unsubscribe()releases all ofthem, it's idempotent (the coordinator can call it from both the reconnect and
shutdown paths), and it still clears state when the connection is already dead.
All 4 fail against the pre-fix code (
AttributeError: no attribute 'unsubscribe').continuous uptime. Measuring the identical 40-64h uptime window the fix: reuse a shared awscrt ClientBootstrap to stop event loop thread leak #75-only
soak used (for a direct comparison): memory growth dropped from +33.3 MB/day to
+0.7 MB/day, effectively flat. Zero
shadow unsubscribe failedlog entriesacross the whole run.