Skip to content

fix: unsubscribe shadow topics so reconnects release the connection graph - #1

Closed
mabrews wants to merge 2 commits into
fix/shared-client-bootstrapfrom
fix/unsubscribe-shadow-topics
Closed

fix: unsubscribe shadow topics so reconnects release the connection graph#1
mabrews wants to merge 2 commits into
fix/shared-client-bootstrapfrom
fix/unsubscribe-shadow-topics

Conversation

@mabrews

@mabrews mabrews commented Aug 25, 2026

Copy link
Copy Markdown
Owner

Summary

Stacked on dahlb#75 — this diff is just the second fix; base it on main once dahlb#75 merges.

ShadowClientSubscriberMixin.__init__ subscribes to two AWS IoT shadow topics per
device and only logged the returned unsubscribe handles, never storing or calling
them. There was no unsubscribe() anywhere in the library.

awscrt holds a reference from native code to each subscription callback, and both
callbacks close over self. Python's garbage collector can't see those references,
so the device, its shadow client, and the TLS context and MQTT buffers underneath
stayed alive even after disconnect() closed the socket and every Python caller
had dropped its reference.

Callers that rebuild the connection therefore strand one full connection graph per
rebuild. ha_hatch rebuilds hourly, on AWS credential expiry, so a long-running
Home Assistant instance accumulates one per hour for the life of the process —
confirmed via HA's built-in memory profiler: 140 live RestIot objects after 137
hours of uptime, an exact 1:1 with the reconnect count, costing roughly 28 MB/day
on top of the thread leak fixed in dahlb#75.

Fix: record the topics returned from each subscribe call, and add unsubscribe()
to release them. Tolerant of an already-dead connection — the UNSUBACK never
arrives, but tearing the connection down releases the same native references, so
the topics are cleared either way.

A companion ha_hatch PR calls this from the coordinator's teardown path, before
disconnecting.

Test plan

mabrews and others added 2 commits August 21, 2026 19:14
…raph

ShadowClientSubscriberMixin subscribed to update/accepted and get/accepted in
__init__ and threw the returned topics away, logging them and nothing more.
There was no unsubscribe path anywhere in the library.

awscrt holds a reference from native code to each subscription callback, and
both callbacks close over self. Python's garbage collector cannot see those
references, so the device, its shadow client, and the TLS context and MQTT
buffers under them stayed alive even after disconnect() closed the socket and
every Python caller had dropped its reference.

Callers that rebuild the connection therefore stranded one full connection
graph per rebuild. ha_hatch rebuilds whenever the AWS credentials expire, which
Hatch issues hourly, so a long-running Home Assistant accumulated one per hour
for the life of the process -- measured at 140 live RestIot objects after 137
hours of uptime, an exact 1:1 with the reconnect count, worth ~28 MB/day.

Record the topics and add unsubscribe() to release them. Tolerates an already
dead connection: the UNSUBACK never arrives, but tearing the connection down
releases the same native references, so the topics are cleared either way.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016q4bTcKK8Peja9RBC3jmKd
No behavior change; the rest of this file predates black adoption and is left
as-is to keep the diff focused, but new code should match project style.
@mabrews

mabrews commented Aug 25, 2026

Copy link
Copy Markdown
Owner Author

Opening this against dahlb/hatch_rest_api directly instead — this fork-internal PR doesn't reach the upstream maintainer.

@mabrews mabrews closed this Aug 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant