fix: the deferred queue was not moved to id keying with the rest of 0.23.0 - #83
Merged
Conversation
….23.0 #82: resolve_deferred() spun forever when the device a deferred spec was waiting for had already been built by an equal-but-distinct DeviceSpec. 0.23.0 moved add()'s bookkeeping to the resolved device id, which was the right change, and did not move the deferred paths with it. add()'s id short-circuit returned the existing device BEFORE reaching the identity-based `if spec in self._deferred: self._deferred.remove(spec)`, so the spec stayed queued; resolve_deferred() then called add(), got a device back, counted that as progress, and looped over an unchanged queue. Every iteration made apparent progress and the queue never drained. It is a regression of exactly the pattern the id-keying was introduced to enable, so following the new guidance is what walked into it. And it hangs rather than misbehaves: the natural place to call resolve_deferred() is a change callback on a single dispatch thread, so it stops everything the producer publishes. Two changes, because either alone would fix this one and only the second makes the class of bug degrade instead of hang: - the queue drains on the short-circuit path as well as the full-materialization path, keyed by resolved id rather than identity - progress is the queue SHRINKING, never add() returning something, so a future path that answers without draining is a no-op not a spin Two more from the same family, both verified before fixing: - remove() re-resolved device_id through the spec's callable. A producer's resolver commonly reads the producer's own model, which stops answering once teardown begins, so remove() got None and returned silently, leaving the device live with its retained topics. The builder now latches the id a spec resolved to when built; remove(), device_for(), homie_properties() and extend() consult it first and fall back to resolving afresh, so a stale resolver is safe and an equal-but-distinct spec still works. The latch dies with the device it named. - the python_type disagreement check from 0.23.0 is removed. It guarded a difference with no runtime consequence (the observable type is metadata; nothing reads it, set_value neither coerces nor validates, and wire coercion belongs to the Homie property), it misfired on the normal case of a model using a richer type than the datatype-derived default (30 of 134 definitions in one real set), and it raised MID materialization, leaving half a device. Logged at debug instead. Both tests that asserted the removed behavior are rewritten rather than deleted, so the change of contract is visible in the diff. Closes #82 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #82, plus the two related defects flagged at the bottom of that issue, both verified before fixing.
#82 — an infinite loop, and my regression
0.23.0 moved
add()'s bookkeeping to the resolved device id. It did not move the deferred paths with it.add()'s id short-circuit returns the existing device before reaching the identity-basedif spec in self._deferred: self._deferred.remove(spec), so the spec stays queued,resolve_deferred()counts the returned device as progress, and loops over an unchanged queue forever.Reproduced with a timeout to confirm it is a spin and not just slow:
Two things make this worse than the earlier keying issues, and the report is right about both. It hangs rather than returning something wrong, and the natural place to call
resolve_deferred()is a change callback on a single dispatch thread, so it stops everything the producer publishes. And it is triggered by exactly the pattern id-keying was introduced to enable — following the new guidance is what walks into it.Two changes, because either alone fixes this instance and only the second makes the class of bug degrade instead of hang:
resolve_deferred()'s progress is the queue shrinking, neveradd()returning somethingA stale resolver could orphan its device
remove()re-resolveddevice_idthrough the spec's callable. A producer's resolver commonly reads the producer's own model, and that model stops answering exactly when teardown begins, soremove()gotNoneand returned silently — device still live on the broker, retained topics still there.The builder now latches the id a spec resolved to when built.
remove(),device_for(),homie_properties()andextend()consult the latch first and fall back to resolving afresh, so a stale resolver is safe and an equal-but-distinct spec still works. The latch is dropped with the device it named, so a rebuilt spec resolves again rather than answering from a dead entry.The python_type check is removed
I added it in 0.23.0 answering #66's open question, and I answered it wrong. It guarded a difference with no runtime consequence: the observable property's
typeis metadata, nothing reads it,set_valueneither coerces nor validates against it, and wire coercion belongs to the Homie property.It also misfired on the normal case — a producer whose model uses a richer python type than the datatype-derived default, an
Enumsubclass for anENUMproperty, which is 30 of 134 definitions in one real declaration set — and it raised mid-materialization, leaving a half-built device rather than failing at declaration time. Logged at debug now.On the tests
Both tests that asserted removed behavior are rewritten, not deleted, so the contract change is visible in the diff. The strand-on-failure test needed a different way to force a raise, since it had been using the type check; it now uses a
node_typecallable that throws on the second capability, which tests the same thing without depending on the removed check.test_resolve_deferred_returns_when_another_spec_built_the_devicehangs rather than fails if this regresses. CI'stimeout-minutes: 5is the net, which is what it was added for in 0.20.1.Verification
706 tests pass (700 before, 6 new plus 2 rewritten). All three reproductions run clean. ruff and markdownlint clean.
🤖 Generated with Claude Code