Skip to content

fix: the deferred queue was not moved to id keying with the rest of 0.23.0 - #83

Merged
dcj merged 1 commit into
mainfrom
fix/deferred-queue-id-keying
Aug 21, 2026
Merged

fix: the deferred queue was not moved to id keying with the rest of 0.23.0#83
dcj merged 1 commit into
mainfrom
fix/deferred-queue-id-keying

Conversation

@dcj

@dcj dcj commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Closes #82, plus the two related defects flagged at the bottom of that issue, both verified before fixing.

#82 — an infinite loop, and my regression

0.23.0 moved add()'s bookkeeping to the resolved device id. It did not move the deferred paths with it. add()'s id short-circuit returns the existing device before reaching the identity-based if spec in self._deferred: self._deferred.remove(spec), so the spec stays queued, resolve_deferred() counts the returned device as progress, and loops over an unchanged queue forever.

Reproduced with a timeout to confirm it is a spin and not just slow:

deferred after add    : 1
device built          : True
deferred still queued : 1
calling resolve_deferred()...
exit code: 124   <- timed out

Two things make this worse than the earlier keying issues, and the report is right about both. It hangs rather than returning something wrong, and the natural place to call resolve_deferred() is a change callback on a single dispatch thread, so it stops everything the producer publishes. And it is triggered by exactly the pattern id-keying was introduced to enable — following the new guidance is what walks into it.

Two changes, because either alone fixes this instance and only the second makes the class of bug degrade instead of hang:

  1. the queue drains on the short-circuit path as well as the full-materialization path, keyed by resolved id rather than object identity
  2. resolve_deferred()'s progress is the queue shrinking, never add() returning something

A stale resolver could orphan its device

remove() re-resolved device_id through the spec's callable. A producer's resolver commonly reads the producer's own model, and that model stops answering exactly when teardown begins, so remove() got None and returned silently — device still live on the broker, retained topics still there.

The builder now latches the id a spec resolved to when built. remove(), device_for(), homie_properties() and extend() consult the latch first and fall back to resolving afresh, so a stale resolver is safe and an equal-but-distinct spec still works. The latch is dropped with the device it named, so a rebuilt spec resolves again rather than answering from a dead entry.

The python_type check is removed

I added it in 0.23.0 answering #66's open question, and I answered it wrong. It guarded a difference with no runtime consequence: the observable property's type is metadata, nothing reads it, set_value neither coerces nor validates against it, and wire coercion belongs to the Homie property.

It also misfired on the normal case — a producer whose model uses a richer python type than the datatype-derived default, an Enum subclass for an ENUM property, which is 30 of 134 definitions in one real declaration set — and it raised mid-materialization, leaving a half-built device rather than failing at declaration time. Logged at debug now.

On the tests

Both tests that asserted removed behavior are rewritten, not deleted, so the contract change is visible in the diff. The strand-on-failure test needed a different way to force a raise, since it had been using the type check; it now uses a node_type callable that throws on the second capability, which tests the same thing without depending on the removed check.

test_resolve_deferred_returns_when_another_spec_built_the_device hangs rather than fails if this regresses. CI's timeout-minutes: 5 is the net, which is what it was added for in 0.20.1.

Verification

706 tests pass (700 before, 6 new plus 2 rewritten). All three reproductions run clean. ruff and markdownlint clean.

🤖 Generated with Claude Code

….23.0

#82: resolve_deferred() spun forever when the device a deferred spec was
waiting for had already been built by an equal-but-distinct DeviceSpec.

0.23.0 moved add()'s bookkeeping to the resolved device id, which was the
right change, and did not move the deferred paths with it. add()'s id
short-circuit returned the existing device BEFORE reaching the
identity-based `if spec in self._deferred: self._deferred.remove(spec)`,
so the spec stayed queued; resolve_deferred() then called add(), got a
device back, counted that as progress, and looped over an unchanged
queue. Every iteration made apparent progress and the queue never
drained.

It is a regression of exactly the pattern the id-keying was introduced to
enable, so following the new guidance is what walked into it. And it
hangs rather than misbehaves: the natural place to call
resolve_deferred() is a change callback on a single dispatch thread, so
it stops everything the producer publishes.

Two changes, because either alone would fix this one and only the second
makes the class of bug degrade instead of hang:

  - the queue drains on the short-circuit path as well as the
    full-materialization path, keyed by resolved id rather than identity
  - progress is the queue SHRINKING, never add() returning something, so
    a future path that answers without draining is a no-op not a spin

Two more from the same family, both verified before fixing:

  - remove() re-resolved device_id through the spec's callable. A
    producer's resolver commonly reads the producer's own model, which
    stops answering once teardown begins, so remove() got None and
    returned silently, leaving the device live with its retained topics.
    The builder now latches the id a spec resolved to when built;
    remove(), device_for(), homie_properties() and extend() consult it
    first and fall back to resolving afresh, so a stale resolver is safe
    and an equal-but-distinct spec still works. The latch dies with the
    device it named.

  - the python_type disagreement check from 0.23.0 is removed. It guarded
    a difference with no runtime consequence (the observable type is
    metadata; nothing reads it, set_value neither coerces nor validates,
    and wire coercion belongs to the Homie property), it misfired on the
    normal case of a model using a richer type than the datatype-derived
    default (30 of 134 definitions in one real set), and it raised MID
    materialization, leaving half a device. Logged at debug instead.

Both tests that asserted the removed behavior are rewritten rather than
deleted, so the change of contract is visible in the diff.

Closes #82

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@dcj
dcj merged commit a7bd1b7 into main Aug 21, 2026
5 checks passed
@dcj
dcj deleted the fix/deferred-queue-id-keying branch August 21, 2026 15:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

resolve_deferred() never returns when a deferred spec's device was built by a different spec object

1 participant