refactor(mcuboot): replace pydantic dataclasses with stdlib dataclasses - #136
Merged
Conversation
Collaborator
Author
|
OK, we can drop 3.10 soon anyway, it's almost EoL |
smp's `screaming-goblin` moves off pydantic to msgspec, so `pip install smp` will stop pulling pydantic in. smpclient never declared pydantic itself -- it has been relying on it arriving transitively via smp 4.x -- so rather than declare a dependency we want gone, remove the need for it. Closes #133. Every class keeps `@dataclass(frozen=True)`; only the import moves. That keeps `ImageTLVValue.__post_init__`, `ImageInfo._map_tlv_type_to_value`'s `cached_property`, and the generic `ImageTLVInfo[T]` exactly as they were, and leaves the public types untouched -- `tlvs` is still a `list`. ## What pydantic was actually doing Three things, and only one of them was interesting: 1. `@dataclass(frozen=True)` on six classes, validating values that `struct.Struct.unpack()` had already produced as ints. 2. Coercion. Two fields were silently narrowed on construction, and both are load-bearing: `ImageHeader.flags` (a bare int from `unpack`) became `IMAGE_F`, and `ImageTLV.type` was resolved left-to-right through `Annotated[Union[IMAGE_TLV, VendorTLV, int], Field(union_mode=...)]`. 3. `VendorTLV.__get_pydantic_core_schema__`, which existed only to expose (2) to pydantic. The actual range check was already plain Python in `__new__`. So `ImageTLVType` becomes a plain `IMAGE_TLV | VendorTLV | int`, and the coercion becomes `_narrow_tlv_type()`, which walks the union in declaration order and lets each member's own constructor decide whether it accepts the value -- so the vendor range stays owned by `VendorTLV` instead of being restated. `ImageHeader.loads` narrows `flags` where the int is produced, and `ImageTLV.__post_init__` narrows `type`, which is what pydantic's coercion did and what the existing tests pin. msgspec was considered and rejected: this module does no CBOR/JSON de/serialisation (`struct` parses the binary), and `IMAGE_TLV | VendorTLV | int` is a union of three int-like types, which msgspec cannot decode anyway. ## Equivalence A characterisation script captured the pydantic behaviour before the change -- field types, every constructor coercion, `IMAGE_F` handling of undeclared bits, length validation, equality/hash, and the full `str()` of both fixture images -- and its output is byte-for-byte identical afterwards. There is no behavioural change at all. The 21 existing tests are unchanged except `test_tlv_type_union_order`, which drove pydantic's `TypeAdapter` directly and now drives `_narrow_tlv_type`; its vendor case is strengthened, since asserting `isinstance(result, int)` was also true of the `VendorTLV` it was meant to pin. `camas matrix` green on 3.10-3.14; `mcuboot.py` at 100% coverage; the integration suite passes 229/229. Note this does not yet make pydantic unreachable: `smpclient/__init__.py` still catches `pydantic.ValidationError`, which is intrinsic to smp 4.x and goes away with the `screaming-goblin` port (intercreate/smpmgr#103). pydantic is not declared in `pyproject.toml` and never was, so nothing changes there. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
JPHutchins
force-pushed
the
refactor/133-mcuboot-namedtuple
branch
from
August 28, 2026 19:59
5d6e5d8 to
095d97d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Warning
LLM Disclosure
This PR was authored by
claude-opus-5[1m]on behalf of @JPHutchins, who asked to pick thescreaming-goblinwork back up at #133 and to treat their CLAUDE.md as law. An earlier revision usedNamedTuple; @JPHutchins judged that it jumped through too many hoops and asked for stdlib dataclasses instead, which is what this is.Closes #133.
smp's
screaming-goblinmoves off pydantic to msgspec, sopip install smpwill stop pulling pydantic in. smpclient never declared pydantic itself — it has been relying on it arriving transitively via smp 4.x. Rather than declare a dependency we want gone, this removes the need for it.Every class keeps
@dataclass(frozen=True); only the import moves. That leavesImageTLVValue.__post_init__,ImageInfo._map_tlv_type_to_value'scached_property, and the genericImageTLVInfo[T]exactly as they were — and the public types untouched, sotlvsis still alist.What pydantic was actually doing
Three things, and only one was interesting:
@dataclass(frozen=True)on six classes, validating valuesstruct.Struct.unpack()had already produced as ints.ImageHeader.flags(a bare int fromunpack) becameIMAGE_F, andImageTLV.typeresolved left-to-right throughAnnotated[Union[IMAGE_TLV, VendorTLV, int], Field(union_mode="left_to_right")].VendorTLV.__get_pydantic_core_schema__— existing only to expose (2) to pydantic. The real range check was already plain Python in__new__.So
ImageTLVTypebecomes a plainIMAGE_TLV | VendorTLV | int, and the coercion becomes an explicit function that walks the union in declaration order, letting each member's own constructor decide whether it accepts the value — so the vendor range stays owned byVendorTLVrather than being restated:ImageHeader.loadsnarrowsflagswhere the int is produced;ImageTLV.__post_init__narrowstype, which is what pydantic's coercion did and what the existing tests pin.msgspec was considered and rejected: this module does no CBOR/JSON de/serialisation (
structparses the binary), andIMAGE_TLV | VendorTLV | intis a union of three int-like types, which msgspec cannot decode anyway.Equivalence — no behavioural change at all
A characterisation script captured the pydantic behaviour before the change — field types, every constructor coercion,
IMAGE_Fhandling of undeclared bits, length validation, equality/hash, and the fullstr()of both fixture images.Its output is byte-for-byte identical afterwards. Empty diff.
The 21 existing tests are unchanged except
test_tlv_type_union_order, which drove pydantic'sTypeAdapterdirectly. It now drives_narrow_tlv_type, and its vendor case is strengthened — the original assertedisinstance(result, int), which is also true of theVendorTLVit was meant to pin, so it could not actually fail if narrowing broke.Verification
camas matrixgreen on Python 3.10–3.14 (format, lint, mypy, pyright, tests)mcuboot.pyat 100% coverage (242 statements, 28 branches, zero missed); total 93.77%Scope note
This does not yet make pydantic unreachable —
smpclient/__init__.pystill catchespydantic.ValidationError, which is intrinsic to smp 4.x and goes away with thescreaming-goblinport. So #133's original "done whengrep -r pydantic src testsis empty" is only reachable after the port.pydantic is not declared in
pyproject.tomland never was — it arrives only assmpclient → smp → pydantic— so nothing changes there, and the port will not need to declare it either.Why not NamedTuple (the earlier revision of this PR)
Measured on 3.10 and 3.14:
NamedTupleMultiple inheritance with NamedTuple is not supported__new__in aNamedTuplebody__new__in a subclass of oneSo NamedTuple needed two private base classes to get constructor coercion back, had to leave
ImageTLVInfo(Generic[T])a dataclass anyway (3.10), and lostcached_property(no instance__dict__on a tuple subclass). It also forcedtlvsto atupleto avoid a NamedTuple holding a mutable list. Stdlib dataclasses need none of that:__post_init__andcached_propertyboth work on a frozen dataclass, verified on both versions.