Skip to content

refactor(mcuboot): replace pydantic dataclasses with stdlib dataclasses - #136

Merged
JPHutchins merged 1 commit into
mainfrom
refactor/133-mcuboot-namedtuple
Aug 28, 2026
Merged

refactor(mcuboot): replace pydantic dataclasses with stdlib dataclasses#136
JPHutchins merged 1 commit into
mainfrom
refactor/133-mcuboot-namedtuple

Conversation

@JPHutchins

@JPHutchins JPHutchins commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator

Warning

LLM Disclosure

This PR was authored by claude-opus-5[1m] on behalf of @JPHutchins, who asked to pick the screaming-goblin work back up at #133 and to treat their CLAUDE.md as law. An earlier revision used NamedTuple; @JPHutchins judged that it jumped through too many hoops and asked for stdlib dataclasses instead, which is what this is.

Closes #133.

smp's screaming-goblin moves off pydantic to msgspec, so pip install smp will stop pulling pydantic in. smpclient never declared pydantic itself — it has been relying on it arriving transitively via smp 4.x. Rather than declare a dependency we want gone, this removes the need for it.

Every class keeps @dataclass(frozen=True); only the import moves. That leaves ImageTLVValue.__post_init__, ImageInfo._map_tlv_type_to_value's cached_property, and the generic ImageTLVInfo[T] exactly as they were — and the public types untouched, so tlvs is still a list.

What pydantic was actually doing

Three things, and only one was interesting:

  1. @dataclass(frozen=True) on six classes, validating values struct.Struct.unpack() had already produced as ints.
  2. Coercion — two fields silently narrowed on construction, and both are load-bearing: ImageHeader.flags (a bare int from unpack) became IMAGE_F, and ImageTLV.type resolved left-to-right through Annotated[Union[IMAGE_TLV, VendorTLV, int], Field(union_mode="left_to_right")].
  3. VendorTLV.__get_pydantic_core_schema__ — existing only to expose (2) to pydantic. The real range check was already plain Python in __new__.

So ImageTLVType becomes a plain IMAGE_TLV | VendorTLV | int, and the coercion becomes an explicit function that walks the union in declaration order, letting each member's own constructor decide whether it accepts the value — so the vendor range stays owned by VendorTLV rather than being restated:

def _narrow_tlv_type(value: int) -> ImageTLVType:
    """Return `value` as the leftmost `ImageTLVType` that accepts it."""
    for tlv_type in (IMAGE_TLV, VendorTLV):
        try:
            return tlv_type(value)
        except ValueError:
            continue
    return value

ImageHeader.loads narrows flags where the int is produced; ImageTLV.__post_init__ narrows type, which is what pydantic's coercion did and what the existing tests pin.

msgspec was considered and rejected: this module does no CBOR/JSON de/serialisation (struct parses the binary), and IMAGE_TLV | VendorTLV | int is a union of three int-like types, which msgspec cannot decode anyway.

Equivalence — no behavioural change at all

A characterisation script captured the pydantic behaviour before the change — field types, every constructor coercion, IMAGE_F handling of undeclared bits, length validation, equality/hash, and the full str() of both fixture images.

Its output is byte-for-byte identical afterwards. Empty diff.

The 21 existing tests are unchanged except test_tlv_type_union_order, which drove pydantic's TypeAdapter directly. It now drives _narrow_tlv_type, and its vendor case is strengthened — the original asserted isinstance(result, int), which is also true of the VendorTLV it was meant to pin, so it could not actually fail if narrowing broke.

Verification

  • camas matrix green on Python 3.10–3.14 (format, lint, mypy, pyright, tests)
  • mcuboot.py at 100% coverage (242 statements, 28 branches, zero missed); total 93.77%
  • Integration suite 229 passed, 0 failures

Scope note

This does not yet make pydantic unreachable — smpclient/__init__.py still catches pydantic.ValidationError, which is intrinsic to smp 4.x and goes away with the screaming-goblin port. So #133's original "done when grep -r pydantic src tests is empty" is only reachable after the port.

pydantic is not declared in pyproject.toml and never was — it arrives only as smpclient → smp → pydantic — so nothing changes there, and the port will not need to declare it either.

Why not NamedTuple (the earlier revision of this PR)

Measured on 3.10 and 3.14:

3.10 3.14
Generic NamedTuple FAILMultiple inheritance with NamedTuple is not supported OK
__new__ in a NamedTuple body FAIL FAIL
__new__ in a subclass of one works works

So NamedTuple needed two private base classes to get constructor coercion back, had to leave ImageTLVInfo(Generic[T]) a dataclass anyway (3.10), and lost cached_property (no instance __dict__ on a tuple subclass). It also forced tlvs to a tuple to avoid a NamedTuple holding a mutable list. Stdlib dataclasses need none of that: __post_init__ and cached_property both work on a frozen dataclass, verified on both versions.

@JPHutchins

Copy link
Copy Markdown
Collaborator Author

OK, we can drop 3.10 soon anyway, it's almost EoL

smp's `screaming-goblin` moves off pydantic to msgspec, so `pip install smp`
will stop pulling pydantic in. smpclient never declared pydantic itself -- it
has been relying on it arriving transitively via smp 4.x -- so rather than
declare a dependency we want gone, remove the need for it. Closes #133.

Every class keeps `@dataclass(frozen=True)`; only the import moves. That keeps
`ImageTLVValue.__post_init__`, `ImageInfo._map_tlv_type_to_value`'s
`cached_property`, and the generic `ImageTLVInfo[T]` exactly as they were, and
leaves the public types untouched -- `tlvs` is still a `list`.

## What pydantic was actually doing

Three things, and only one of them was interesting:

1. `@dataclass(frozen=True)` on six classes, validating values that
   `struct.Struct.unpack()` had already produced as ints.
2. Coercion. Two fields were silently narrowed on construction, and both are
   load-bearing: `ImageHeader.flags` (a bare int from `unpack`) became
   `IMAGE_F`, and `ImageTLV.type` was resolved left-to-right through
   `Annotated[Union[IMAGE_TLV, VendorTLV, int], Field(union_mode=...)]`.
3. `VendorTLV.__get_pydantic_core_schema__`, which existed only to expose (2)
   to pydantic. The actual range check was already plain Python in `__new__`.

So `ImageTLVType` becomes a plain `IMAGE_TLV | VendorTLV | int`, and the
coercion becomes `_narrow_tlv_type()`, which walks the union in declaration
order and lets each member's own constructor decide whether it accepts the
value -- so the vendor range stays owned by `VendorTLV` instead of being
restated. `ImageHeader.loads` narrows `flags` where the int is produced, and
`ImageTLV.__post_init__` narrows `type`, which is what pydantic's coercion did
and what the existing tests pin.

msgspec was considered and rejected: this module does no CBOR/JSON
de/serialisation (`struct` parses the binary), and `IMAGE_TLV | VendorTLV | int`
is a union of three int-like types, which msgspec cannot decode anyway.

## Equivalence

A characterisation script captured the pydantic behaviour before the change --
field types, every constructor coercion, `IMAGE_F` handling of undeclared bits,
length validation, equality/hash, and the full `str()` of both fixture images --
and its output is byte-for-byte identical afterwards. There is no behavioural
change at all.

The 21 existing tests are unchanged except `test_tlv_type_union_order`, which
drove pydantic's `TypeAdapter` directly and now drives `_narrow_tlv_type`; its
vendor case is strengthened, since asserting `isinstance(result, int)` was also
true of the `VendorTLV` it was meant to pin.

`camas matrix` green on 3.10-3.14; `mcuboot.py` at 100% coverage; the
integration suite passes 229/229.

Note this does not yet make pydantic unreachable: `smpclient/__init__.py` still
catches `pydantic.ValidationError`, which is intrinsic to smp 4.x and goes away
with the `screaming-goblin` port (intercreate/smpmgr#103). pydantic is not
declared in `pyproject.toml` and never was, so nothing changes there.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@JPHutchins
JPHutchins force-pushed the refactor/133-mcuboot-namedtuple branch from 5d6e5d8 to 095d97d Compare August 28, 2026 19:59
@JPHutchins JPHutchins changed the title refactor(mcuboot): replace pydantic dataclasses with NamedTuple refactor(mcuboot): replace pydantic dataclasses with stdlib dataclasses Aug 28, 2026
@JPHutchins
JPHutchins merged commit c04ca8d into main Aug 28, 2026
29 checks passed
@JPHutchins
JPHutchins deleted the refactor/133-mcuboot-namedtuple branch August 28, 2026 20:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remove pydantic: port mcuboot.py to NamedTuple

1 participant