Design note surfaced while planning OTA (#27). The Phase 1 automatic bootloader rollback only catches an image that cannot get healthy (will not boot, or boots but never reaches the broker). It does not catch a healthy-but-misbehaving image: for example a logic bug the compiler never flagged that breaks the telemetry path while MQTT still connects. From the bootloader's view that image is fine, because it confirmed itself, so the automatic net will not revert it.
For that case we need an explicit, host-commanded rollback.
Decision
Keep confirming the image on health (mark-valid on the first MQTT connect, as in Phase 1), so a good image is locked in and never reverts by accident. Add a separate commanded-rollback path for the healthy-but-wrong case. (This is option A from the design discussion; option B, delaying mark-valid until the host approves, was rejected because a spurious reset during the pending window would revert a good image, which is bad for an always-on rack.)
Scope (firmware / device side, pairs with the Phase 3 MQTT control plane)
- Handle a rollback command received over MQTT.
- Revert by setting the boot partition to the previous OTA slot and rebooting. After a single update the previous firmware is still intact in the other slot, so this is a cheap operation (no re-download).
- Report both the running firmware version and the fallback-slot version (in hello or status), so the host knows what a rollback would land on before it issues one.
Caveats
- The fallback slot only holds the previous version until a second update overwrites it. The host must track this, which is why the device reports the fallback version.
- This is distinct from the automatic bootloader rollback; both mechanisms coexist (automatic catches "cannot get healthy", commanded catches "healthy but wrong").
Out of scope here
- The host-side logic that decides an image is misbehaving and issues the command (canary health-gate). Tracked separately for the fleet phases.
Part of #27. Pairs with the Phase 3 control plane. Relates to #31 (Phase 2).
Design note surfaced while planning OTA (#27). The Phase 1 automatic bootloader rollback only catches an image that cannot get healthy (will not boot, or boots but never reaches the broker). It does not catch a healthy-but-misbehaving image: for example a logic bug the compiler never flagged that breaks the telemetry path while MQTT still connects. From the bootloader's view that image is fine, because it confirmed itself, so the automatic net will not revert it.
For that case we need an explicit, host-commanded rollback.
Decision
Keep confirming the image on health (mark-valid on the first MQTT connect, as in Phase 1), so a good image is locked in and never reverts by accident. Add a separate commanded-rollback path for the healthy-but-wrong case. (This is option A from the design discussion; option B, delaying mark-valid until the host approves, was rejected because a spurious reset during the pending window would revert a good image, which is bad for an always-on rack.)
Scope (firmware / device side, pairs with the Phase 3 MQTT control plane)
Caveats
Out of scope here
Part of #27. Pairs with the Phase 3 control plane. Relates to #31 (Phase 2).