feat(simu): unify UI automation harness

[bultodepapas]

Fixes: N/A

Summary

This draft is the single collaborative replacement direction for the overlapping simulator UI automation work in #7337 and #7646.

It keeps the strongest parts of both:

• from Add TX16S UI harness #7337: a generic simulator harness, host-side session/CLI and scenarios, structured responses, input primitives, and framebuffer capture;
• from feat(simu): add opt-in Widget Studio automation hooks #7646: Windows-native support, direct simulator helpers, explicit key/touch transitions, switch/analog/telemetry injection, Lua reload, restart behavior, and capture synchronization.

It deliberately leaves out duplicate or product-specific surfaces: the WIDGET_STUDIO build option, simulator-only Lua simu API, append-only command-file transport, native PNG encoding, and MCP integration.

Delivery

This branch and this draft PR contain both the plan and implementation. There is no second implementation branch and no branch per phase.

• Technical baseline 0T passed at 0c07a3d648a929732190ab9b7378a7f6c272ab55.
• Phase 1 at 5139415071 adds the bounded SDL-independent protocol and state model.
• Phase 2 at c1be4bd5fb adds opt-in cross-platform non-blocking stdio transport and SDL-loop integration.
• Phase 3 at f6c104ea4c adds the dependency-free Python session/CLI lifecycle, strict status/describe discovery, and real first-frame readiness.
• Phase 4 at 164ff21dfe adds target-filtered key/rotary/touch primitives, release-all, real-LCD frame barriers, timed host composites, and cleanup fallback.
• Phase 5 at bfd59db236 adds render-complete RGB565 capture, static-screen LVGL invalidation, exclusive PPM publication, verified host-side PNG conversion, deterministic SHA-256 metadata, and focused coverage.
• Phase 6 at ada9972144 adds switch/analog/telemetry state injection, generation-observed Lua reload, deterministic warm restart, and isolated cold process restart.

The Phase 6 plan records 13 primary-source checks, rejected alternatives, explicit owner/thread boundaries, restart invariants, implementation subphases, and the V01–V12/L01–L16 verification matrix.

Contract review G0 remains open. It is required before this PR leaves draft or is merged; it does not block small reviewable implementation commits.

Phase 5 design

• Capture uses simuLcdBuf at a strictly newer simuLcdNotify, not SDL texture readback.
• A one-shot flag asks the firmware-owned LVGL loop to invalidate a static screen.
• The LCD callback performs one bounded copy into preallocated RGB565 storage; conversion and filesystem work run on one session worker.
• The native artifact is canonical P6 PPM, published without replacing an existing path.
• The dependency-free Python client strictly validates PPM, writes and independently decodes PNG, and records stable hashes and capture metadata.
• Capture is advertised only by tested 16-bit LCD targets. Native capture remains excluded from Emscripten/WASM.

Phase 6 design

• Switches stay on the SDL owner thread and use the existing simulator switch driver; analog overrides use one lock-free packed atomic per discovered ADC input.
• Telemetry and Lua requests cross into firmware context through fixed 16-entry SPSC request/completion queues, with at most two requests consumed per periodic UI cycle.
• Telemetry reuses the existing Lua telemetry path while temporarily and locally enabling explicit sensor allocation; repeated tuples update the same sensor.
• Lua reload succeeds only after the requested generation is observed in running; interpreter panic is a correlated failure.
• Warm restart joins firmware tasks, starts a new epoch before the first new frame can arrive, and preserves a process-monotonic display sequence.
• Cold restart fully reaps the old child, copies immutable settings and SD-card fixtures into a unique run directory, and returns a new session object.
• Automation startup skips interactive splash/calibration checks; normal simulator startup is unchanged. Native-only guards keep the runtime out of physical firmware and WASM builds.

Collaboration and credit

Thank you, @onliner10, for the generic harness and capture direction in #7337. This consolidation builds on that work and preserves its provenance in the plan, commit messages, and PR narrative. Phases 5 and 6 retain the portable artifact and reusable host-session direction while combining it with the synchronized state/lifecycle requirements explored in #7646.

@onliner10 and @pfeerick are explicitly invited to review the resulting behavior and protocol contract before this PR is marked ready. Neither source PR needs to be closed until the consolidated implementation demonstrates the retained behavior and maintainers agree on the direction.

No Co-authored-by trailer is added without contributor approval; coherent imported commits should retain their original authorship.

Current validation

• Phase 6 focused native suite: 38/38 SimuAutomation* tests pass under AddressSanitizer, covering protocol, capture/state, mailbox, analog, telemetry, Lua result serialization, and restart behavior.
• Phase 6 host suite: 57/57 protocol/session tests pass, including strict validation and cold-restart preflight/failure cleanup.
• The final Windows TX16S simu.exe compiles and links with Clang 22.
• A real TX16S run discovered 8 switches and 13 analog inputs and passed switch rules, analog set/replace/clear, telemetry create/update, and observed Lua generation 1 in running.
• Warm restart advanced epoch 1 → 2 and display sequence 2 → 4; cold restart launched a new PID after the old process and reader threads were fully reaped.
• Python compileall, strict formatting for the new native files, and git diff --check pass.
• Current-head Phase 6 GitHub Actions, firmware builds, documentation, packaging, and CodeRabbit passed 45/45.

This PR intentionally remains draft while G0 and the later scoped phases remain open.

[bultodepapas]

Correction to the delivery rule above: this effort now stays deliberately simple.

Implementation proceeds in this same draft PR and branch; there is no second implementation branch and no branch per phase. G0 remains an explicit review requirement before the PR leaves draft or is merged, but it does not block small implementation commits.

Phase 1 is now concrete at 5139415071686414f19fa6227e9bdd0835da8080: one pure bounded protocol/state module plus focused tests, with no SDL transport or runtime activation yet. Local validation passed 122/122 tests with ASan, and the complete GitHub test/build/documentation/packaging matrix is green.

@onliner10 and @pfeerick are invited to review this concrete slice and shape the next one. Credit and provenance from #7337 and #7646 remain explicit, and neither source PR needs to close until parity is demonstrated.

[bultodepapas]

Phase 2 is now published at c1be4bd5fb on this same branch and draft PR.

This slice adds the opt-in cross-platform stdio transport and SDL-loop activation while keeping the scope narrow: ping and stop work now; input, telemetry, Lua, restart, and framebuffer capture stay for later phases.

Validation completed:

• 129/129 gtests-radio tests passed with ASan in the official Linux environment;
• native Linux simulator built and passed real-pipe probes, including clean JSON-only stdout, EOF, malformed input, queue limits, paths with spaces, and dormant normal mode;
• parser/protocol/transport compiled on Windows with both Clang and MinGW under -Wall -Wextra -Werror;
• POSIX and Windows protocol vectors produced the same byte hash;
• strict documentation build and git diff --check passed.

The implementation intentionally does not add a permanent Python client yet; a disposable probe was enough to validate this transport slice. @onliner10 and @pfeerick remain invited to review the contract and retained behavior before the draft is marked ready.

[bultodepapas]

Phase 3 slice 3A is now published at 49f2e3db9b on this same branch and draft PR.

This keeps the scope intentionally narrow: a dependency-free, binary-pipe Python session and minimal edgetx-ui probe for the native commands that exist today (ping and stop). It adds strict ID/JSON validation, separate bounded stderr draining, command deadlines, crash detection, and guaranteed process/thread cleanup without select or startup sleeps.

Validation:

• 22/22 focused tests pass with warnings treated as errors on Windows Python 3.11 and official-image Linux Python 3.10;
• the suite includes malformed/partial stdout, wrong/repeated IDs, stderr flooding, command errors, crash, timeout, terminate fallback, CLI failure exit, and 100 subprocess cycles on each host;
• the real Linux TX16S simulator built and completed 100 start/ping/stop cycles with exit code zero and no live reader thread;
• the exact CLI also passed end-to-end with SDL offscreen and paths containing spaces;
• strict docs and git diff --check pass.

Credit: the reusable host-session and CLI direction comes from #7337 by @onliner10. I reviewed that implementation, but this protocol-v1/Windows-safe lifecycle is a substantial rewrite with no coherent source block copied; the provenance ledger records that distinction. A co-author trailer remains intentionally absent unless approved.

Phase 3 remains open for a real Windows-simulator run, explicit kill-fallback injection, and status/frame/capability readiness. @onliner10 and @pfeerick are invited to review this concrete host boundary before those pieces are added.

CI update: the complete GitHub test, firmware-build, documentation, and packaging matrix passed for this head (45/45).

[bultodepapas]

Phase 3 is now complete at f6c104e on this same branch and draft PR.

This final slice adds only the readiness boundary needed before inputs: bounded status/describe responses, target/LCD/command/capability validation, and startup completion after the simulator owns a real first LCD frame. It intentionally does not advertise or implement input, capture, telemetry, Lua, or restart yet.

Validation completed:

• 29/29 host tests pass with warnings as errors on Windows Python 3.11 and official-image Linux Python 3.10, including 100 lifecycle cycles on each host and forced kill-then-wait cleanup;
• 131/131 native TX16S tests pass;
• the real Linux simulator completes 100 readiness/stop cycles without an orphan;
• the exact current Windows simulator build completes ping/describe/first-frame-ready/stop with paths containing spaces and exit code zero;
• git diff --check passes.

Thank you again, @onliner10 and @pfeerick. This slice combines the reusable host-session direction from #7337 with the guarded epoch/first-frame model from #7646. Please review the resulting boundary and help shape Phase 4 inputs; G0 remains open before this draft can be marked ready.

CI update: the complete current-head GitHub Actions and CodeRabbit matrix passed 45/45 for f6c104e.

[bultodepapas]

Hi @onliner10 — Phase 4 of the consolidated simulator UI automation work is now available in this PR.

Thank you again for the work and direction you established in #7337, especially the reusable host-side harness and action model. Those ideas have materially shaped this consolidation, and I have tried to keep that provenance clear in both the plan and the PR history.

I would genuinely value your review of the protocol, host API, and retained behavior before this draft is marked ready. Please feel free to challenge the design or suggest a simpler direction where appropriate.

If you are comfortable with commit-level attribution for the parts substantially derived from your work, I would also be happy to add a Co-authored-by trailer. I will not rewrite commit history or use an email address without your approval; if you would like that attribution, please confirm the GitHub-associated email you prefer (a GitHub noreply address is completely fine).

No pressure either way — your review and feedback would already be greatly appreciated.

[bultodepapas]

Phase 5 is now published at bfd59db236 on this same branch and draft PR.

This slice adds one focused capability: render-complete framebuffer capture. It combines the portable PPM and host-conversion direction from #7337 with the static-screen invalidation requirement explored in #7646. Native capture waits for a strictly newer RGB565 framebuffer, publishes without replacing an existing artifact, and keeps file I/O out of the LCD notification. The Python client validates the PPM, converts and independently decodes PNG, and writes deterministic SHA-256 metadata.

Validation is complete:

• 34/34 focused native tests pass under AddressSanitizer;
• 49/49 host tests pass;
• the Windows TX16S simulator compiles, links, and captures through real binary pipes;
• 20 static captures are byte-identical with increasing display sequences;
• an ENTER interaction produces a different framebuffer hash;
• Unicode-and-space paths work, metadata hashes match, and no temporary artifacts remain;
• the complete current-head GitHub matrix passes 45/45.

Thank you, @onliner10, for the original harness and capture direction in #7337. I would especially value your review of the freshness boundary, artifact contract, and host API. @pfeerick, your review of the simulator/LVGL integration would also be very welcome. G0 remains open, so the PR stays draft.

[bultodepapas]

Phase 6 is now published at ada9972144 on this same branch and draft PR. This slice completes the state/lifecycle layer without adding another control surface: discovered switches and analogs use the existing simulator drivers, firmware-owned telemetry and Lua work cross a bounded SPSC mailbox, warm restart completes on the first frame of a new epoch, and cold restart returns a new process backed by fresh fixture copies. Automation uses deterministic no-splash startup, while normal simulator behavior and physical/WASM builds remain outside the change. Final local validation: - 38/38 focused native SimuAutomation* tests pass under AddressSanitizer; - 57/57 host protocol/session tests pass; - the Windows TX16S simulator compiles and links; - a real run passed switch and analog control, telemetry create/update, generation-observed Lua reload, warm restart, and cold restart with a new PID and a fully reaped old child; - formatting, compileall, and git diff --check pass. Thank you again, @onliner10, for the reusable harness/session direction established in #7337. Phase 6 deliberately keeps that direction while integrating the state-injection and restart goals from #7646. I would genuinely value your review of the host API, mailbox boundary, and restart semantics, and any simplification or correction you would like to propose. @pfeerick, review of the firmware/UI hook placement would also be very welcome. Current-head GitHub Actions, firmware builds, documentation, packaging, and CodeRabbit passed 45/45. G0 remains open, so the PR stays draft.

[pfeerick]

Just a friendly FYI ... tell your AI assistant to be less eager to tag and mention people... it's the quickest way to get yourself ignored if you are pinging people every couple of hours. Commits generally also speak for themselves if well written and described, so iteratively updating the initial PR body post would have been more than sufficient here, and better than individual progress comments with each commit, as the posts here also flow into things like discord channels, meaning there are will be several effectively "noise" posts there about this PR.