Skip to content

ci: add E2E inventory and retry reporting - #291

Draft
piotr-iohk wants to merge 11 commits into
mainfrom
codex/264-inventory-shadow
Draft

piotr-iohk wants to merge 11 commits into
mainfrom
codex/264-inventory-shadow

Conversation

@piotr-iohk

@piotr-iohk piotr-iohk commented Nov 3, 2026 •

Copy link
Copy Markdown
Collaborator

Part of #264.

Companion: synonymdev/bitkit-ios#1199

This PR supplies the inventory and attempt reporting consumed by the smaller iOS gate. The iOS repository owns path policy and automatic selection. Android's workflow selection and legacy retry behavior stay unchanged.

Description

  • Supplies stable case IDs and backend/disposition metadata to the iOS-owned selector. No iOS source paths or app selection rules live in this repository.
  • Inventories all 69 existing declarations, including fixtures, manual cases and explicit gaps. The executable iOS nightly set is 42 local, six staging and six migration cases.
  • Allows up to three attempts, stops on complete success and retains first-failure evidence. Exact case coverage and app/E2E/artifact identities are required for a pass.
  • Loads selected spec files as one sequential WDIO worker per simulator and adds an optional immutable source pin for migration fixture preparation.
  • Removes the earlier replacement smoke implementation. The final change uses existing test assertions and has no shadow rollout.

Merge this producer before its iOS consumer. The consumer pins the reusable fixture workflow so development branches can be validated before merge.

Validation

  • 99 shared unit/contract tests and 29 iOS policy tests passed, covering inventory, actual app paths and policy drift, retries/recovery, missing or stale reports, installed WDIO scheduling and fixture-source pinning.
  • Inventory validation and TypeScript checking passed.
  • Cross-repository workflow contract checks and actionlint passed, including 19 aggregate-status scenarios.
  • Independent implementation review completed with source findings corrected.

The runner contract tests use controlled fixtures; they are not hosted Appium evidence. The initial hosted five-case gate passed first attempt at E2E 39f8964 / iOS f0a4aa3 (run); that is historical runtime evidence before policy relocation. The current hosted policy check, full lanes and optional cost comparison are recorded separately. Existing iOS mainnet/skip gaps are documented, not counted as passes.

Current full-suite validation

Validation is in progress at iOS 2117bd2bd0ad2997c2b4763f6fa4adb70cc94d61 / E2E fce4fa10317fb595efeadd48a9af8dd8763545d0: full one-shard run 41734988107. The matched two-shard run follows sequentially at the same revisions. Neither full coverage nor cost improvement is claimed yet; both PRs remain drafts.

The diagnostic run 41727376871 at iOS 0eb027d / E2E ee9e8be exposed a legacy RN sweep-fixture navigation error: Back after an already-closed success sheet backgrounded the source app. The fix retains the wallet-home assertion. It also exposed inadequate 40-minute budgets for full packs. Expanded local and six-case staging packs now allow 90-minute attempts within 285-minute jobs; local backends expire after five hours and are destroyed during cleanup. Base-sized local and individual migration cases retain 40-minute attempts / 135-minute jobs. All still allow up to three attempts and stop on success.

Before cancellation of that superseded diagnostic run, native restore passed first attempt and native upgrade passed second attempt; the upgrade's first source-onboarding failure is retained. Seven local and four staging cases completed, but both packs correctly reported incomplete after cancellation. All four RN consumers were blocked by the fixture failure. These are partial diagnostic results, not a full-suite pass. Artifact upload and the local backend teardown step completed.

The corrected runner/selector passed 101 shared tests and 30 policy tests; timeout tests exercise termination, retries and a sealed time budget. Workflow/actionlint checks passed after the staging correction. Independent follow-up review found no source gaps. Historical native-unit evidence remains at its original pins after reviewing the complete intervening delta.

Models used

  • Planning/scoping: gpt-6-astra (reasoning: high), gpt-6-astra (reasoning: xhigh)
  • Implementation: gpt-6-astra (reasoning: high), gpt-6-astra (reasoning: xhigh)
  • Review: gpt-6-astra (reasoning: high)

@piotr-iohk piotr-iohk changed the title ci: add iOS risk selection and retry reporting ci: add E2E inventory and retry reporting Nov 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant