Repository navigation
ci: shrink iOS gate and run full nightly coverage - #1199
Draft
piotr-iohk wants to merge 17 commits into
Draft
piotr-iohk wants to merge 17 commits into
piotr-iohk wants to merge 17 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of synonymdev/bitkit-e2e-tests#264.
Dependency: synonymdev/bitkit-e2e-tests#291
This PR replaces the broad iOS PR selection with five existing base cases plus automatically selected risk coverage, and runs the full applicable suite nightly.
Description
send_1,send_2,lightning_1,transfer_2andboost_2as the base gate. The app-owned.github/e2e-policy.jsonmaps actual feature paths and explicit services to existing cases. Named wallet/lifecycle/build dependencies select all local and staging cases on PRs. Unclassified paths fail cheap validation before builds.e2e-tests.ymlowns local/staging and its requirede2e-status. The independently schedulede2e_migration.ymlowns migration and its result.Merge the E2E producer first, then this consumer after branch validation. The pinned migration fixture workflow supports premerge full validation without relying on new inputs already being on
main.Design
N/A — no UI changes.
Preview
N/A
QA Notes
Setup
Use the workflow from
codex/264-inventory-shadowand explicitly select the E2E branch of the same name. The branch name is historical; its final tree has no shadow workflow.Docs/e2e-ci.mdcontains executable dispatch commands. Local cases use isolated Namespace backends; staging/migration use existing remote-regtest fixtures. All selected app and test revisions are recorded in the selection artifact.Journeys
N/A — no user-visible behaviour change.
Manual Tests
N/A
Automated Checks
e2e-tests.yml— smaller base selection, automatic risk coverage, full nightly lanes and required aggregate (after merge).Evidence
The earlier hosted iOS policy check passed all policy validation/tests and produced the actual PR selection preview, using PR merge
7513f821a0e157859453a161d4a19797136d486aand E2Eee9e8bed6147ce0cc6ccb446e7df598e9614ff31. The shared inventory/reporting check also passed. This verifies selection, not device execution.Inventory validation, TypeScript checks, all 99 shared tests, 29 iOS policy tests and workflow validation passed at E2E
ee9e8bed6147ce0cc6ccb446e7df598e9614ff31/ iOS0eb027de7f2d24dd9e1a4134552c17a14997089e. Policy tests cover the actual tree, exact domain cases, caller dependencies (scanner/backup/Keychain/payment flows), renames and unknown/stale paths. Independent review verified the correction.The actual-source audit changes full selection from 347/619 Swift files (all 76 services) to 28/619 (three explicitly shared services). This is a selection audit, not a measured billing reduction.
The initial five-case hosted run passed all five on its first attempt at iOS
f0a4aa3/ E2E39f8964: run 41709148174. Test execution was 27m36s, Mac job about 30 minutes, complete workflow about 36 minutes. This predates policy relocation; its coverage is not relabeled as current-head execution.The earlier native-unit invocation built the app and recorded 2,969 passes, zero failures and five skips at iOS
5c79e56aa6e31c2fd197fd90bf926818ccf56153, on the local iOS 26.0 simulator. Independent review retained this evidence after inspecting the entire intervening delta: native source, project/build inputs, dependencies and unit-test selection are unchanged. This is historical native evidence, not a current hosted E2E pass.Gaps
Hosted full-lane execution remains to be recorded before cutover; the cheap current-PR selection preview is checked separately. The five-case runtime pass remains at its original pins. The two-shard cost comparison has not run; no elapsed-time or billing improvement is claimed. Five existing skipped cases and six unsupported iOS mainnet cases remain documented gaps.
Current full-suite validation
At iOS
94088e98, the regular E2E workflow owns 42 local and six staging cases. Migration has an independent nightly andrelease-*push workflow, app build and result. The E2E workflow has no migration call or dependency. A regression check enforces that boundary.The hosted one-shard baseline at iOS
5ba352f7and E2Eee7a429passed all 54 cases under the preceding combined orchestration. Its 42 local cases passed first try in 95m01s. Observed Namespace usage was 96.57 Mac job-minutes and 95.93 backend minutes, estimated at $6.56 prepaid-equivalent or $9.84 overage. All six migrations passed, including passphrase on its first attempt. BaselineThe first two-shard dispatch mistakenly repeated migration and was cancelled. Its additional estimated compute was $1.55 prepaid-equivalent or $2.32 overage. The replacement selected 42 local cases in two shards and six staging cases at the identical revisions. Its selection artifact confirms
migration: false. Only local elapsed time and paid Namespace compute are comparable between the two runs. Cancelled dispatch · Regular E2E comparisonThe preceding run restored the correct 449,502 sats in RN, but its upgrade and passphrase tests could not read the nested amount on iOS. The corrected test reads the balance from its parent accessibility label. A local iOS simulator using the hosted target build and funded RN fixture passed the passphrase upgrade, including balances, history and tags after relaunch. Preceding run
The previous full run passed all six staging cases on the first attempt and five of six migration cases within three attempts. All 12 RN source wallets were prepared. The RN passphrase case failed. One recording showed the correct 449,502 sats behind a USDT introduction. Another showed a truncated passphrase and an empty wallet before the upgrade. The harness now checks the exact passphrase and funded RN balance before installing the native app, then dismisses the introduction before verifying balances and history. Migration job
The previous local job reached its 90-minute limit three times, with 37, 38 and 39 of 42 cases passing on the respective attempts. Security and transfer each passed on a subsequent attempt. The first PIN reset showed a regtest log-deletion error and returned to home instead of onboarding. The transfer recording showed QuickPay covering the activity assertions. The security assertion remains in the suite. The transfer case handles the one-time introduction and still verifies the balance and both activity rows. Local job
That unsuccessful run consumed 271.72 Namespace Mac job-minutes and 271.00 observed backend minutes. Published rates imply $18.47 prepaid-equivalent or $27.71 overage for compute. The estimate excludes storage, credits, rounding and account discounts. Run · Namespace pricing
The initial diagnostic also exposed a redundant Android Back after RN send success that backgrounded the source wallet. Its correction was exercised by all three sweep wallets in the next run. Initial diagnostic
Models used
gpt-6-astra(reasoning:high),gpt-6-astra(reasoning:xhigh)gpt-6-astra(reasoning:high),gpt-6-astra(reasoning:xhigh)gpt-6-astra(reasoning:high)