Skip to content

Swift SDK: IdentityResolverSignIntegrationTests flakes on CI due to real-login-keychain writes #4026

Description

@Claudius-Maginificent

Expected Behavior

IdentityResolverSignIntegrationTests should pass reliably in CI regardless of the runner's keychain state.

Current Behavior

Two tests in IdentityResolverSignIntegrationTests intermittently fail on the self-hosted macOS CI runner (latte-ssh) with a keychain error thrown from WalletStorage.swift:95:

  • testResolverRejectsWrongPathBeforeSigningkeychainError(-60008)
  • testResolverSignsForConsistentBreadcrumbkeychainError(-60005)

Both tests call storage.storeMnemonic(mnemonic, for: walletId), which writes a real BIP-39 mnemonic to the host macOS login keychain via SecItemAdd — no kSecUseDataProtectionKeychain, no test-specific access group, no mock storage. On a headless/SSH CI session the login keychain write can fail nondeterministically. Two different OSStatus codes surfacing across the same run (-60008 vs -60005) is the signature of nondeterministic securityd state, not a deterministic logic error.

Confirmed as an environment flake, not a code defect:

  • Same tests pass on v4.1-dev runs on the same runner pool.
  • A clean rerun of the exact same CI job (run 28791155484, job 85376353259) passed with zero code changes (Swift SDK build / Swift SDK build + tests completed green in 9m19s).
  • The test file was introduced by feat: complete dashpay in platform wallet and swift example app #3841 (already on v4.1-dev) and is unrelated to whatever PR happens to trigger the run.

Possible Solution

Isolate these tests from the real login keychain, e.g.:

  • Use kSecUseDataProtectionKeychain: true with a dedicated test access group, or
  • Inject an in-memory/mock WalletStorage for these tests, or
  • throw XCTSkip(...) when the keychain is unavailable in a headless session, so CI success stops depending on runner keychain state.

Steps to Reproduce (for bugs)

  1. Run the Swift SDK CI job (Swift SDK build / Swift SDK build + tests) on the latte-ssh self-hosted runner enough times.
  2. Occasionally IdentityResolverSignIntegrationTests.testResolverRejectsWrongPathBeforeSigning and/or testResolverSignsForConsistentBreadcrumb fail with a keychainError thrown from WalletStorage.swift:95, while an immediate rerun with no code change passes.

Context

This flake blocked PR #3692's CI checks for reasons unrelated to that PR's own diff, costing review/merge time. Any other PR that happens to run the Swift SDK job on this runner is equally exposed.

Your Environment

🤖 Co-authored by Claudius the Magnificent AI Agent

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions