From b4125f35c93412292f72780e645bccf1e3facb39 Mon Sep 17 00:00:00 2001 From: "Shane B." Date: Thu, 13 Aug 2026 17:10:38 +0200 Subject: [PATCH 001/131] Add clean-room protocol specification for iCloud FindMy key export Specifies the network protocol for reading FindMy accessory key material directly from iCloud, removing the macOS VM from the export path. Six stages, in docs/findmy-export/. Written as a clean room. The only working implementation of this flow is SSPL-licensed with an exception granted to one other project, so it cannot be reused. These documents describe Apple's wire protocol -- endpoints, header sets, payload layouts, cryptographic constructions, state transitions -- which is fact rather than anyone's expression, and carry no type name, function name or code shape from any reference. The implementation must be written by someone who has not read those references; if something here is wrong, the fix is to amend the document. Four stages are verified against a live Apple account, by probes written from these documents rather than from any reference: 1 Authentication SRP handshake, server proof verified, 2FA, tokens 2 MobileMe delegate 11 iCloud service tokens 3 Keychain trust escrow record listing (read-only half) 4 CloudKit container, zones, 35 records fetched Stage 3's recovery and join, and Stage 5's decryption, are specified but unrun. Every claim is marked observed or derived, and open questions are listed rather than papered over. Two findings shape the design and are recorded prominently: - Signing in registers a device on the account, and that registration is what suppresses two-factor on later logins. It is the application's identity, not residue, and should be kept and labelled rather than churned. - Escrow records outlive the devices that created them, are shown in no Apple interface, and accumulate invisibly. They are the artefact that actually needs cleaning up. Also moves the native-Anisette proof-of-concept log out of an ignored scratch directory into docs/, and adds the CLAUDE.md stub that imports AGENTS.md. Co-Authored-By: Claude Opus 5 --- CLAUDE.md | 9 + docs/README.md | 22 + docs/anisette-native-android.md | 469 ++++++++++ docs/findmy-export/01-authentication.md | 956 +++++++++++++++++++++ docs/findmy-export/02-mobileme-delegate.md | 462 ++++++++++ docs/findmy-export/03-keychain-trust.md | 518 +++++++++++ docs/findmy-export/04-cloudkit.md | 625 ++++++++++++++ docs/findmy-export/05-pcs-decryption.md | 311 +++++++ docs/findmy-export/06-output.md | 151 ++++ docs/findmy-export/README.md | 287 +++++++ 10 files changed, 3810 insertions(+) create mode 100644 CLAUDE.md create mode 100644 docs/README.md create mode 100755 docs/anisette-native-android.md create mode 100644 docs/findmy-export/01-authentication.md create mode 100644 docs/findmy-export/02-mobileme-delegate.md create mode 100644 docs/findmy-export/03-keychain-trust.md create mode 100644 docs/findmy-export/04-cloudkit.md create mode 100644 docs/findmy-export/05-pcs-decryption.md create mode 100644 docs/findmy-export/06-output.md create mode 100644 docs/findmy-export/README.md diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 00000000..2d585447 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,9 @@ +# CLAUDE.md + +The rules for this repo live in [AGENTS.md](./AGENTS.md), so that every agent reads the same +file rather than two copies that drift apart. This one exists only to pull it in — Claude Code +loads `CLAUDE.md` automatically, and the line below imports the real thing. + +**Put new rules in `AGENTS.md`, not here.** + +@AGENTS.md diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 00000000..5c066d8c --- /dev/null +++ b/docs/README.md @@ -0,0 +1,22 @@ +# docs + +Longer-form documents that are neither rules nor setup instructions. + +The rules for changing this project live in [AGENTS.md](../AGENTS.md); how to build and test it +lives in [CONTRIBUTING.md](../CONTRIBUTING.md). What lands here is the other thing: protocol +specifications and investigation records — the material that is expensive to rediscover and has +no natural home in a source comment. + +| Document | What it is | +| --- | --- | +| [anisette-native-android.md](./anisette-native-android.md) | Proof-of-concept log for running Apple's ADI libraries in-process on Android, and the measurements that came out of it. Historical; the implementation shipped. | +| [findmy-export/](./findmy-export/) | Clean-room specification of the protocol for reading FindMy accessory keys from iCloud without a Mac. Stage 1 written; five stages to go. | + +## If you add a document here + +Add a row above in the same commit. A list that nothing checks goes stale silently, which is +the failure mode [AGENTS.md rule 10](../AGENTS.md) exists to prevent. + +Records of investigations are worth keeping even when they describe work that has since +shipped, provided they say so at the top. What makes them worth the disk is the measurements and +the dead ends — the things a reader would otherwise pay for twice. diff --git a/docs/anisette-native-android.md b/docs/anisette-native-android.md new file mode 100755 index 00000000..f52ac8c6 --- /dev/null +++ b/docs/anisette-native-android.md @@ -0,0 +1,469 @@ +# PoC: native Anisette on Android — what is verified + +> **Status: historical record, kept for its measurements.** This is the proof-of-concept log +> that preceded the app's native Anisette implementation, and it is retained because the numbers +> in it — download sizes, load timings, the stub surface, the eleven entry points — cost real +> device time to obtain and would be expensive to rediscover. +> +> It is a lab notebook, not a description of the shipped code. Two kinds of reference in it have +> since moved on: +> +> - **The measurement scripts were throwaway** and are not in the tree. Each is described well +> enough to rewrite, and `python -m zipfile` plus an ELF parser is the whole toolkit. +> - **The probe and its tests were implemented properly** and renamed. See +> `app/src/main/cpp/adi.cpp`, `app/src/main/java/.../anisette/`, and the tests in +> `app/src/androidTest/.../anisette/`. +> +> For the authentication flow this Anisette work feeds, see +> [findmy-export/01-authentication.md](./findmy-export/01-authentication.md). + +2026-08-10. Everything below was run from a Windows desktop with no Android device involved, +so it verifies the parts that do not need one. It stops exactly where a device becomes +necessary. + +Reference implementation: [Dadoum/Provision](https://github.com/Dadoum/Provision) — +`lib/provision/adi.d` and `docs/ADI.md`, plus +[anisette-v3-server](https://github.com/Dadoum/anisette-v3-server) `source/app.d` for the +server wrapper around it. + +--- + +## 1. The libraries are still published, and reachable + +``` +HTTP/1.1 200 OK +Content-Length: 142139820 +Content-Type: application/vnd.android.package-archive +``` + +`https://apps.mzstatic.com/content/android-apple-music-apk/applemusic.apk` — Apple's own CDN, +no account, no auth. Nothing is redistributed by us, which is the same position every public +Anisette server is in. + +## 2. We do not need to download 142 MB + +`anisette-v3-server` downloads the whole APK, unzips it, and keeps two files. That is fine for +a server doing it once and unacceptable on a phone. + +A zip's central directory is at the end, so HTTP range requests can read the directory, locate +the two members, and fetch only those. Measured: + +``` +APK is 142 MB +central directory: 272 kB at offset 141867668 + lib/arm64-v8a/libCoreADI.so: 1.79 MB (deflated), ELF=True + lib/arm64-v8a/libstoreservicescore.so: 2.39 MB (deflated), ELF=True + +Downloaded 2.6 MB of 142 MB (1.8%) +``` + +**1.8% of the bytes.** On a phone that is the difference between a feature people accept and +one they cancel. Script: `scratchpad/poc_fetch_adi.py`. + +The APK also carries `armeabi-v7a`, `x86` and `x86_64`, so the ABI is a runtime choice — on a +device, simply the device's own. + +## 2a. But two libraries is not the real number — `DT_NEEDED` is + +2026-08-10. `libstoreservicescore.so` does not stand alone: + +``` +libstoreservicescore.so NEEDED libmediaplatform.so, libCoreFoundation.so, libc++_shared.so, ... +libCoreADI.so NEEDED liblog, libdl, libc <- clean +``` + +`dlopen()` follows `DT_NEEDED`. Dadoum's hand-written loader does not, which is why +anisette-v3-server gets away with keeping two files. The full transitive closure, measured +(`scratchpad/poc_closure.py`): + +``` +libstoreservicescore.so 2.39 MB -> libmediaplatform, libCoreFoundation, libc++_shared +libmediaplatform.so 2.36 MB -> libcurl, libCoreFoundation, libicu{i18n,uc,data}_sv_apple, + libdispatch, libxml2, libc++_shared +libCoreFoundation.so 2.01 MB -> libicu*, libcurl, libxml2, libBlocksRuntime, libdispatch +libicudata_sv_apple.so 10.23 MB +libcurl.so 3.40 MB libicui18n_sv_apple.so 2.60 MB +libicuuc_sv_apple.so 1.75 MB libxml2.so 2.00 MB +libc++_shared.so 1.03 MB libdispatch.so 0.38 MB +libBlocksRuntime.so 0.01 MB + +Closure: 11 libraries, 28.2 MB unpacked, 11.3 MB downloaded (7.9% of the APK) +``` + +Apple shipped their own CoreFoundation, libdispatch and ICU for Android, and ADI sits on top +of them. All four ABIs carry the complete set, so no ABI is a special case. + +This splits the feature into two shapes: + +| Path | Cost | Needs | +| --- | --- | --- | +| **A — honour `DT_NEEDED`** | 11.3 MB download, 28 MB on disk, 11 libraries | nothing; plain `dlopen` in dependency order | +| **B — ignore them** (Provision's approach) | 2.6 MB, 2 libraries | the manual ELF mapper | + +B is the shipping form. A is the one worth building first, because it needs no mapper and so +can answer *"do the eleven ADI calls work on Android at all"* immediately. If A works, the +mapper stops being a blocker and becomes a size optimisation — a far better project shape than +writing an ELF loader and only then finding out. + +Ordering is what makes A work without any linker trickery: bionic resolves `DT_NEEDED` against +already-loaded libraries by SONAME, so opening the closure bottom-up satisfies each library's +dependencies before it is opened. Nothing is on the default search path and nothing needs to be. + +## 3. `libCoreADI.so` is built to be loaded by an Android JVM + +It exports exactly three symbols: + +``` +JNI_OnLoad +cvu8io98wun +vdfut768ig +``` + +`JNI_OnLoad` is the entry point Android's `System.loadLibrary` calls. This library expects to +live in a JVM process. That is the whole premise of doing this in-app rather than over the +network, confirmed at the binary level. + +The two scrambled names are CoreADI's real entry points, and per `docs/ADI.md` they are +deliberately hard to reverse. **We do not call them.** We call the wrappers exported by +`libstoreservicescore.so`, which call into them. + +## 4. All eleven ADI entry points exist in today's build + +From `lib/provision/adi.d`, with signatures. Every one verified present in the +`libstoreservicescore.so` pulled today: + +| Function | Symbol | Signature | +| --- | --- | --- | +| `ADILoadLibraryWithPath` | `kq56gsgHG6` | `int(const char*)` | +| `ADISetAndroidID` | `Sph98paBcz` | `int(const char*, uint)` | +| `ADISetProvisioningPath` | `nf92ngaK92` | `int(const char*)` | +| `ADIProvisioningErase` | `p435tmhbla` | `int(ulong)` | +| `ADISynchronize` | `tn46gtiuhw` | `int(ulong, ubyte*, uint, ubyte**, uint*, ubyte**, uint*)` | +| `ADIProvisioningDestroy` | `fy34trz2st` | `int(uint)` | +| `ADIProvisioningEnd` | `uv5t6nhkui` | `int(uint, ubyte*, uint, ubyte*, uint)` | +| `ADIProvisioningStart` | `rsegvyrt87` | `int(ulong, ubyte*, uint, ubyte**, uint*, uint*)` | +| `ADIGetLoginCode` | `aslgmuibau` | `int(ulong)` | +| `ADIDispose` | `jk24uiwqrg` | `int(void*)` | +| `ADIOTPRequest` | `qi864985u0` | `int(ulong, ubyte**, uint*, ubyte**, uint*)` | + +``` + FOUND kq56gsgHG6 FOUND fy34trz2st + FOUND Sph98paBcz FOUND uv5t6nhkui + FOUND nf92ngaK92 FOUND rsegvyrt87 + FOUND p435tmhbla FOUND aslgmuibau + FOUND tn46gtiuhw FOUND jk24uiwqrg + FOUND qi864985u0 +``` + +Symbol dumper: `scratchpad/poc_elf_symbols.py` (pure Python, no NDK). + +**These names are not stable across builds** — `docs/ADI.md` says obfuscation differs per +build. So the loader has to fail legibly when a symbol is missing, and the version of the APK +it pulled should be recorded. + +--- + +## 5. RESOLVED (2026-08-10): it loads, and ADI initialises + +Run on the Gradle managed device (Pixel 6, API 34, x86_64, aosp-atd) by a load test against a +native probe. All three steps passed, `skipped="0"`. (Both have since been implemented as +`app/src/main/cpp/adi.cpp` and the tests in `app/src/androidTest/.../anisette/`.) + +**W^X does not block it.** SELinux said so itself: + +``` +avc: granted { execute } for path=".../files/adi-poc/x86_64/libc++_shared.so" + scontext=u:r:untrusted_app:s0 tcontext=u:object_r:app_data_file:s0 tclass=file +``` + +All eleven libraries `dlopen`'d out of `filesDir` in dependency order, RTLD_NOW | RTLD_GLOBAL, +no linker tricks and nothing on the search path. All eleven ADI entry points resolved. Then: + +``` +ADILoadLibraryWithPath(/data/user/0/.../files/adi-poc/x86_64) = 0 +``` + +0 is ADI's success code. Note what that implies: `libCoreADI.so` was never opened by us - +`libstoreservicescore.so` found and loaded it itself. Apple's CoreFoundation, libdispatch, ICU +and CoreADI all initialise inside an ordinary Android app process. + +**Consequence: the manual mmap+relocate ELF loader is not needed.** That was the only +open item that could have made this a multi-week project, and it is gone. Path B (ignoring +DT_NEEDED to save ~25 MB) is now purely a size optimisation and can wait indefinitely. + +### Confirmed on real hardware the same day + +Re-run on a Galaxy S25 (SM-S938B), **arm64-v8a, Android 16**, from Studio with +`-e adiPoc true` as an instrumentation extra param. All three steps passed, and there is no +`avc: denied` anywhere in logcat: + +``` +downloaded 12.6 MB of 142 MB (8.9%) +dlopen(.../arm64-v8a/libstoreservicescore.so) ok +resolved kq56gsgHG6 ... qi864985u0 (all eleven) +ADILoadLibraryWithPath(.../arm64-v8a) = 0 +``` + +So the emulator result was not a policy artefact of aosp-atd. Two numbers worth keeping: + +- **The download took 2.1 seconds.** This is a brief one-time spinner, not a progress-bar + problem. Design for the slow case rather than around it. +- **All eleven libraries loaded in 11 ms.** Dependency-ordered loading costs nothing. + +Correction to section 2a: the real cost is **12.6 MB, not 11.3 MB**. That script counted only +the DT_NEEDED graph, and `libCoreADI.so` (1.79 MB) is not a declared dependency of anything - +libstoreservicescore opens it by path at runtime. 12.6 MB is the measured figure. + +## 7. RESOLVED (2026-08-10): provisioning works end to end + +`AdiProvisioningTest` on the managed device, against Apple's live servers. +`tests="1" failures="0" skipped="0"`: + +``` +provisioning as 3ED7500C-29FA-4A26-99A2-47D8BD863E01 (ADI id bb1037520408cb27) +ADI initialised, persisting to .../files/adi-provisioning-state +provisioned: session 1458087437 completed +already provisioned <- second call correctly a no-op +``` + +Apple accepted an entirely invented machine identity, ADI swallowed the persistent token and +trust key, and ADIGetLoginCode then returned 0. **There is no research left in this feature.** + +Three things it took to get there, all of them things the reference does differently: + +**1. The identity lengths are checked.** See section on invented identity - `rndGen.take(2)` is +two uints, not two bytes. Wrong length gives -45001. + +**2. `gsa.apple.com` fails TLS against Android's trust store.** It chains to the original +"Apple Root CA", which Android does not ship: + +``` +SSLHandshakeException: Unacceptable certificate: CN=Apple Root CA, O=Apple Inc., C=US +``` + +Provision solves this with `request.sslSetVerifyPeer(false)`. **We do not** - turning off +certificate verification in an authentication flow inside a phone app makes it interceptable by +anything on the network path. Instead the one missing root is added for that one domain in +`network_security_config.xml`, alongside `system`, with the certificate stored as PEM text +(SHA-256 `b0b1730e...f024`, published at https://www.apple.com/appleca/). That is *stricter* +than the default, not looser. + +**3. The stub question is answered on the path that mattered.** `makeWorkQueue` was called, +returned NULL, and provisioning completed anyway. So stubbing holds through provisioning, and +that symbol is now recorded in `libmediaplatform.expected` - logged at INFO rather than raising +an alarm, because an alarm that fires on every successful run is one nobody reads. + +### What this does not yet prove + +- **ADIOTPRequest**, the per-login one-time password. Provisioning is the hard half; this is + the half that runs on every login. +- **The Python side** - a provider that hands FindMy.py the headers, alongside the existing + remote one. +- Historical note, now resolved: `ADILoadLibraryWithPath` returning 0 meant ADI initialised. The + ProvisioningStart -> Apple -> ProvisioningEnd round trip is untested. +- Apple can change the APK layout or re-obfuscate the symbols at any time, so the failure + paths matter as much as the happy one. + +## 6. The 28 MB can probably be stubbed away, without an ELF loader + +Measured by `scratchpad/poc_stub_surface.py`. `libstoreservicescore.so` has 405 undefined +symbols, and they come from only three libraries: + +``` +libc++_shared.so 141 symbols [1.0 MB] +libmediaplatform.so 125 symbols [2.4 MB] +libCoreFoundation.so 93 symbols [2.0 MB] +(platform / bionic) 46 symbols - already present +``` + +**ICU, curl, libxml2, libdispatch and BlocksRuntime are never referenced.** They are in the +closure only because CoreFoundation and mediaplatform depend on them. ICU alone is 14.6 MB. + +bionic satisfies DT_NEEDED by SONAME against already-loaded libraries and does not check who +built them. So two generated stub libraries - SONAME `libCoreFoundation.so` and +`libmediaplatform.so`, exporting those 218 names - satisfy the linker at a few kB each, and the +20 MB tail behind them stops being reachable because nothing declares a dependency on it. + +``` +today (path A): 11 libraries 11.3 MB downloaded 28.2 MB on disk +with stubs: 3 libraries 2.9 MB downloaded 5.2 MB on disk +``` + +Keep Apple's real `libc++_shared.so`: the mangled names are `__ndk1`-namespaced, so NDK 27's +own libc++ should not be assumed ABI-identical without checking. + +### CONFIRMED (2026-08-10): it works, and one function is called + +`StubbedAdiLoadTest` on the managed device. All three steps passed, `skipped="0"`. Three +libraries downloaded instead of twelve, all eleven ADI entry points still resolved, and +`ADILoadLibraryWithPath` still returned 0. + +Of **218 stubbed symbols, exactly one was reached**: + +``` +adi-stub: mediaplatform::WorkQueue::makeWorkQueue(std::string const&, WorkQueueType) +``` + +None of the 93 CoreFoundation symbols were touched at all. + +**And checking Provision shows this is lower risk than it looks.** Two things: + +*When it happens.* By timestamp, `makeWorkQueue` fired at `48.008` - the same millisecond as +`dlopen(libstoreservicescore.so) ok`, and before `ADILoadLibraryWithPath` at `48.026`. So it is +a **static constructor running during dlopen**, not something ADI called. + +*Why Provision never sees it.* Their symbol table (`lib/provision/symbols.d`) is 29 entries and +every one is libc/bionic - `dlopen`, `malloc`, `pthread_*` as `emptyStub`, `read`, `write`, +`__errno`, `__system_property_get`, `arc4random`. No CoreFoundation, no mediaplatform. Anything +else becomes a trampoline that *throws* `UndefinedSymbolException`. And their loader **never +runs `.init_array`** - `androidlibrary.d` has no constructor handling at all. + +So in the reference implementation that constructor never runs, and whatever global holds the +work queue stays zero: functionally identical to our stub returning NULL. Every public Anisette +server is a live demonstration that ADI's provisioning paths work with no work queue. Worth +watching during provisioning, but not an open question. + +This is why the stubs log and count instead of aborting. A stub that returns 0 produces a +*silent wrong answer*, which is worse than a crash, so each one logs an actionable error and +increments a counter that `_adi_stub_calls()` exposes - the hook the runtime fallback +to a remote Anisette server should hang off. + +## What is left, and where a device becomes necessary + +Everything above is acquisition and symbol resolution. The remaining work is the actual +integration: + +1. **JNI wrapper** over the eleven functions. `dlopen` `libstoreservicescore.so`, then + `ADILoadLibraryWithPath` pointing at the directory holding `libCoreADI.so`. +2. **Provisioning**: `ADISetAndroidID` with a generated identifier, `ADISetProvisioningPath` + into app storage, then `ADIProvisioningStart` → POST to Apple's `midStartProvisioning` → + `ADIProvisioningEnd`. One time, persisted. `adi.d` around lines 380–500 has the exact + request shape. +3. **Headers per login**: `ADIOTPRequest` produces the one-time password and machine ID that + the login flow needs. +4. **Python side**: a provider that calls back into Java, alongside `RemoteAnisetteProvider`. +5. **Fallback**: keep the remote provider. A local path that fails must degrade to today's + behaviour, not to a broken login. + +None of steps 1–3 can be tested without a device — the libraries are arm64 Android binaries, +and this desktop cannot load them. + +### The device identity is invented, not derived + +Worth settling, because it sounds like it should be the hard part and is not. The whole +identity a public Anisette server presents to Apple is randomly generated on first run: + +```d +v1Device.serverFriendlyDescription = clientInfo; // a UA-ish string +v1Device.uniqueDeviceIdentifier = randomUUID().toString().toUpper(); +v1Device.adiIdentifier = (cast(ubyte[]) rndGen.take(2).array()).toHexString().toLower(); +v1Device.localUserUUID = (cast(ubyte[]) rndGen.take(8).array()).toHexString().toUpper(); +``` + +**Careful with those lengths - I got them wrong first time round and ADI rejected it with +-45001 (invalid parameters).** `rndGen.take(2)` takes two *uints*, not two bytes: that is +8 bytes, 16 hex characters. `take(8)` is 32 bytes, 64 hex characters - which is what +`X-Apple-I-MD-LU` looks like in real Apple traffic. The values are invented, but the sizes are +checked. + +It is then persisted and reused, so the machine stays the same machine across logins. + +This matches what rustpush's author said independently in +[rustpush#18](https://github.com/OpenBubbles/rustpush/issues/18): + +> You still need apple hw IDs to send to Apple's server (X-MME-Client-Info or whatever) but you +> should be able to make something up, i don't think that stuff is strictly validated + +So there is no hardware attestation to defeat and nothing to extract from a real device. Every +public Anisette server in use today runs on invented identity, which is the working proof. + +What matters is that the identity is **generated once and kept** — regenerating per login would +make every session look like a new machine, which is exactly the pattern 2FA is designed to +notice. + +## Honest assessment + +The scope has not changed: this is still weeks, and the hard part was never acquisition. What +this PoC removes is uncertainty. Every question that could be answered without a device now is +answered, and none of the answers were bad: + +- the libraries are still published, and fetching them costs 2.6 MB +- they are Android-native, JNI-ready, and require no emulation +- all eleven entry points exist, with known signatures, in the current build + +There is no research left in the *sequence*. The identity is invented, the entry points are +known, the libraries are fetchable cheaply, and every public Anisette server is a live +demonstration that the calls work. + +### But there is no precedent for doing it on Android, and that matters + +Checked, expecting to find one: + +- **Provision / anisette-v3-server** (D) — `androidlibrary.d`, a hand-written ELF loader, for + running Android libraries on desktop Linux. +- **omnisette** (Rust, OpenBubbles) — depends on Dadoum's `android-loader`, and uses it in + `store_services_core/posix_macos.rs` and `posix_windows.rs`. **There is no Android file.** + Its only Anisette features are `remote-anisette-v3` and `remote-clearadi`; `clearadi` itself + is an empty submodule pointing at a closed-source stub. + +So **OpenBubbles on Android uses a remote Anisette server**, exactly as this app does today. The +whole ecosystem writes Android-library loaders *for desktop*, and on the one platform where the +libraries are native, everybody calls a server. + +That is not evidence it cannot work — these projects are desktop-first, and OpenBubbles' Android +client is a Flutter app that already had a server to talk to. But it does mean there is nothing +to copy for the loading step, and the one Android-specific obstacle is unanswered: + +**W^X.** Apps targeting API 29+ cannot execute code from writable app-home storage, and +`targetSdk` here is 35. `dlopen()` on a `.so` downloaded into the app's data directory is the +thing the platform is designed to refuse. If that holds, the options are: + +| Option | Consequence | +| --- | --- | +| `System.load()` from app data dir | free if it works; needs a fallback if it does not | +| **Map it ourselves** | always works, no file is ever executed — see below | +| Bundle in `jniLibs` | always works, but **we would be redistributing Apple's binaries** — the exact position fetching-from-Apple's-CDN avoids | + +### `dlopen` is not the only way to load a library + +W^X restricts executing **files** from app-writable storage. It does not stop an app mapping +anonymous memory and marking it executable — ART's own JIT depends on precisely that, which is +why `execmem` is granted to the `untrusted_app` SELinux domain. + +And a loader that does this already exists, in the reference implementation. `androidlibrary.d`: + +```d +allocation = MmapAllocator.instance.allocate(allocSize + shift); // anonymous, not file-backed +mprotect(..., PROT_READ | PROT_WRITE); // writable while loading +allocation[headerStart..headerEnd] = elfFile[fileStart..fileEnd]; // copy segments in +mprotect(..., prot); // then read+execute +// ... relocations applied by hand +``` + +No file is ever executed. The bytes are copied into anonymous pages and relocated there. + +**On Android this loader gets simpler, not harder.** Its companion `symbols.d` exists to map +bionic symbols (`open`, `pthread_create`, `__errno`, ...) onto glibc equivalents, because it +runs on desktop. On Android those symbols are the real ones — resolution is `dlsym` against the +platform's own libc, and that entire table becomes unnecessary. + +Implementation would be an NDK component: port the segment mapping and relocation logic to +C/C++ (Android's own linker is the reference), or reuse Dadoum's `android-loader` as a Rust +cdylib. Either is real work, but it is bounded, well-understood work with a working +implementation to read. + +### The experiment, in order + +1. **Try `System.load()` from `filesDir` first.** If it works, none of the above is needed. + Ten minutes on a device. +2. **If W^X refuses it**, map it manually. Known to work, code to copy, and simpler here than + in the project it comes from. +3. Bundling stays the last resort, because it is the only option that changes the licensing + position. + +Either way the feature is reachable without shipping Apple's binaries — which was the real +question. + +What remains is a JNI layer, a provisioning flow persisted to app storage, a Python provider +that calls back into Java, and a fallback to `RemoteAnisetteProvider` when any of it fails. +That is ordinary work — a fair amount of it, on a device, but ordinary. diff --git a/docs/findmy-export/01-authentication.md b/docs/findmy-export/01-authentication.md new file mode 100644 index 00000000..b3f50f75 --- /dev/null +++ b/docs/findmy-export/01-authentication.md @@ -0,0 +1,956 @@ +# Stage 1 — Apple ID authentication (Grand Slam / GSA) + +Specification of the authentication exchange between a client and Apple's Grand Slam +authentication service, from an Apple ID and password to a set of session tokens that later +stages consume. + +Read [README.md](./README.md) first. In particular: **implement from this document alone.** + +> **Verified against a live account on 2026-08-13.** A probe built from this document +> authenticated end to end: SRP handshake, server proof verified, session payload decrypted, +> trusted-device two-factor, 25 service tokens collected. Values from that run are marked +> **[observed]**. Everything unmarked is still derived by reading rather than running — the SMS +> path in particular was not exercised. +> +> That the server proof verified is the strongest single result: `M2` only matches if the +> client's entire SRP computation agrees with Apple's, so all three deviations in §4.6 are +> confirmed rather than merely plausible. +> +> The run also settled the question that shapes the whole feature: **signing in registers a +> device on the account.** See §13. + +--- + +## 1. What this stage produces + +On success the client holds: + +| Value | Origin | Used by | +| --- | --- | --- | +| `adsid` | decrypted session payload | every later stage; Apple's numeric account identifier (the "DSID") | +| `GsIdmsToken` | decrypted session payload | building the two-factor identity token | +| `acname` | decrypted session payload | the account's canonical Apple ID; may differ from what the user typed | +| **PET** (`com.apple.gs.idms.pet`) | response header, or the session payload | Stage 2. Short-lived — assume 5 minutes. | +| Service tokens, keyed by reverse-DNS service name | response headers, or the session payload | later stages; `com.apple.gs.idms.hb` is needed by several GSA endpoints | +| The password's SHA-256 digest | computed locally | silently re-authenticating when a token expires | + +The PET ("password-equivalent token") is the handover to Stage 2: it is presented as the +password in an HTTP Basic credential against the MobileMe delegate endpoint. Because it expires +in minutes, Stage 1 and Stage 2 must run back to back, or Stage 1 must be repeatable without +user interaction (see §11). + +## 2. Prerequisites + +### 2.1 Anisette + +Every request in this stage carries **Anisette** headers: a one-time password and machine +identifier produced by Apple's ADI libraries from provisioned, per-installation state. Without +valid Anisette, GSA rejects the request regardless of whether the password is correct. + +This document treats Anisette as a solved dependency and specifies only the header names it +must supply. OpenTagViewer generates these on-device from Apple's own Android ADI libraries — +see `app/src/main/java/.../anisette/` — falling back to a remote Anisette server when it +cannot. + +The Anisette provider must supply: + +| Header | Meaning | +| --- | --- | +| `X-Apple-I-MD` | The one-time password. Changes per request. | +| `X-Apple-I-MD-M` | The machine identifier from the provisioned ADI state. | +| `X-Apple-I-MD-RINFO` | Routing information, a decimal integer as a string. | +| `X-Apple-I-MD-LU` | The local user identifier — a stable hex string. | +| `X-Mme-Device-Id` | A stable UUID identifying this installation. | +| `X-Apple-I-Client-Time` | Current UTC time, ISO 8601, whole seconds, `Z` suffix — e.g. `2026-08-13T09:41:07Z`. | +| `X-Apple-I-TimeZone` | `UTC`. | +| `X-Apple-Locale` | `en_US`. | + +Headers are selected **by exact name**, and anything unrecognised is silently dropped rather +than rejected. So a provider that omits one — or spells it differently — does not fail at the +point of the mistake; authentication fails later, for a reason that looks unrelated. Verify the +provider's output against the six names above before debugging anything else. (Observed: +macOS's own Anisette omits `X-Apple-I-MD-RINFO` and `X-Apple-I-Client-Time` entirely.) + +Two hard requirements: + +- **The identity must be stable across logins.** `X-Apple-I-MD-LU` and `X-Mme-Device-Id` must be + derived from the same persisted state that provisioned ADI, and must not be regenerated. An + identity that changes per login makes every session look like a new machine, which is the + pattern two-factor authentication exists to detect. +- **A session is bound to the identity that established it.** Switching between a local Anisette + provider and a remote server, or between two different remote servers, presents a different + machine to Apple and invalidates the session. This is inherent to how Apple binds sessions. + +Anisette headers may be cached and reused for a short window; **60 seconds** is the interval the +reference implementations use before regenerating. The one-time password is genuinely one-time +in the sense that it is time-derived, not that Apple rejects a repeat within its validity window. + +### 2.2 Client identity + +The client presents itself as an iPhone. The values are invented and are not validated against +real hardware, but they must be **internally consistent** and **stable across logins**. + +A worked example, using an iPhone 14 Pro on iOS 17.4: + +| Field | Example value | +| --- | --- | +| Hardware model | `iPhone15,2` | +| OS version | `17.4` | +| Build | `21E219` | +| CFNetwork version | `1494.0.7` | +| Darwin version | `23.4.0` | +| Device serial | **User-visible — choose it as a label.** Uppercase alphanumeric, 10–12 characters. `0PENTAGVIEWR` is confirmed accepted and displayed. See §13. | +| Device name | any user-visible string | +| Device UUID | a random v4 UUID, **uppercase**, generated once and persisted | +| UDID | 32 uppercase hex characters, generated once and persisted | + +The OS version, build, CFNetwork version and Darwin version must correspond to one another — +they describe one real iOS release, and Apple's own clients never disagree with themselves. +Changing the hardware model means changing all of them together. + +From those, three composite strings are built: + +``` +mmeClientInfo = +mmeClientInfoAkd = +akdUserAgent = akd/1.0 CFNetwork/1494.0.7 Darwin/23.4.0 +``` + +The general shape is ` `, where the trailing bundle +identifies which Apple daemon or app is speaking. The client also uses a fixed browser +user-agent string for the two-factor web endpoints: + +``` +browserUserAgent = Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) +``` + +That it names macOS while everything else names iOS is not a mistake to correct — the +two-factor endpoints are web endpoints and are addressed as a browser would address them. + +The client also presents an application identity. Throughout this stage it claims to be +Messages: + +``` +akContextType = imessage +clientAppName = Messages +clientBundleId = com.apple.MobileSMS +``` + +Finally, a **hardware headers** map exists for values a genuine Apple device would send +(`X-Apple-I-SRL-NO` and similar). For an invented identity it is **empty**, and every header +set below that mentions it contributes nothing. + +### 2.3 TLS + +`gsa.apple.com` presents a certificate chaining to Apple's original **Apple Root CA**, which is +not in Android's system trust store. A plain HTTPS request fails the handshake: + +``` +Unacceptable certificate: CN=Apple Root CA, O=Apple Inc., C=US +``` + +**Do not disable certificate verification.** Add that single root, for those Apple hosts only, +alongside the system trust store — a strictly narrower trust decision than the default, not a +looser one. The certificate is published at ; its SHA-256 +fingerprint is: + +``` +B0:B1:73:0E:CB:C7:FF:45:05:14:2C:49:F1:29:5E:6E:DA:6B:CA:ED:7E:2C:68:C5:BE:91:B5:A1:10:01:F0:24 +``` + +OpenTagViewer already carries this root in its network security configuration for the Anisette +provisioning work. Reuse that rather than adding a second copy. + +### 2.4 HTTP client + +- **Cookies must persist across requests for the whole stage.** The two-factor endpoints depend + on cookies set during the SRP exchange. A stateless client fails partway through with errors + that do not mention cookies. +- Apple's servers are sensitive to header casing. Send headers in the conventional + `X-Apple-I-MD` title case rather than lowercasing them, and do not rely on an HTTP/2 client + that normalises them. + +--- + +## 3. Message envelope + +The GSA service endpoint is: + +``` +POST https://gsa.apple.com/grandslam/GsService2 +``` + +Requests and responses are **XML property lists**. Every request body is a dictionary with +exactly two keys: + +``` +Header → dictionary: { "Version": "1.0.1" } +Request → dictionary: the operation-specific payload +``` + +Every response body is a dictionary whose `Response` key holds the reply dictionary. + +### 3.1 Errors + +Inside the reply, a `Status` dictionary carries the outcome. If `Status` is absent, the reply +dictionary itself carries these keys: + +| Key | Type | Meaning | +| --- | --- | --- | +| `ec` | integer | Error code. `0` means success. | +| `em` | string | Human-readable message. | + +Check `ec` after **every** exchange, before reading any other field. A non-zero `ec` with its +`em` is the only diagnostic Apple gives, so surface both verbatim in logs and in whatever the +user sees; the messages are specific enough to act on and guessing at them wastes hours. + +Content type for these requests is `text/x-xml-plist`, and `Accept` is `*/*`. + +--- + +## 4. The SRP handshake + +Authentication is **SRP-6a** — a password-authenticated key exchange in which the password +never crosses the network and both sides end up holding a shared session key. It runs as two +round trips against the endpoint in §3. + +Apple's variant deviates from RFC 5054 in three specific ways. Each is called out below, and +each will produce an indistinguishable "authentication failed" if implemented to the letter of +the RFC instead. **§4.6 lists them together; read it before writing any of this.** + +### 4.1 Parameters + +- Group: the **2048-bit group from RFC 5054, appendix A** — modulus `N`, generator `g = 2`. Both + are public constants; take them from the RFC. +- Hash: **SHA-256** throughout. +- Client secret `a`: **32 random bytes** from a cryptographically secure source. +- `A = g^a mod N`. + +### 4.2 Round trip one — `init` + +Request payload: + +| Key | Type | Value | +| --- | --- | --- | +| `o` | string | `init` | +| `u` | string | The Apple ID as the user typed it | +| `A2k` | data | `A`, big-endian | +| `ps` | array of string | `["s2k", "s2k_fo"]` — the password-derivation schemes offered | +| `cpd` | dictionary | See §5 | + +Reply: + +| Key | Type | Meaning | +| --- | --- | --- | +| `s` | data | Salt. **[observed]** 16 bytes. | +| `B` | data | Server public value, big-endian. **[observed]** 256 bytes. | +| `i` | integer | PBKDF2 iteration count. **[observed]** 20082 — note it is neither round nor a power of two. Do not hardcode it; it is Apple's to choose and may vary by account or over time. | +| `c` | string | An opaque continuation value, echoed in round trip two | +| `sp` | string | The scheme the server chose: `s2k` or `s2k_fo`. **If absent, assume `s2k`.** **[observed]** present, and `s2k`. | + +### 4.3 Deriving the SRP password + +The raw password is never used directly. Let `P₀ = SHA-256(password_utf8)` — this digest is +also what gets cached in §11. + +Then, depending on `sp`: + +| `sp` | Input to PBKDF2 | +| --- | --- | +| `s2k` | `P₀` — the 32 raw digest bytes | +| `s2k_fo` | The **lowercase hexadecimal ASCII encoding** of `P₀` — 64 bytes, not 32 | + +Then: + +``` +P = PBKDF2-HMAC-SHA256(input, salt = s, iterations = i, output length = 32) +``` + +`P` is what the SRP computation below calls the password. Note that the `s2k_fo` case is a +double transformation: hash the password, then hex-encode that hash *as text*, then stretch it. + +**[observed]** Apple selected `s2k`. The `s2k_fo` branch is therefore specified but unexercised, +and should be treated as the less trustworthy of the two — it is offered in `ps`, so Apple may +select it for some accounts, but nothing here has seen it happen. + +### 4.4 Client computation + +``` +x = SHA-256( s ‖ SHA-256( "" ‖ ":" ‖ P ) ) ← deviation 1, see §4.6 +u = SHA-256( A ‖ B ) ← deviation 2, see §4.6 +k = SHA-256( N ‖ PAD(g) ) +S = (B − k·g^x) ^ (a + u·x) mod N +K = SHA-256(S) +M1 = SHA-256( (SHA-256(N) ⊕ SHA-256(PAD(g))) ‖ SHA-256(I) ‖ s ‖ A ‖ B ‖ K ) +M2 = SHA-256( A ‖ M1 ‖ K ) +``` + +where `I` is the Apple ID as typed, `PAD(g)` is `g` left-padded with zero bytes to the byte +length of `N` (256 bytes), and `‖` is concatenation. + +`M1` and `M2` are standard SRP-6a. `K` is the session key that decrypts the payload in §6. + +Reject the exchange if `B mod N == 0`. + +Computing `B − k·g^x` in modular arithmetic can go negative. Add `N` before subtracting: +`((N + B) − (k·g^x mod N)) mod N`. + +### 4.5 Round trip two — `complete` + +Request payload: + +| Key | Type | Value | +| --- | --- | --- | +| `o` | string | `complete` | +| `u` | string | The same Apple ID string sent in `init` | +| `M1` | data | `M1` | +| `c` | string | The `c` value from the `init` reply, echoed unchanged | +| `cpd` | dictionary | See §5 — **the same request UUID as `init`** | + +Reply: + +| Key | Type | Meaning | +| --- | --- | --- | +| `M2` | data | Server proof. Compare against the locally computed `M2` and abort on mismatch. | +| `spd` | data | The encrypted session payload — see §6 | +| `Status` | dictionary | `ec`/`em`, plus `au` if a second factor is required — see §7 | + +Verifying `M2` is what authenticates Apple to the client. It is not optional: skipping it means +a party who never knew the password can complete the exchange. + +### 4.6 The three deviations, and the serialisation trap + +**Deviation 1 — the identity in `x` is empty.** RFC 5054 computes +`x = H(s ‖ H(I ‖ ":" ‖ P))` with `I` the username. Apple computes the inner hash with the +username **omitted**: `H("" ‖ ":" ‖ P)`, which is `SHA-256(":" ‖ P)` — the colon is still there. +Note the asymmetry: `M1` *does* use the real username, as `SHA-256(I)`. Getting this backwards +is the single most likely cause of a handshake that fails with correct credentials. + +**Deviation 2 — `u` is not padded.** RFC 5054 computes `u = H(PAD(A) ‖ PAD(B))`. Apple hashes +`A` and `B` at their natural lengths, with no padding. `k` and `M1` *do* pad `g`. So padding +applies to `g` and not to `A` or `B`, within the same handshake. + +**Deviation 3 — big integers are serialised minimally.** `A`, `B` and the `g` inside `k` and +`M1` are big-endian byte strings **with no leading zero bytes and no sign byte**. Two things +follow, and both are silent failures rather than errors: + +- Whenever `A` or `B` has a leading zero byte in its natural 256-byte form — roughly one time in + 256 — a fixed-width encoding produces a different `u` and the handshake fails. This is an + intermittent, unreproducible login failure if implemented wrongly, so it will not show up in + testing. +- Many big-integer libraries emit a leading `0x00` on export to keep the value positive. That + byte must be stripped. Equally, `g = 2` must be encoded as the single byte `0x02` before + padding, not as a wider machine word. + +**[observed]** On the verified run both `A` and `B` were the full 256 bytes, so the +leading-zero case did not arise. That is the expected outcome roughly 255 times in 256 and is +emphatically not evidence the trap is absent — it is the reason the trap is dangerous. A wrong +implementation passes this test almost every time. + +Round-trip your encoder against a known value before debugging anything else. + +--- + +## 5. The `cpd` dictionary + +`cpd` is a dictionary carried inside both SRP request payloads. It repeats some of the Anisette +headers as payload fields and adds client capability flags. + +Generate **one** uppercase v4 UUID per login attempt and use it for `X-Apple-I-Request-UUID` in +both round trips. A different UUID in `complete` breaks the correlation with `init`. + +Copy these straight from the Anisette headers, unchanged: + +``` +X-Apple-I-Client-Time X-Apple-I-MD X-Apple-I-MD-LU +X-Apple-I-MD-M X-Apple-I-MD-RINFO X-Mme-Device-Id +``` + +Add, as strings: + +| Key | Value | +| --- | --- | +| `X-Apple-I-Device-Configuration-Mode` | `0` | +| `X-Apple-I-Request-UUID` | the per-login UUID | +| `X-Apple-Requested-Partition` | `0` | +| `X-Apple-Security-Upgrade-Context` | `com.apple.authkit.generic` | +| `capp` | `Messages` | +| `cbid` | `com.apple.MobileSMS` | +| `cou` | `US` | +| `loc` | `en_US` | +| `svct` | `imessage` | + +Add, as booleans and integers — the plist types matter, a boolean sent as the string `"true"` +is not the same value: + +| Key | Type | Value | +| --- | --- | --- | +| `X-Apple-Offer-Security-Upgrade` | boolean | true | +| `at` | integer | 0 | +| `bootstrap` | boolean | true | +| `ckgen` | boolean | true | +| `fcd` | boolean | true | +| `icdrsDisabled` | boolean | false | +| `icscrec` | boolean | true | +| `pbe` | boolean | false | +| `prkgen` | boolean | true | +| `webAccessEnabled` | boolean | false | + +Finally, merge in the hardware headers map (§2.2), which is empty for an invented identity. + +A push token field `ptkn` also exists here, carrying an uppercase hex APNs token. It belongs to +clients that maintain a push connection. This flow does not, so **omit the key entirely** — +do not send it empty. + +--- + +## 6. Decrypting the session payload + +`spd` from the `complete` reply is AES-encrypted under keys derived from the SRP session key +`K`. Derive them by HMAC, using `K` as the HMAC key and two fixed ASCII strings as messages: + +``` +key = HMAC-SHA256( K, "extra data key:" ) → 32 bytes, the AES key +iv = HMAC-SHA256( K, "extra data iv:" )[0 .. 16] → first 16 bytes only +``` + +Both trailing colons are part of the strings. + +Decrypt `spd` with **AES-256-CBC** and **PKCS#7** padding. The plaintext is an XML property list +dictionary: + +| Key | Type | Meaning | +| --- | --- | --- | +| `adsid` | string | The account's numeric identifier (DSID) | +| `GsIdmsToken` | string | Paired with `adsid` to form the two-factor identity token (§7.1) | +| `acname` | string | The account's canonical Apple ID. **Store this** in preference to what the user typed. | +| `fn`, `ln` | string | Given and family name | +| `t` | dictionary | Service tokens — see §6.1 | + +The payload is decrypted even when a second factor is still required; `adsid` and `GsIdmsToken` +are available at that point and are needed to request the second factor. + +**[observed]** The dictionary held **49 keys** from 4096 bytes of ciphertext on a first login, +and **50 keys** from 18160 bytes on a subsequent one — the second being larger because the full +token set is inlined. **Its keys are not fixed**: `url` appeared only when a second factor was +pending, while `ck` and `prk` appeared only when one was not. So treat it as an open-ended +structure, read by name, and never assume a key is present. Beyond those above, several +matter to later stages and are worth knowing exist now: + +| Key | Why it matters later | +| --- | --- | +| `DsPrsId` | a second form of the account identifier, alongside `adsid` | +| `sk`, `c` | session key and cookie for the separate app-token exchange, which this flow does not use but which exists | +| `SOSNeeded`, `SOSCompatibilityOptInNeeded`, `hasSOSActiveDevice` | "Secure Object Sharing" — Apple's name for the iCloud Keychain circle. Directly relevant to Stage 3, and `hasSOSActiveDevice` looks like the signal for whether the account has anything to recover from. | +| `isPasscodeAuth`, `passcodeAuthEnabled` | relates to the device-passcode authentication Stage 3 needs | +| `hasRK` | presence of a recovery key | +| `passkeyEligible`, `passkeyPresent` | an alternative authentication route this document does not cover | +| `primaryEmail`, `countryCode`, `authmode`, `ut` | account attributes | + +### 6.1 The token dictionary + +Each key of `t` is a service name; each value is a dictionary: + +| Key | Meaning | +| --- | --- | +| `token` | The token string | +| `expiry` | Absolute expiry, **milliseconds since the Unix epoch** | +| `duration` | Lifetime in **seconds** from now | + +**[observed]** `expiry` and `duration` are **not** alternatives — the one token present before +two-factor carried both. Prefer `expiry`, since it is absolute and needs no clock arithmetic, +and fall back to `now + duration`. Do not assume the presence of one implies the absence of the +other. + +**[observed]** Which tokens arrive here depends on whether a second factor was needed: + +| Situation | `t` contained | +| --- | --- | +| Second factor required | **one** entry, `com.apple.gs.appleid.auth`. The rest arrive later via response headers (§8). | +| No second factor (known machine) | **all 25**, including the PET and the heartbeat token. No header parsing needed. | + +So both routes must be implemented: an implementation that only reads tokens from headers works +on first run and silently gets nothing afterwards, and one that only reads the payload fails the +other way round. + +**[observed]** In the no-second-factor case every one of the 25 entries carried *both* `expiry` +and `duration`, reinforcing that they are not alternatives. + +--- + +## 7. Two-factor authentication + +> **[observed] A second login from the same machine identity required no second factor at all.** +> `au` was absent, the login completed in zero extra rounds, and the full token set arrived in +> the decrypted payload rather than in headers. This is the behaviour §11 depends on for silent +> refresh, and it is the practical reward for the identity stability demanded in §2.1 — Apple +> remembers the machine, and stops challenging it. +> +> It also means **the two-factor path is the first-run path**. An implementation that only ever +> tests against an already-known identity will not exercise §7 at all. + +After a successful `complete`, inspect `Status.au`: + +| `au` | Meaning | Go to | +| --- | --- | --- | +| absent | Login is complete. | §9 | +| `trustedDeviceSecondaryAuth` | A code can be pushed to the user's trusted devices. | §7.2 | +| `secondaryAuth` | A code must be sent by SMS. | §7.3 | +| anything else | An unhandled requirement. Report the value verbatim. | — | + +Do not hard-fail on an unrecognised `au` before checking whether a PET was nonetheless issued; +some variants complete anyway. + +The endpoints in this section are **not** the plist service of §3. They are web endpoints, they +mostly speak JSON, and they carry a different header set (§7.1). + +### 7.1 Two-factor header set + +Take from the Anisette headers: + +``` +X-Apple-I-MD X-Apple-I-MD-M X-Apple-I-MD-LU X-Apple-I-MD-RINFO X-Mme-Device-Id +``` + +Add the identity token, which is what authenticates these calls: + +``` +X-Apple-Identity-Token = base64( adsid + ":" + GsIdmsToken ) +``` + +Add the following fixed headers. Note that `X-MMe-Client-Info` here is the **Messages** variant +from §2.2, not the `akd` variant used in §8, and the user-agent is the **browser** one. + +| Header | Value | +| --- | --- | +| `X-Apple-Client-App-Name` | `Messages` | +| `X-Apple-I-Client-Bundle-Id` | `com.apple.MobileSMS` | +| `X-MMe-Client-Info` | `mmeClientInfo` | +| `X-Apple-AK-Context-Type` | `imessage` | +| `User-Agent` | `browserUserAgent` | +| `Accept-Language` | `en-US,en;q=0.9` | +| `X-Apple-I-Locale` | `en_US` | +| `X-Apple-I-TimeZone` | `UTC` | +| `X-Apple-I-TimeZone-Offset` | `0` | +| `X-MMe-Country` | `US` | +| `X-Apple-Requested-Partition` | `0` | +| `X-Apple-I-Device-Configuration-Mode` | `0` | +| `X-Apple-I-DeviceUserMode` | `0` | +| `X-Apple-Security-Upgrade-Context` | `com.apple.authkit.generic` | +| `X-Apple-I-CDP-Status` | `false` | +| `X-Apple-I-CDP-Circle-Status` | `false` | +| `X-Apple-I-OT-Status` | `false` | +| `X-Apple-I-ICSCREC` | `true` | +| `X-Apple-I-PRK-Gen` | `true` | +| `Sec-Fetch-Site` | `same-origin` | +| `Sec-Fetch-Mode` | `cors` | +| `Sec-Fetch-Dest` | `empty` | +| `X-Apple-I-CFU-State` | base64 of an XML plist whose root is an empty `` | + +Then merge the hardware headers map, and set content negotiation per endpoint: + +- **Plist endpoints** (`/grandslam/GsService2/validate`): `Content-Type: text/x-xml-plist` and + `Accept: text/x-xml-plist`. +- **JSON endpoints** (everything under `/auth`): `Accept: application/json`, and + `Content-Type: application/json` where there is a body. + +For reference, the `X-Apple-I-CFU-State` value is the base64 of this exact document, trailing +newline included: + +```xml + + + + + +``` + +### 7.2 Trusted-device flow + +**Request a code:** + +``` +GET https://gsa.apple.com/auth/verify/trusteddevice +``` + +with the §7.1 headers. Any non-2xx is a failure; report the status code. On success Apple pushes +a code to the user's trusted devices and the client prompts for it. + +**Submit the code:** + +``` +GET https://gsa.apple.com/grandslam/GsService2/validate +``` + +with the §7.1 headers plus a header carrying the code: + +``` +security-code: 123456 +``` + +The reply is a **property list**, not JSON. Check `ec`/`em` per §3.1. Tokens arrive in the +response headers — see §8. + +### 7.3 SMS flow + +**Discover the numbers.** SMS delivery needs the identifier of a specific trusted phone number, +obtained from: + +``` +GET https://gsa.apple.com/auth +``` + +with the §7.1 headers and `Accept: application/json`. The JSON reply carries +`trustedPhoneNumbers`, an array of: + +| Field | Meaning | +| --- | --- | +| `id` | Numeric identifier used to request delivery | +| `numberWithDialCode` | Display form, already partly masked by Apple | +| `lastTwoDigits` | Display aid | +| `pushMode` | Delivery mode | + +Two response statuses have specific meanings: + +- **`201`** — Apple has *already sent* a code to the first trusted number as a side effect of + this request. Go straight to submitting the code; requesting delivery again sends a second + code and invalidates the first. +- **`403`** — the two-factor configuration cannot be read. The reference implementations + attribute this to **Advanced Data Protection** being enabled on the account. That attribution + is unverified here, but it is the message to surface, because if it is right the user's only + remedy is to turn ADP off. Do not present it as a network error. + +> **Do not treat "no trusted phone numbers" as fatal on the trusted-device path.** This endpoint +> is the only source of phone numbers, so it is tempting to call it for both flows and abort +> when the list is empty — which breaks accounts that have trusted devices but no registered +> phone number. Only the SMS path needs a number. This is a known defect in at least one +> implementation of this protocol; do not reproduce it. + +Offer SMS whenever trusted numbers exist, even when `au` was `trustedDeviceSecondaryAuth`. A +user without a second Apple device to hand needs it, and Apple accepts it. + +**Request delivery:** + +``` +PUT https://gsa.apple.com/auth/verify/phone +``` + +§7.1 headers, `Accept: application/json`, JSON body: + +```json +{ "phoneNumber": { "id": 1 }, "mode": "sms" } +``` + +**Submit the code:** + +``` +POST https://gsa.apple.com/auth/verify/phone/securitycode +``` + +§7.1 headers, `Accept: application/json`, and the **same body as the delivery request** with the +code added: + +```json +{ "phoneNumber": { "id": 1 }, "mode": "sms", "securityCode": { "code": "123456" } } +``` + +The `phoneNumber` and `mode` fields must match what was sent to request delivery. Anything other +than HTTP 200 means the code was rejected. + +--- + +## 8. Harvesting tokens from response headers + +Both code-submission endpoints return tokens in **response headers**, not in the body. Three +header names matter, and a header may appear **more than once** — collect all values. + +| Header | Carries | +| --- | --- | +| `X-Apple-GS-Token` | A service token | +| `X-Apple-HB-Token` | The heartbeat token, service `com.apple.gs.idms.hb` | +| `X-Apple-PE-Token` | The PET | + +Each value is base64. Decode to ASCII and split on `:`. Field 0 is the service name, field 1 is +the token; what follows is inconsistent: + +| Source | Layout | Fields | +| --- | --- | --- | +| `validate` — PET | `identifier:token` — no lifetime at all | **2** [observed] | +| `validate` — heartbeat | `service:token` — **no lifetime either** | **2** [observed] | +| `validate` — service tokens | `service:token:durationSeconds` | **3** [observed] | +| `securitycode` — all | `service:token:durationSeconds:absoluteExpiry` | 4, unverified | + +**[observed] The heartbeat token carries no lifetime.** A reference implementation documents it +as `service:token:absoluteExpiry`; the live run returned two fields, not three, so it falls to +the one-year default like everything else. Trust the field count you actually receive rather +than any documented layout, this one included. + +**[observed] The first field of the PET is not a service name.** Service tokens key themselves +by their own first field, but the PET's is an account identifier — so it must be filed under +`com.apple.gs.idms.pet` by the code, not by what the header says. + +**[observed] `X-Apple-GS-Token` appeared 23 times in one response**, alongside one +`X-Apple-HB-Token` and one `X-Apple-PE-Token` — 25 tokens in total. An HTTP client that returns +only the first value for a header name will silently discard 22 of them. + +So the third field means different things depending on which endpoint answered. Rather than +special-casing the endpoint, disambiguate by magnitude — an absolute expiry in milliseconds +since 1970 is a far larger number than any plausible lifetime in seconds: + +``` +if value > 40 years expressed in milliseconds: + treat as milliseconds since the Unix epoch +else: treat as a lifetime in seconds, counted from now +``` + +For the `securitycode` layout, prefer field 3 and fall back to field 2. Defaults when no +lifetime is present: **300 seconds for the PET**, and one year (31,536,000 seconds) for +everything else. The one-year default is a guess made by the reference implementations, not +something Apple states — treat any token as possibly-expired and be ready to re-authenticate. + +### 8.1 After the second factor + +If the code-submission response carried an `X-Apple-PE-Token`, the login is complete. + +If it did not, the second factor succeeded but no session was issued. **Repeat the entire SRP +handshake from §4** with the same credentials. It will now complete without an `au` in `Status`, +and the token dictionary will arrive in the decrypted payload (§6.1) instead of in headers. This +second pass needs no user interaction — the cached password digest is enough. + +--- + +## 9. State machine + +``` + ┌──────────────┐ + │ SRP §4 │ + └──────┬───────┘ + │ Status.au? + ┌───────────────┼────────────────┐ + absent trustedDevice secondaryAuth + │ │ │ + │ ┌──────┴───────┐ ┌────┴─────┐ + │ │ offer choice: trusted device │ + │ │ or any trusted phone number │ + │ └──────┬───────┘ └────┬─────┘ + │ §7.2 request §7.3 request + │ └────────┬────────┘ + │ prompt for code + │ │ + │ submit code + │ │ + │ PET in headers? + │ ┌────┴────┐ + │ yes no + │ │ │ + └───────────────────┴──────────┴──→ repeat §4 once + │ + ┌───▼────┐ + │ Done │ + └────────┘ +``` + +Two rules that fall out of it: + +- **Bound the loop.** A repeat of §4 that again asks for a second factor is a failure, not a + reason to prompt again. Cap re-entry at one pass. +- **Present both second-factor options at once**, rather than following `au` blindly. The user + picks; the client already knows which endpoints each choice needs. + +--- + +## 10. Endpoint header sets, at a glance + +Three distinct header sets appear in this stage. Mixing them up is a plausible failure and a +tedious one to diagnose, so they are tabulated together. + +| | §3 plist service | §7.1 two-factor web | Anisette provisioning | +| --- | --- | --- | --- | +| Anisette headers copied | `MD`, `MD-M`, `MD-LU`, `MD-RINFO`, `X-Mme-Device-Id` | same five | n/a | +| `X-MMe-Client-Info` | `mmeClientInfoAkd` | `mmeClientInfo` | `mmeClientInfo` | +| `User-Agent` | `akdUserAgent` | `browserUserAgent` | `akdUserAgent` | +| `X-Apple-Client-App-Name` | `Messages` | `Messages` | `akd` | +| `X-Apple-I-Client-Bundle-Id` | ✓ | ✓ | — | +| `X-Apple-AK-Context-Type` | ✓ | ✓ | — | +| `Accept-Language` | `en-US,en;q=0.9` | `en-US,en;q=0.9` | `en-US,en;q=0.9` | +| `Sec-Fetch-*` | — | ✓ | — | +| `X-Apple-Identity-Token` | — | ✓ | — | +| Content type | `text/x-xml-plist` | per endpoint | — | + +The full §3 header set is: the five Anisette headers above, plus `X-Apple-AK-Context-Type: +imessage`, `X-Apple-Client-App-Name: Messages`, `X-Apple-I-Client-Bundle-Id: +com.apple.MobileSMS`, `X-MMe-Client-Info: mmeClientInfoAkd`, `Accept-Language: en-US,en;q=0.9`, +`User-Agent: akdUserAgent`, `Content-Type: text/x-xml-plist`, `Accept: */*`. + +--- + +## 11. Persistence, refresh and storage + +To avoid a full interactive login on every run, persist: + +| Item | Why | +| --- | --- | +| `acname` | the canonical Apple ID | +| The decrypted session payload | `adsid` and `GsIdmsToken` are needed by later stages | +| Every token, with its absolute expiry | so an unexpired token can be reused | +| **SHA-256 of the password** | so §4 can be replayed without prompting | + +Include a schema version and refuse to load anything else, so a format change fails loudly +rather than misreading old data. + +**Refreshing.** When a needed token has expired, re-run §4 with the stored Apple ID and stored +password digest. On an account with two-factor enabled this normally completes without a second +factor, because Apple has seen this machine identity before. That is exactly why §2.1 insists +the identity be stable. + +### 11.1 Security + +> **The stored password digest is password-equivalent.** SRP consumes `SHA-256(password)`, not +> the password, so anything holding that digest can authenticate as the user indefinitely. It is +> not a "hashed password" in the sense that protects a leaked database — there is no salt and no +> stretching before storage, and possession is sufficient. + +On Android this means: + +- Store it, the session payload and the tokens under hardware-backed encryption. Never in plain + `SharedPreferences` and never in a file the app's own export or backup paths can reach. +- Never log it, and never log the PET, the service tokens, `GsIdmsToken`, or `M1`/`K`. Truncated + is still too much: these are secrets, not identifiers. +- The raw password itself must never be written to disk in any form. +- Offer an explicit sign-out that deletes all of it. + +Apply the same care to Anisette provisioning state, which identifies the machine to Apple. + +--- + +## 12. Open questions + +Things this specification cannot answer without running against a real account. Each needs +confirming on first implementation, and the answer belongs back in this document. + +**Answered by the run of 2026-08-13:** + +1. ~~Which scheme does Apple select?~~ **`s2k`.** The `s2k_fo` branch remains specified but + unexercised — Apple could still select it for another account. +2. ~~Does the invented client identity pass?~~ **Yes**, and it is reflected straight back to the + user in their device list. See §13. +3. ~~Does signing in register a device?~~ **Yes.** This was the most consequential unknown, and + the answer is the unwelcome one. See §13. + +**Still open:** + +4. **Is the 403-means-Advanced-Data-Protection attribution correct?** The run returned 200 from + `GET /auth`, so the 403 path never arose. It comes from another implementation's error text, + not from Apple. If wrong, the app will tell users to disable a feature that is not the + problem. +5. **Does the SMS path work as specified?** Untested end to end. The account had one trusted + number and the trusted-device path was taken instead. Both the request and the + code-submission bodies, and the four-field token layout in §8, are unverified. +6. **How does an account with no second factor at all behave?** The `au`-absent path is + specified but is the least likely to have been exercised by anyone. +7. **Do the two code-submission endpoints return different token sets?** `validate` returned 25 + tokens; what `securitycode` returns is unknown. Stage 2 needs the PET from either. +8. **What does GSA do on repeated sign-ins from the same synthetic machine?** One run tells us + nothing about rate limiting or flagging over time. +9. **Does the iteration count vary?** One account, one observation (20082). Whether it differs + per account, or drifts, is unknown — which is the argument for never hardcoding it. + +--- + +## 13. Signing in registers a device [observed] + +Completing this stage adds an entry to the account's device list, visible at appleid.apple.com +and in iOS and macOS Settings. **No further call is required** — there is a separate endpoint by +which a client announces itself, and the verified run never touched it. Authentication alone was +enough. + +The entry is synthesised entirely from the client identity of §2.2: + +| Shown to the user | Comes from | On the verified run | +| --- | --- | --- | +| Model | the claimed hardware model | `iPhone15,2` → **"iPhone 14 Pro"** | +| Version | the claimed OS version | `17.4` → **"iOS 17.4"** | +| Name | **nothing this stage sends** | defaulted to a bare **"iPhone"** | + +So Apple resolves the model code into a marketing name and presents the result as an ordinary +device. The registered entry was, to the eye, indistinguishable from a real iPhone 14 Pro. + +**There is exactly one distinguishing marker**, on the device's detail page: + +> This device cannot be used to receive Apple Account verification codes. + +It is one line, on a subpage, and a user with a real iPhone 14 Pro must open both entries and +compare to know which is which. As a way for a person to find the right entry, that is thin. + +### Three display fields, three independent levers [observed] + +Comparing our entry against a device registered by the older macOS-VM export route — itself +synthetic, so the difference is purely what each flow sent: + +| Shown to the user | VM-registered entry | This stage alone | Set by | +| --- | --- | --- | --- | +| Name | `sb's iMac Pro` | bare `iPhone` | the device-name field of the announce call | +| Serial Number | `C02XXXXXXXXX` *(redacted)* | **absent entirely** | `X-Apple-I-SRL-NO`, among the hardware headers | +| Trusted for verification codes | **yes** | **no** | almost certainly the push token | + +**The serial is displayed when it is sent, and this stage can send it.** [observed] Supplying +`X-Apple-I-SRL-NO` among the hardware headers put the value straight onto the device's page as +its Serial Number, with no announce call and no other change. A serial is an arbitrary uppercase +alphanumeric string of roughly 10–12 characters, so it is the one identity field that can be +made to read as a label rather than as hardware. The verified run registered as: + +``` +iPhone +iPhone 14 Pro + Model iPhone 14 Pro + Version iOS 17.4 + Serial Number 0PENTAGVIEWR <- ours, and legible +This device cannot be used to receive Apple Account verification codes. +``` + +**So send one, and make it self-describing.** It costs a single header, it is the only field +this stage controls that the user actually sees, and without it Apple omits the row entirely — +leaving an entry with nothing to tell it apart from real hardware. The device page also carries +a *Remove from Account* button next to the text "If you do not recognise this device", so a +legible serial is the difference between a user confidently removing the right entry and not +daring to touch either. + +Beware the spelling: `X-Apple-I-SRL-NO` is the form the hardware-header map and remote Anisette +servers use, while macOS's own Anisette emits `X-Apple-SRL-NO`, without the `I`. The `-I-` form +is the one confirmed to work here. Note also that a serial arriving *from* an Anisette provider +is filtered out before the request is built (§10) — it must be supplied as a hardware header. + +### Do not make the device trusted + +The synthetic device is *not* trusted for verification codes, and the difference from the VM +entry points at the push token: this flow maintains no push connection and omits the field +(§5), while the VM had one. + +**Treat that as a boundary to hold rather than a gap to close.** A trusted device can receive +Apple Account verification codes — so an installation that became trusted would make itself a +second factor for the user's Apple ID. For an application whose purpose is reading accessory +locations, that is a large and unnecessary expansion of what a compromise of it would mean. +Implementing push in order to look more like a real device would buy legitimacy at exactly the +wrong price. + +### What follows for the implementation + +**The name is the problem, and this stage cannot set it.** The name field is carried by the +announce-yourself call, not by anything here. So an implementation that does only Stage 1 gets +an anonymous entry named after the hardware it claims. Two ways out, and the second is better: + +- **Choose the claimed model for recognisability.** Weak: every real model code resolves to a + name that looks like real hardware, and claiming an implausible one risks rejection. +- **Make the announce call once, with a deliberate name.** It carries a device-name field, and + is the mechanism by which an entry stops reading as "iPhone" and starts reading as + "OpenTagViewer". This needs specifying in Stage 2, and it is the recommended route. + +**One identity means one entry.** The registration keys on the machine identity, so a stable +identity re-registers nothing on later sign-ins. This is the concrete payoff of the stability +requirement in §2.1: it is not merely good hygiene, it is the difference between one entry and +one per run. + +**Removal is manual today.** The user can remove the device from appleid.apple.com or Settings. +A sign-out sequence exists in the protocol — a per-service sign-out event followed by a global +one — but whether it removes the account entry or merely signs services out is **unverified**, +and should not be claimed until it is. diff --git a/docs/findmy-export/02-mobileme-delegate.md b/docs/findmy-export/02-mobileme-delegate.md new file mode 100644 index 00000000..6f52e8ef --- /dev/null +++ b/docs/findmy-export/02-mobileme-delegate.md @@ -0,0 +1,462 @@ +# Stage 2 — MobileMe delegate + +Specification of the exchange that converts a Grand Slam session into iCloud service tokens. +This is the step between authenticating (Stage 1) and doing anything with iCloud. + +Read [README.md](./README.md) first. In particular: **implement from this document alone.** + +> **Verified against a live account on 2026-08-13**, and verified in the strongest available +> sense: the probe that exercised it was written **from this document**, not by calling any +> reference implementation. HTTP 200, top-level status 0, delegate status 0, service data +> returned, 11 tokens. So what follows is a description that someone has successfully built from. +> +> Values from that run are marked **[observed]**. One section — §6, the configuration +> dictionary — was **wrong** and has been corrected; the terms-of-service flow of §5.2 and the +> device lifecycle of §7 remain unexercised. + +--- + +## 1. What this stage produces + +A single request returns a set of named tokens and a service configuration dictionary. Three +tokens matter to this project: + +| Token | Needed by | +| --- | --- | +| `cloudKitToken` | Stage 4 — CloudKit record fetches | +| `mmeAuthToken` | Stage 4 — the general iCloud credential, paired with the numeric account id | +| `searchPartyToken` | the Find My "search party" service, which is what actually holds beacon data | + +**[observed] Eleven tokens were returned**, and all three above were among them. The full set is +in §5. + +What does **not** come back is any service configuration — see §6, and do not plan on getting +endpoints from this stage. + +## 2. Prerequisites + +From Stage 1: + +| Value | Where it came from | +| --- | --- | +| **PET** | `com.apple.gs.idms.pet`. Used as the HTTP Basic password. | +| `acname` | the account's canonical Apple ID. Used as the HTTP Basic username. | +| `adsid` | the account identifier, sent as a header | +| Anisette headers | the same provider as Stage 1, and it must be the *same machine identity* | + +> **The PET expires in about five minutes.** This is the binding constraint on how Stage 1 and +> Stage 2 fit together: either they run back to back, or Stage 1 must be replayable without user +> interaction from the cached password digest. There is no way to hold a PET across a session. + +A session is bound to the machine identity that established it (Stage 1 §2.1), so the Anisette +source must not change between the two stages. + +## 3. The request + +``` +POST https://setup.icloud.com/setup/iosbuddy/loginDelegates +``` + +**This URL encodes the client's claimed platform.** `iosbuddy` is the iOS Setup Assistant's +endpoint, and it matches the iPhone identity of Stage 1 §2.2. A client presenting as a Mac uses +a different path. Claiming iOS in the identity and macOS in the URL is the kind of mismatch that +produces an unhelpful error, so keep them consistent. + +### 3.1 Authentication + +HTTP Basic, with: + +- **username** — the Apple ID (`acname` from Stage 1, not necessarily what the user typed) +- **password** — the **PET** + +The PET is why this works without re-entering a password, and why it must be fresh. + +### 3.2 Headers + +Send **all** the Anisette headers — unlike Stage 1's GSA requests, this endpoint is not given a +filtered subset. Then add: + +| Header | Value | +| --- | --- | +| `X-Apple-ADSID` | `adsid` | +| `X-Mme-Client-Info` | `mmeClientInfoAkd` — the `com.apple.akd/1.0` variant from Stage 1 §2.2 | +| `User-Agent` | `com.apple.iCloudHelper/282 CFNetwork/ Darwin/` | +| `Accept-Encoding` | `gzip` | + +The user-agent follows the same ` CFNetwork/ Darwin/` shape as elsewhere, with the +item naming the Apple component being impersonated — here the iCloud helper rather than `akd`. +Its version must match the CFNetwork and Darwin versions of the claimed OS release. + +Two conditional headers: + +- `X-Mme-Nas-Qualify` — base64 **validation data**, an attestation blob a genuine Apple device + produces. This flow has none and **must omit the header entirely** rather than send it empty. + See §3.4. +- `Cookie` — only when re-sending after accepting terms of service. See §5.2. + +### 3.3 Body + +An XML property list. **There are two forms, and which one to send depends on whether you have +validation data.** With none — which is this project's case — send the legacy form, whose keys +are hyphenated: + +| Key | Type | Value | +| --- | --- | --- | +| `apple-id` | string | the Apple ID, same as the Basic username | +| `client-id` | string | a fresh v4 UUID, **lowercase**, per request | +| `password` | string | the **PET**, repeated here as well as in the Basic credential | +| `delegates` | dictionary | which services to request — see below | + +The newer form, for clients that do have validation data, uses camelCase `delegates`, +`protocolVersion` (`"1.0"`) and a `userInfo` sub-dictionary containing hyphenated `client-id` +(**uppercase** UUID), `language` (`en-US`) and `timezone`. It is documented here only so that an +implementer recognises it; **this project sends the legacy form.** + +`delegates` is a dictionary keyed by service identifier: + +| Key | Value | Wanted here | +| --- | --- | --- | +| `com.apple.mobileme` | empty dictionary | **yes** | +| `com.apple.private.ids` | `{ "protocol-version": "4" }` | no — iMessage identity, irrelevant to this project | + +Request only `com.apple.mobileme`. Asking for delegates you do not need means more of the +account's surface touched for no benefit. + +### 3.4 On validation data + +Validation data is an attestation blob that a real Apple device generates to prove it is one. +Producing it is a separate reverse-engineering problem, and this flow does not attempt it. + +**The endpoint accepts requests without it**, via the legacy request form — this is load-bearing +for the whole project, and is the reason the two body shapes exist. An implementation must +therefore never send an empty `X-Mme-Nas-Qualify` or an empty validation field; absence and +emptiness are not the same to this endpoint. + +## 4. The response + +An XML property list dictionary. Check it in this order — the first two checks are cheap and +produce far better diagnostics than the third. + +**1. `localizedError`.** If present, the request failed: + +| Value | Meaning | +| --- | --- | +| `UNAUTHORIZED` | the credential was rejected. Usually an expired PET — re-run Stage 1 rather than reporting an auth failure to the user. | +| anything else | report verbatim, along with the `description` key if present | + +**2. `status`.** An integer, and `0` means success. Anything else is a failure and the whole +response is the diagnostic. + +**3. `delegates`.** A dictionary keyed by the same service identifiers that were requested. Each +value is: + +| Key | Type | Meaning | +| --- | --- | --- | +| `service-data` | dictionary | the payload. **Absent when that delegate failed.** | +| `status` | integer | per-delegate status | +| `status-message` | string | per-delegate message | + +**Per-delegate status is separate from the top-level status.** The request as a whole can +succeed while an individual delegate fails, so a missing `service-data` is not a parse error — +it is that delegate's failure, and its `status` and `status-message` are the explanation. Report +them rather than dereferencing nothing. + +### 4.1 The MobileMe service data + +The `service-data` dictionary for `com.apple.mobileme` holds: + +| Key | Type | Contents | +| --- | --- | --- | +| `tokens` | dictionary | token name → token string. See §5. | +| everything else | — | the service configuration. See §6. | + +> **The configuration is the service-data dictionary itself, not a nested key inside it.** An +> earlier reading of this protocol expected the configuration under a `com.apple.mobileme` key +> *within* `service-data`, and that is wrong. Read the tokens from the `tokens` key, and treat +> the same dictionary as the configuration. Expect a nested `com.apple.mobileme` key not to +> exist, and do not fail if it does. + +## 5. Tokens + +`tokens` maps names to opaque strings. **[observed]** All eleven returned for an ordinary +account, requesting only the MobileMe delegate: + +| Name | Purpose | +| --- | --- | +| `mmeAuthToken` | the general iCloud credential. Presented as an HTTP Basic password with the **numeric** account id (`DsPrsId` from the Stage 1 session payload) as the username — note that is a *different* identifier from `adsid`. | +| `cloudKitToken` | CloudKit access, Stage 4 | +| `searchPartyToken` | **the one this project exists for** — the Find My search-party service, which holds beacon data | +| `mmeFMIPToken`, `mmeFMIPAppToken`, `mmeFMIPSiriToken` | Find My iPhone: service, application and Siri variants | +| `mmeFMFToken`, `mmeFMFAppToken`, `mmeFMFNotificationToken` | Find My Friends: service, application and notification variants | +| `keyTransparencyToken` | Apple's key-transparency service | +| `mapsToken` | Maps | + +Keep the whole map, look tokens up by name, and fail with the name that was missing rather than +an index error. The Find My families each have three variants whose distinctions are not +established here, so store all of them rather than guessing which one a later stage wants. + +### 5.1 Lifetime and refresh + +These tokens are long-lived relative to the PET — a reference implementation refreshes them +**weekly** — but the refresh path is a full repeat of this stage, which needs a fresh PET, which +needs Stage 1. So the dependency chain for any long-running use is: + +``` +cached password digest -> Stage 1 -> PET (~5 min) -> Stage 2 -> service tokens (~1 week) +``` + +Cache the delegate response with the time it was obtained. Do not try to infer expiry from the +tokens themselves; nothing in the response states one. + +### 5.2 Terms of service + +If the account has unaccepted iCloud terms, this stage fails until they are accepted. **This is +implemented in the app**, because sending the user to an Apple device to do it is a dead end for +the very users this feature exists for. + +The one rule that makes this acceptable: **the terms are fetched and displayed, and the user +performs the acceptance.** The protocol returns the terms text, so there is no reason to accept +on their behalf, and doing so would be agreeing to a contract on someone else's account without +showing it to them. Fetch, render, let them read, let them decide. + +**Step 1 — fetch the terms.** + +``` +POST https://setup.icloud.com/setup/iosbuddy/ui/genericTermsUI +``` + +with the setup headers of §5.3, plus `Accept: application/x-buddyml`, +`Content-Type: text/plist` and `X-Apple-I-Appearance: 1`. Body is a plist: + +| Key | Value | +| --- | --- | +| `format` | `plist/buddyml` | +| `terms` | array of dictionaries, each `{ "name": "iCloud" }` | + +**Step 2 — read the response.** It is *not* a plist. It is **BuddyML**, an Apple XML dialect +used by Setup Assistant, and it must be parsed as XML: + +| Element | What to take | +| --- | --- | +| `clientInfo` | its `agreeUrl` attribute — the URL that records acceptance | +| `page` | its `id` attribute, identifying which terms document this is (`iCloud`) | +| `html` | character data — the terms themselves, as HTML, usually inside CDATA | + +Configure the XML reader to **treat CDATA as characters**, or the terms text arrives empty. Take +the HTML for the page whose id matches the terms requested, and render it. + +**Step 3 — the user accepts.** Only after they have actually seen it: + +``` +POST +``` + +with the setup headers, `Content-Type: application/xml`, and no body. A non-success status means +acceptance failed and must be surfaced, not swallowed — an app that reports "accepted" when +Apple said otherwise will fail confusingly at the next step instead. + +**Step 4 — repeat this stage.** The delegate request of §3 should now succeed. + +A client presenting as a Mac uses a different route, re-sending the delegate request with a +`Cookie: termsAccepted=true` header. That path is irrelevant here — the identity of Stage 1 §2.2 +is an iPhone — and is noted only so it is not mistaken for the one to implement. + +### 5.3 Setup headers + +The terms endpoints take a different header set from §3.2. All the Anisette headers, plus: + +| Header | Value | +| --- | --- | +| `Authorization` | `Basic ` + base64 of the Apple ID + `:` + the **PET** | +| `User-Agent` | `iOS iPhone iPhone Setup Assistant` | +| `X-MMe-Client-Info` | ` ` | +| `X-MMe-Country` | `US` | +| `X-MMe-Language` | `en,en-US` | +| `Cookie` | `repairSteps=` | + +Note the bundle in `X-MMe-Client-Info` is Preferences here, not `akd` — these endpoints are +addressed as Settings would address them. + +## 6. The configuration dictionary + +> **Correction. [observed] There is no configuration.** An earlier draft of this section, taken +> from reading how other services use this protocol, said the service data carries per-dataclass +> endpoints keyed as `com.apple.Dataclass.`. For a request that asks only for +> `com.apple.mobileme`, it does not. The service data held `tokens` and exactly one other key: +> +> | Key | Value | +> | --- | --- | +> | `protocolVersion` | a version marker | +> +> No dataclass keys, no service URLs, and no nested `com.apple.mobileme` key either. + +Dataclass configuration does exist in this protocol — other clients read a quota URL and a +shared-streams URL from exactly such keys — so the likeliest explanation is that Apple returns +configuration only for dataclasses the client asks about or claims to support, and this request +asks for none. That is a hypothesis, not a finding. + +**What follows for Stage 4, now settled:** the CloudKit *credential* comes from here, but the +CloudKit *endpoint* does not — and it turns out not to be needed from here either. Opening the +container returns the service URLs directly. See +[04-cloudkit.md §2.3](./04-cloudkit.md). Nothing further is required of this stage. + +## 7. Naming the registered device + +[Stage 1 §13](./01-authentication.md#13-signing-in-registers-a-device-observed) established that +signing in registers a device on the account, and that this stage's predecessor cannot set its +name — the entry appears named after whatever hardware the client claims to be. A separate +endpoint carries a device name, and calling it once is what turns a bare `iPhone` into something +the user recognises. + +``` +POST https://gsas.apple.com/grandslam/GsService2/postdata +``` + +> **Note the host: `gsas.apple.com`, not `gsa.apple.com`.** One letter, a different host, and no +> useful error if you get it wrong. + +This is a Grand Slam endpoint, not an iCloud one, so it takes GSA-style headers rather than the +ones in §3.2: the Anisette subset, plus `Content-Type: text/x-xml-plist`, plus + +| Header | Value | +| --- | --- | +| `X-Apple-HB-Token` | base64 of `adsid` + `:` + the `com.apple.gs.idms.hb` token from Stage 1 | +| `X-Apple-I-UrlSwitch-Info` | base64 of `adsid` + `:postdata` | +| `X-MMe-Client-Info` | `mmeClientInfoAkd` | +| `User-Agent` | `akdUserAgent` | +| `X-Apple-I-Service-Type` | `itunesstore` | +| `X-Apple-I-CDP-Status`, `X-Apple-I-OT-Status`, `X-Apple-I-CK-Presence` | `true` | +| `X-Apple-AK-DataRecoveryService-Status` | `1` | +| `X-Apple-I-Device-Configuration-Mode`, `X-Apple-I-DeviceUserMode`, `X-Apple-Requested-Partition`, `X-Apple-I-TimeZone-Offset` | `0` | +| `x-apple-i-device-type` | `1` | + +Body is the usual `Header`/`Request` plist envelope. The `Request` dictionary announces the +client's state; the field that matters here is `dn`, the device name: + +| Key | Type | Value | +| --- | --- | --- | +| `dn` | string | **the device name** — what appears in the account's device list | +| `event` | string | `liveness` | +| `loc` | string | `en_US` | +| `services` | array | empty, for a client that provides none | +| `cdpStatus`, `circleStatus`, `otStatus`, `icscStatus`, `prkgen`, `denyICloudWebAccess` | boolean | `true`, except `icloudMailEnabled` and `stingrayDisabledIndicator` which are `false` | +| `rep`, `ut`, `signinPartition`, `isLegacyContactAssignee`, `isRecoveryContactAssignee` | integer | `1` | +| `reason` | integer | `5` | +| `usrt` | integer | `4` | +| `pkc` | string | `"1"` | +| `cfuids` | array | empty | +| `ptkn` | string | an APNs push token, **omitted entirely** by this flow — see below | + +**Never send `ptkn`.** The push token is the most likely reason a device becomes trusted for +verification codes, and Stage 1 §13 argues at length that this client must never become a second +factor for the user's Apple ID. Omit the key entirely rather than sending it empty. + +## 7.1 The device is the app's identity, not a temporary artefact + +**[observed, Stage 1 §7]** A second sign-in from the same machine identity required no second +factor. Apple remembers the machine, and stops challenging it. + +That matters because this feature is not a one-shot export. Noticing a newly-paired accessory +means re-reading the account periodically, which means re-authenticating periodically — and that +is only silent while a device registration exists for Apple to recognise. **Registering and +removing around each operation would demand a two-factor code every time**, which is fatal for a +background refresh and worse for the user than the entry it avoids. + +So the registration is created once and kept for the life of the installation. + +### Announce it, in the app, in plain terms + +The user should learn what exists on their account from the application, not from finding it +themselves: + +> Signing in registers this app as a device on your Apple account, so Apple will keep talking to +> it without asking for a verification code every time. It appears in your device list as +> **OpenTagViewer**, serial **0PENTAGVIEWR**. It cannot receive verification codes. Signing out +> removes it. + +Both identifiers must be the ones actually sent — the name from `dn` above, the serial from +`X-Apple-I-SRL-NO` in Stage 1 §2.2 — because those are the two fields visible in the device list, +and they are what the user will match against. + +### Remove it at sign-out, and only then + +Sign-out is the one moment removal is correct: the user has deliberately disconnected the +account, so the registration has no further purpose and a two-factor prompt is no longer a cost +anyone is paying. The protocol offers a sequence of sign-out events through the same endpoint, +with the request body changed: + +| Key | Value | +| --- | --- | +| `event` | `signout-`, and finally `signout-all` | +| `dn` | the same device name | +| `services` | the services **still** signed in — i.e. shrinking to empty across the sequence | +| `cdpStatus`, `circleStatus`, `otStatus` | `false` | +| `prkgen` | `true` | +| `rep`, `ut` | `1` | +| `loc` | `en_US` | + +and the headers to match: `X-Apple-I-CDP-Status` and `X-Apple-I-OT-Status` become `false`, and +`X-Apple-I-Service-Type` and `X-Apple-AK-DataRecoveryService-Status` are dropped entirely. + +The reference walks every service it might have used — iCloud, iMessage, FaceTime — signing each +out in turn with the remainder listed, then a final `signout-all`. This client only ever uses +iCloud, so `signout-icloud` followed by `signout-all` is the plausible minimum, **and that is an +inference, not an observation**. + +> **Two things here are unverified and must not be stated to the user as fact.** Whether this +> sequence removes the device from the account list at all, rather than merely signing services +> out. And how far `signout-all` reaches — it carries a device name, which *suggests* it is +> scoped to this device, but nothing here has confirmed that. Until both are established, prefer +> attempting `signout-icloud` alone, report what was attempted rather than what was achieved, +> and point the user at their device list. + +### The failure path is the one that matters + +A crash, a lost connection or a force-quit during sign-out leaves the registration behind. That +is the path back to an accumulating device list, so: + +- Removal must be **idempotent and retried** — attempt it again on next launch if the last + sign-out did not complete. +- The app should be able to say *"an earlier sign-out may not have finished; a device called X + may still be listed"* and link the user to their device list, rather than staying silent. +- **Never regenerate the identity to recover from a failed removal.** A fresh identity does not + clean up the old entry, it adds a second one — which is exactly how twenty of them appear. + +### The escrow record follows the same logic, in reverse + +Stage 3 creates an escrow record whose purpose is to let the client *re-establish* keychain trust +if it loses its local state. So the two artefacts have opposite lifetimes: the device registration +is what keeps ordinary access working and is kept, while the escrow record is only insurance and +can be deleted once trust is established — at the cost of needing the user's device passcode +again should local state ever be lost. Stage 3 specifies that trade properly. + +## 8. Open questions + +**Answered by the run of 2026-08-13:** + +1. ~~Does the legacy request form still work?~~ **Yes.** This was the load-bearing unknown for + the whole project — the basis for authenticating without an attestation blob. HTTP 200, + status 0, tokens returned. Apple has not withdrawn it. +2. ~~What is the full token set?~~ **Eleven names**, listed in §5. +3. ~~Which configuration key carries the CloudKit endpoint?~~ **None of them — no configuration + is returned at all.** See the correction in §6. This moves to Stage 4. +4. ~~Is the configuration nested under a `com.apple.mobileme` key?~~ **No**, and there is no such + key. + +**Still open:** + +5. **Is `UNAUTHORIZED` really always an expired PET?** It is the obvious cause and the useful + default, but treating every rejection as "retry Stage 1" risks an infinite loop against an + account that is genuinely refused. Bound the retry. +5. **Does the device-name call in §7 rename an existing entry, or only name a new one?** If it + only applies at registration, calling it later is useless and the serial is the only lever. +6. **Does the sign-out sequence remove the device from the account, or only sign services out?** + §7.1 commits the app to removing what it creates, and this decides whether that promise can + be kept at all. It is directly testable: run it and watch the device list. +7. **Does re-registering after a removal create a fresh entry or restore the old one?** Decides + whether "remove at sign-out" or "reuse one identity forever" is the right policy. +8. **What do the terms endpoints return for an account with nothing outstanding?** The + acceptance flow of §5.2 must not fire spuriously, so the no-terms-pending response needs to + be distinguishable. +6. **Does requesting only `com.apple.mobileme` return the same service data as requesting it + alongside the IDS delegate?** Assumed yes; unverified. diff --git a/docs/findmy-export/03-keychain-trust.md b/docs/findmy-export/03-keychain-trust.md new file mode 100644 index 00000000..d44a7512 --- /dev/null +++ b/docs/findmy-export/03-keychain-trust.md @@ -0,0 +1,518 @@ +# Stage 3 — iCloud Keychain trust circle + +Specification of how a client joins the user's iCloud Keychain trust circle, which is what makes +the PCS keys of Stage 5 reachable and therefore what makes the accessory records of Stage 4 +readable. + +Read [README.md](./README.md) first. In particular: **implement from this document alone.** + +> **The read-only half is verified against a live account on 2026-08-13** — host derivation, +> transport, error envelope, and the record schema of §5, all by a probe written from this +> document. Values from that run are marked **[observed]**, and several corrected what reading +> the reference had suggested. +> +> **Everything from §6 onward is unverified**: recovery, joining the circle, enrolling and +> deleting. Those are derived by reading implementations whose authors report them working. +> +> It is also the stage that **needs the user's device passcode** and **writes to their account**. +> Everything before it was read-only; this is not. Treat the residue rules of §7 as part of the +> specification rather than as advice. + +--- + +## 1. What this stage is for + +Find My accessory records are encrypted. Their keys live in the user's iCloud Keychain, which is +shared between the devices the user has explicitly trusted. To read the keys, this client has to +become one of those trusted devices. + +Apple's mechanism for admitting a device that cannot be approved by tapping "Allow" on an +existing one is **escrow recovery**: the user's keychain material is escrowed in a record +protected by a device's screen-lock passcode, and anyone who knows that passcode can recover it. + +That is why this feature needs a passcode it never uses for anything else, and it is the honest +answer to "does this remove the need for an Apple device": it removes the need to *have* one, not +the need for one to have *existed*. An Apple ID whose keychain circle has never contained a +device has nothing to recover from. + +## 2. Two services, joined by an identifier + +This stage talks to two unrelated-looking services, and understanding the split makes everything +else legible: + +| Service | Transport | Holds | +| --- | --- | --- | +| **Escrow proxy** | plain HTTPS, plist bodies | the escrow *records* — the passcode-protected material and its metadata | +| **Cuttlefish** | CloudKit function invocation | the trust *circle* — peers, bottles, keys, who trusts whom | + +"Cuttlefish" is the server side of Octagon, Apple's trust-circle protocol. "Bottle" is its name +for a sealed piece of key material a peer can be reconstituted from. + +Listing what the account has requires **both**: the escrow proxy knows the human-readable +metadata, Cuttlefish knows which bottles are actually usable, and they are joined on the bottle +identifier. Neither alone answers "what can I recover from". + +### 2.1 Cuttlefish rides on CloudKit + +Cuttlefish is not a REST service. It is invoked as a **server-side function through a CloudKit +container**: + +| Property | Value | +| --- | --- | +| Container id | `com.apple.security.keychain` | +| Bundle id | `com.apple.security.cuttlefish` | +| Database | private | +| Environment | `Production` | + +Calls are made with the function-invoke operation — field `1101` of `RequestOperation`, per +[Stage 4 §3.2](./04-cloudkit.md) — naming the target `Cuttlefish`, the method, and a protobuf +body. The response body is protobuf too. + +**The endpoint is a different service from the record database**: + +``` +POST https://gateway.icloud.com/ckcoderouter/api/client/code/invoke +``` + +derived from `cloudKitCodeGatewayUrl` rather than `cloudKitDatabaseGatewayUrl`, both of which +[Stage 4 §2.3](./04-cloudkit.md) returns. The CloudKit headers, envelope and authentication are +otherwise identical to any other operation. + +> **So Stage 4 is a hard prerequisite for Stage 3, not merely a convenient one.** The CloudKit +> transport, its container-open step, its headers and its envelope are all needed here. The +> ordering in this document set — specifying and verifying Stage 4 before writing Stage 3 — was +> the correct dependency order and not just risk management. + +A second container, `com.apple.securityd` over the same `com.apple.security.keychain` container +id, exists for keychain item operations proper. + +## 3. Prerequisites + +| Value | From | +| --- | --- | +| **PET** | Stage 1. Used as the escrow proxy's HTTP Basic password — so the five-minute expiry applies here too. | +| The account's email | Stage 1 `acname`, the Basic username | +| A working CloudKit container | Stage 4 §2, for every Cuttlefish call | +| `com.apple.gs.icloud.escrow.auth` | Stage 1 issues this token; whether this stage needs it, or only the PET, is unestablished | +| **The user's device passcode** | the user, at the moment of recovery. Never stored. | +| Anisette headers | the same identity as every other stage | + +## 4. The escrow proxy + +``` +POST /escrowproxy/api/ +``` + +> **[observed] The derivation below is confirmed.** `p-escrowproxy.icloud.com` answered a +> request from an account on partition 24 with a well-formed service response. The escrow proxy +> is partitioned like every other per-account host, and deriving the host from the CloudKit +> partition works. + +**Deriving the host is the first problem, and the protocol does not hand it over.** + +The intended source appears to be a key named `escrowProxyUrl`, nested somewhere inside the +MobileMe delegate's configuration — the reference searches its whole configuration tree +recursively for it, which is itself a sign that its position is not fixed. + +> **[observed] It is not there.** The delegate response for this client contains no configuration +> at all beyond a `protocolVersion` — see [Stage 2 §6](./02-mobileme-delegate.md). Independently, +> the reference exporter reports the same thing and falls back to a **hardcoded** +> `p97-escrowproxy.icloud.com`. + +That hardcoded value is a bug waiting to happen, and the reason is visible in +[Stage 4 §2.3](./04-cloudkit.md): `p97` is a **partition prefix**. iCloud shards accounts across +numbered partitions, every per-account service host is named `p-.icloud.com`, and the +account observed for these documents is on a different partition entirely. + +**So derive it.** Opening the CloudKit container returns URLs such as +`p-ckdatabase.icloud.com`; take the partition number from any of them and construct: + +``` +https://p-escrowproxy.icloud.com:443 +``` + +**[observed]** This works. A client should still fail legibly rather than mysteriously if the +host does not answer, since the derivation is a pattern rather than a documented rule. Whether +another partition's proxy would also serve the account is untested and does not need to be. + +### 4.1 Errors + +A failure returns a plist with `success`, `errorCode` and `errorMessage`. **[observed]** The +messages are specific and worth surfacing verbatim — `Wrong command sent: get_records` named the +exact mistake. Check `success` before reading anything else. + +### 4.2 Transport + +| Aspect | Value | +| --- | --- | +| Authentication | HTTP Basic: the account **email**, and the **PET** as password | +| `Content-Type` | `application/x-apple-plst` | +| Body | an XML property list | +| `User-Agent` | `com.apple.sbd/638.100.48 CFNetwork/ Darwin/` | +| `X-Mme-Client-Info` | the client-info string of Stage 1 §2.2, bundle `com.apple.AuthKit/1 (com.apple.sbd/638.100.48)` | +| `Accept` | `*/*` | +| `Accept-Language` | `en-US,en;q=0.9` | +| `X-Apple-I-Locale` | `en_US` | +| `x-apple-i-device-type` | `1` | + +Plus all the Anisette headers. + +`sbd` is *secure backup daemon* — Apple's internal name for this mechanism, and the reason +escrow records are called secure-backup records throughout. Note the unusual content type: +`application/x-apple-plst`, not the `text/x-xml-plist` used by Grand Slam. + +### 4.3 Commands + +> **Each command is named twice, in two different spellings, and both must be right.** The URL's +> last path segment is lower_snake_case; the body's `command` field is SCREAMING_SNAKE_CASE, and +> it is *not* simply the uppercase of the path. Sending the path spelling in the body is rejected +> with `Wrong command sent: ` — **[observed]**, and it is the failure this document +> originally caused by describing the two as one thing. + +| Command | URL path segment | Body `command` value | Purpose | +| --- | --- | --- | --- | +| Get records | `get_records` | `GETRECORDS` | **list escrow records** — read-only | +| Get club certificate | `get_club_cert` | `GETCLUB` | fetch the certificate chain used when enrolling | +| SRP init | `srp_init` | `SRP_INIT` | begin passcode-authenticated recovery | +| Recover | `recover` | `RECOVER` | complete it and obtain the escrowed material | +| Enroll | `enroll` | `ENROLL` | **create** an escrow record | +| Delete | `delete` | `DELETE` | **destroy** an escrow record | + +Note the irregularity: `get_club_cert` becomes `GETCLUB`, dropping a word, while `srp_init` +becomes `SRP_INIT`, keeping its separator, and `get_records` becomes `GETRECORDS`, losing its. +There is no rule to derive one from the other — the pairs must be tabulated. + +Only `get_records` is free of consequences. `enroll` and `delete` both modify the account, and +`delete` is destructive to something a real device may depend on. + +### 4.4 Request body + +A dictionary. Fields vary by command; the ones common to all: + +| Key | Type | Meaning | +| --- | --- | --- | +| `command` | string | the command's **body spelling** — see the table above, not the URL segment | +| `label` | string | which record class is being addressed — see below | +| `transactionUUID` | string | a fresh uppercase v4 UUID per request | +| `userActionLabel` | string | a free-text description of *why*, for Apple's logs | +| `version` | integer | `1` | + +and, per command: `blob`, `blobDigest`, `metadata`, `dsid`, `silentAttempt`, +`baseRootCertVersions`, `trustedRootCertVersions`. + +Two labels appear: + +| Label | Used for | +| --- | --- | +| `com.apple.securebackup.record` | escrow records themselves — the label for `get_records` | +| `com.apple.icdp.record` | the club certificate, when enrolling | + +> **`userActionLabel` is a free-text string that Apple keeps.** The reference sends descriptions +> of the operation being performed. It is not authentication and nothing checks it, but it is +> written into someone else's audit trail, so it should describe what is actually happening and +> should not impersonate a first-party Apple process. + +## 5. Listing what the account has — read-only + +This is the part worth building first: it shows the user what exists, creates nothing, and needs +no passcode. + +**Step 1 — escrow metadata.** `get_records` with label `com.apple.securebackup.record`. + +**[observed]** The response envelope carries `status`, `message`, `version`, `dsid` and +`metadataList`. Check `status` and surface `message`; the list is the payload. `metadataList` is +an array of: + +| Key | Meaning | +| --- | --- | +| `label` | **the bottle identifier** — the join key | +| `metadata` | base64 of an XML plist: the record's descriptive fields | + +**Step 2 — viable bottles.** Cuttlefish `fetchViableBottles`, with a filter value of `1` and an +empty metrics list. The response carries a list of valid bottles, each with an id. + +**Step 3 — join them.** Match each bottle's id against a metadata entry's `label`. A bottle +without metadata cannot be described to the user; metadata without a viable bottle cannot be +recovered from. **Report both mismatches rather than silently dropping them** — they are the +signal that this specification has drifted from what Apple returns. + +### 5.1 What the metadata contains + +Decoding the base64 plist gives: + +| Key | Meaning | +| --- | --- | +| `ClientMetadata` | a dictionary holding `device_name`, `device_model`, `device_model_class` | +| `serial` | the device's serial number | +| `build` | its OS build | +| `com.apple.securebackup.timestamp` | when the record was escrowed | +| `bottleID` | the bottle's own identifier | +| `escrowedSPKI` | the escrowed public key information | +| `passcodeGeneration` | which generation of the device's passcode protects it. **[observed]** camelCase, and present on only the newest records — 2 of 12 on the observed account. | +| `SecureBackupUsesMultipleiCSCs` | a legacy flag, from the iCloud Security Code era | + +**This is a copy of a device's identity**, which is why there is no separate escrow UI in any +Apple interface — see the [README](./README.md). Present it to the user as device name, model, +serial and escrow date, because those are the fields they can match against their own device +list. + +> ### [observed] Escrow records outlive the devices that created them +> +> On the account these documents were written against, the device list showed **one** iMac Pro +> while the escrow proxy held **eight** `iMacPro1,1` records — phantoms left by an older +> macOS-VM export route whose device entries had since been removed. +> +> **Removing a device from the account does not remove its escrow record.** The device entry is +> the visible artefact; the escrow record is the durable one, and it accumulates in the one place +> Apple provides no interface to inspect. +> +> Two consequences, and they point in opposite directions: +> +> - **The residue rules matter more here than for the device registration**, not less. An +> uncleaned escrow record is permanent as far as the user is concerned, because they cannot +> see it to clean it up. +> - **The list a user is shown before any deletion will contain records they no longer recognise** +> — devices they have sold, wiped or removed years ago. That makes "delete the one you don't +> recognise" actively dangerous advice, and is why deletion must confirm against the serial of +> a specific record rather than inviting a judgement call. + +**Treat the schema as unstable — [observed], it genuinely is.** Of twelve records on one +account, eleven had the device shape above and one had a different shape entirely: +`BackupKeybagDigest`, `ClientMetadata` and a timestamp, with **no** serial, build, `bottleID` or +`escrowedSPKI`. `passcodeGeneration` appeared on only two. + +So there is more than one kind of record under this label, and a client that assumes the device +shape will fail on the others. Decode defensively, report what could not be understood, and never +discard a record silently. A record without a `bottleID` cannot be a recovery candidate and +should be filtered out rather than treated as broken. + +## 6. Recovering — the passcode step + +Recovery is an **SRP exchange in which the device's passcode is the password**. That is why a +passcode is required: it is not a credential Apple checks, it is the secret that decrypts the +escrowed material, and Apple cannot recover it either. + +**The passcode is used twice** — once as the SRP password, and again as a PBKDF2 input to unwrap +the innermost blob (§6.5). Both uses are essential; neither substitutes for the other. + +### 6.1 The KeyVault message framing + +Both escrow blobs use a framing that appears nowhere else in this protocol. Given a header length +*H* and a section count *S*: + +| Offset | Content | +| --- | --- | +| 0 | 4 bytes, skipped | +| 4 | the header, *H* bytes | +| 4 + *H* | *S* offsets, each a 4-byte big-endian integer | + +Each section then lives at `(H + 4 + (S + 1) × 4) + offset`, and begins with its own 4-byte +big-endian length followed by that many bytes. + +> Note the `S + 1` in the base calculation: there is **one more offset than there are sections**, +> the last marking end-of-data. Computing the base from *S* rather than *S* + 1 puts every +> section four bytes out and produces garbage that looks like a decryption failure. + +### 6.2 Begin the exchange + +Send `srp_init` (§4.3) with `label` set to the chosen record's `bottleID`, and `blob` set to the +base64 of the SRP client's public value **A**. + +SRP parameters match [Stage 1 §4.1](./01-authentication.md): the RFC 5054 2048-bit group, +SHA-256, and a 32-byte client secret. + +The response carries `respBlob`, `dsid`, and an optional `clubTypeID`. Parse `respBlob` with +*H* = 24 and *S* = 3: + +| Section | Contents | +| --- | --- | +| 0 | a request identifier, echoed back in §6.3 | +| 1 | the SRP **salt** | +| 2 | the server public value **B** | + +The 24-byte header is two 32-bit values followed by a 16-byte request id. + +### 6.3 Compute the proof + +> **This is standard SRP-6a, unlike Stage 1.** Stage 1 §4.6 documents three deviations for Grand +> Slam; **none of them apply here.** The identity *is* included in the private-key hash, and the +> password is the passcode **directly** — not stretched through PBKDF2 first as `s2k` requires. +> The same SRP code serves both only if it is parameterised, and getting this backwards produces +> an authentication failure indistinguishable from a wrong passcode. + +The identity is the **`dsid` string from the `srp_init` response**, not the Apple ID and not the +`adsid` from Stage 1. + +Then build the response blob: take the 24-byte header from §6.2 and change two fields — set the +first 32-bit value to **165**, and the second to **2** if `clubTypeID` is 1, otherwise **0**. +Frame it as a KeyVault message with two sections: the request identifier from section 0, **sized +to 20 bytes**, followed by the SRP proof **M1**. + +Send that as `blob` with the `recover` command, same `label` and same transaction id. + +### 6.4 Unwrap the outer blob + +Parse the response `respBlob` with *S* = 3 and *H* = **40 if `clubTypeID` is 1, otherwise 24**. + +Verify the server's proof **M2** against section 0 before going further. + +The version is a 4-byte big-endian integer at **offset 4 within the header**, and it selects the +cipher: + +| Version | Cipher | Key | IV / nonce | Ciphertext | +| --- | --- | --- | --- | --- | +| 0 | AES-256-CBC | the SRP session key | section 1 | section 2 | +| 2 | AES-256-GCM, **16-byte nonce** | the SRP session key | section 1 | section 2 | +| 1 | **unknown** — refuse rather than guess | | | | + +### 6.5 Unwrap the inner blob + +The plaintext from §6.4 is itself a KeyVault message, with *H* = 16 and *S* = 6. Its header is +four 32-bit values, of which **the third is a PBKDF2 iteration count**. + +``` +derived = PBKDF2-HMAC-SHA256(passcode, salt = section 1, iterations = header[2], length = 16) +material = AES-128-CBC-decrypt(derived, iv = first 16 bytes of section 1, data = section 3) +``` + +Note that section 1 serves as **both** the PBKDF2 salt and, in its first 16 bytes, the CBC +initialisation vector. The result is the escrowed keychain material. + +> **Sections 0, 2, 4 and 5 are unaccounted for.** Six sections are framed; three are used. What +> the others carry is unestablished. + +### 6.6 Certificate pinning + +The escrow service is authenticated against a **pinned set of certificates**, referenced by the +`baseRootCertVersions` and `trustedRootCertVersions` fields of §4.4 — the reference offers +versions 101, 102, 103 and 500. The certificates themselves must be obtained; this document does +not carry them. + +**This matters more here than elsewhere.** The material crossing this exchange is the user's +entire keychain, and the passcode protecting it is short enough to brute-force offline if an +attacker obtains the blob. Do not disable verification, and do not fall back to the system trust +store because pinning was inconvenient. + +### 6.7 After recovery + +The recovered material lets the client reconstitute itself as a peer in the trust circle via +Cuttlefish (§2.1). **That sequence is not specified here** — establishing or joining via a +voucher, and syncing trust changes, are named in the protocol and remain unwritten. See §8. + +> **Passcode handling.** Read it, use it twice, discard it. Never written to disk, never logged, +> never in diagnostics, and not retained in memory past the exchange. It is the most sensitive +> value this project touches: it is the key to the user's whole keychain, and unlike an Apple ID +> password it cannot be rotated without physical access to the device. + +### What this stage must actually deliver + +[Stage 5 §2](./05-pcs-decryption.md) makes the output contract concrete, and it is narrower than +"join the circle": + +| Required | Detail | +| --- | --- | +| The **`Manatee`** keychain view, synced | the view holding Find My's Protected Cloud Storage keys | +| The **`ProtectedCloudStorage`** view, synced | synced alongside it | +| The service key within `Manatee` | the item labelled `com.apple.ProtectedCloudStorage-com.apple.icloud.searchparty` | +| Items readable as dictionaries | key lookup matches on the `acct` attribute, which holds base64 of a compressed EC public key | + +So success for this stage is not "we are in the circle" but "we can enumerate `Manatee` and find +an item whose `acct` matches a key referenced by a record's protection structure". That is the +thing to test against. + +> **Passcode handling.** It must be read, used, and discarded. Never written to disk, never +> logged, never included in diagnostics, and not retained in memory past the exchange. It is the +> single most sensitive value this project ever touches: it is the key to the user's entire +> keychain, and unlike an Apple ID password it cannot be rotated without physical access to the +> device. + +## 7. What this stage creates, and the rules for it + +Joining the circle enrolls **an escrow record of this client's own**, via `enroll`. That is a +second artefact on the user's account, after the device registration of Stage 1 §13. + +The [README](./README.md) sets out the standing rules. Applied here: + +- **One record, created once.** Never a new one per run. +- **It inherits the labelling for free.** A record created with the identity of Stage 1 §2.2 + carries the same recognisable serial, so it identifies itself by the same means the device + entry does — provided the identity is consistent, which it must be anyway. +- **It is insurance, not access.** Its purpose is to let the client re-establish trust if it + loses local state. Once trust is established and the peer identity is stored locally, the + record can be deleted — at the cost of needing the user's passcode again should that local + state ever be lost. This is the opposite lifetime to the device registration, which is kept + precisely because it is what keeps ordinary access working. +- **Deletion must confirm against the serial.** The list contains the user's *real* devices, and + every entry looks structurally alike. Destroying the wrong record destroys that device's + ability to recover its keychain. Require the user to re-enter the serial of the record being + deleted; never accept a list position. +- **Never delete without listing first**, and never offer a "clean up all" action. + +## 7.1 Deleting an escrow record + +Deletion is **two calls**, both `delete`, in this order: + +| Order | `label` | +| --- | --- | +| 1 | `.double` — a companion record | +| 2 | `` | + +Both carry only `command`, `label`, `transactionUUID`, `userActionLabel` and `version`. The +`.double` suffix names a paired record that exists alongside the main one; deleting only the main +record appears to leave it behind. + +> **The protocol requires nothing but the label.** No password, no blob, no proof that the caller +> could have recovered the record. Any client holding a valid PET can delete **any** escrow record +> on the account, including one belonging to a real device that the user still depends on. + +Two things follow, and they are the whole design of this operation: + +**The client-side confirmation is the only protection that exists.** There is no server-side +check to fall back on. So: list first, show the full identifying detail — device name, model, +serial, escrow date — require the user to re-enter the **serial** of the record being deleted, +and never accept a list position. Never offer a bulk or "clean up all" action. + +**Requiring recovery before deletion is a choice, not a constraint.** The reference exporter +demands a successful recovery with the record's password before it will delete, which is a sound +guard against deleting the wrong thing — but it also makes it *impossible* to remove records +whose passcode is no longer known. That matters: [§5](#5-listing-what-the-account-has--read-only) +observes that escrow records outlive their devices, so an account can accumulate records for +hardware that is long gone and whose passwords nobody remembers. An implementation that copies +the guard uncritically cannot clean those up. + +**The recommendation** is to separate the two cases: + +- **Deleting this application's own record** — identified by the serial it was created with, per + Stage 1 §2.2 — needs no recovery guard. The client knows it made it, and it knows its own + escrow password. +- **Deleting anything else** is a maintenance action, should be presented as one, and should + require the serial typed out in full. Whether to additionally require recovery is a judgement + about whether the user is more likely to be cleaning up known debris or about to destroy a real + device's recoverability. + +## 8. Open questions + +This stage has more of them than the rest of the set combined, and they are load-bearing. + +1. ~~Is the escrow host really `p-escrowproxy.icloud.com`?~~ **[observed] Yes.** Derived from + the account's CloudKit partition, and the service answered. Ruled out as sources along the + way: the MobileMe delegate config (carries no configuration at all), and `ckAppInit`'s + `values` array — that is a per-environment endpoint table for the container being opened, one + row each for `PRODUCTION` and `SANDBOX`. +2. **What is the exact SRP exchange for recovery?** §6 is a sketch. The parameters, the + derivation from the passcode, and the response format all need specifying before anything can + be implemented. +3. **What does joining the circle actually involve after recovery?** Reconstituting a peer, + establishing or joining via a voucher, and syncing trust changes are all named in the protocol + and none are specified here. +4. **Which token authenticates what?** Stage 1 issues `com.apple.gs.icloud.escrow.auth`, but the + escrow proxy is authenticated with the PET. Whether that token is needed at all is unknown. +5. **Can bottles be listed without any prior trust state?** §5 is presented as read-only and + safe, and the reference performs it before joining — but it also constructs a keychain client + first, and what that construction requires is not established. If listing turns out to need + local trust state that only exists after joining, the safe read-only step is not available. +6. **Does an account with Advanced Data Protection enabled behave differently?** ADP changes how + keychain material is protected, and Stage 1 §7.3 already notes an unverified claim that it + breaks the two-factor configuration endpoint. It is likely to matter more here. +7. **What happens on an account whose keychain circle is empty?** Presumably no viable bottles + and no way in. That case needs detecting and explaining to the user, not failing obscurely. diff --git a/docs/findmy-export/04-cloudkit.md b/docs/findmy-export/04-cloudkit.md new file mode 100644 index 00000000..9bce0486 --- /dev/null +++ b/docs/findmy-export/04-cloudkit.md @@ -0,0 +1,625 @@ +# Stage 4 — CloudKit + +Specification of how to reach the Find My accessory records, which live in a private CloudKit +container. + +Read [README.md](./README.md) first. In particular: **implement from this document alone.** + +> **§2 is verified against a live account on 2026-08-13**, by a probe written from this document +> rather than from any reference implementation: HTTP 200, container opened, `cloudKitUserId` +> returned. Values from that run are marked **[observed]**. +> +> **§3 is now detailed enough to implement from, but is unverified** — it is derived by reading, +> and no record has been fetched. Expect corrections; Stage 1 had three and Stage 2 had one, +> and both were better understood going in than this is. +> +> It is written out of protocol order, before Stage 3, deliberately: §2 needs nothing from the +> keychain, so it can prove CloudKit access works *before* anyone commits to the expensive and +> account-modifying work of joining the trust circle. If §2 fails, Stage 3 is pointless. + +--- + +## 1. What this stage needs, and what it produces + +**Needs**, all of which [Stage 2](./02-mobileme-delegate.md) already produced: + +| Value | From | +| --- | --- | +| `mmeAuthToken` | the MobileMe delegate tokens | +| `cloudKitToken` | the same, for record operations | +| `DsPrsId` | the **numeric** account identifier from the Stage 1 session payload — not `adsid` | +| Anisette headers | the same identity as every other stage | + +> **These are already in OpenTagViewer's database.** The app's existing FindMy.py session +> persists the whole MobileMe service-data, `cloudKitToken` included, even though FindMy.py has +> no CloudKit code and never uses it. This stage may not need any new authentication at all. + +**Produces:** the encrypted Find My accessory records. They are **not readable at this point** — +their contents are protected by PCS (Stage 5), whose keys come from the keychain (Stage 3). This +stage gets the ciphertext and the metadata around it. + +## 2. Opening the container + +Find My accessory data lives in one private CloudKit container: + +| Property | Value | +| --- | --- | +| Container id | `com.apple.icloud.searchparty` | +| Bundle id | `com.apple.icloud.searchpartyd` | +| Database | **private** | +| Environment | `Production` | + +"Search party" is Apple's internal name for the Find My network. It is the same service the +`searchPartyToken` of Stage 2 belongs to. + +### 2.1 The request + +``` +POST https://gateway.icloud.com/setup/setup/ck/v1/ckAppInit?container=com.apple.icloud.searchparty +``` + +**Authentication is HTTP Basic**, and the credential pair is worth stating precisely because both +halves are easy to get wrong: + +- **username** — `DsPrsId`, the *numeric* account identifier from the Stage 1 session payload. + Not `adsid`, which is a different identifier in the same dictionary. +- **password** — `mmeAuthToken`. Not `cloudKitToken`, despite this being CloudKit. + +Headers are the CloudKit set of §2.2. + +### 2.2 CloudKit headers + +Every request in this stage carries these, including `ckAppInit`: + +| Header | Value | +| --- | --- | +| `x-cloudkit-containerid` | `com.apple.icloud.searchparty` | +| `x-cloudkit-bundleid` | `com.apple.icloud.searchpartyd` | +| `x-cloudkit-databasescope` | the database scope — `PRIVATE` for this container | +| `x-cloudkit-environment` | `Production` | +| `x-cloudkit-duetpreclearedmode` | `None` | +| `x-apple-operation-group-id` | a random identifier, **uppercase hex**, per logical operation group | +| `x-apple-operation-id` | a random identifier, **uppercase hex**, per operation | +| `x-apple-request-uuid` | a v4 UUID, **uppercase**, per request | +| `x-apple-c2-metric-triggers` | `0` | +| `user-agent` | `CloudKit/1970 (19H384)` | +| `x-mme-client-info` | the client-info string of Stage 1 §2.2, with bundle `com.apple.cloudkit.CloudKitDaemon/1970 (com.apple.cloudd/1970)` | +| `accept` | `application/x-protobuf` | +| `accept-encoding` | `gzip` | +| `accept-language` | `en-US,en;q=0.9` | +| `cache-control` | `no-transform` | + +Plus **all** the Anisette headers, as in Stage 2. + +The three identifier headers are what CloudKit uses to correlate and deduplicate requests. +Generate them fresh per request rather than reusing a constant — a client that sends the same +operation id repeatedly is describing a retry of one operation, which is not what is meant. + +### 2.3 The response + +JSON. **[observed]** Eleven keys, and they matter more than expected — this response is where +the client learns *where* CloudKit lives: + +| Key | Meaning | +| --- | --- | +| `cloudKitUserId` | this account's CloudKit user identifier. **[observed]** 33 characters. | +| `cloudKitDatabaseUrl` | **the endpoint record operations go to** | +| `cloudKitDatabaseGatewayUrl` | the gateway-routed form of the same | +| `cloudKitShareUrl`, `cloudKitShareGatewayUrl` | sharing operations | +| `cloudKitDeviceUrl`, `cloudKitDeviceGatewayUrl` | device operations | +| `cloudKitCodeUrl`, `cloudKitCodeGatewayUrl` | server-side code | +| `cloudKitMetricsUrl` | telemetry, of no interest here | +| `values` | further server-supplied configuration; contents not yet examined | + +> **This is where the service endpoint comes from.** [Stage 2 §6](./02-mobileme-delegate.md) +> records that the MobileMe delegate returns no configuration and no URLs, which left an open +> question about where CloudKit's host was meant to be found. It is answered here: **`ckAppInit` +> hands it back.** Do not hardcode a CloudKit host — take `cloudKitDatabaseUrl` from this +> response, because iCloud partitions accounts across service hosts and a URL that works for one +> account is not guaranteed to work for another. + +**[observed] Each service appears twice, and the pair means something specific:** + +| Form | Points at | Example | +| --- | --- | --- | +| plain `…Url` | the account's **own partition**, directly | `p-ckdatabase.icloud.com` | +| `…GatewayUrl` | the shared front door, with a routing path segment | `gateway.icloud.com/…` | + +The `p` prefix is a partition shard — iCloud spreads accounts across numbered partitions, and +the observed account's was a two-digit value. That is what makes hardcoding a CloudKit host wrong: another account will +be on another partition, and the plain URL for one is meaningless for the other. + +`cloudKitMetricsUrl` is the exception, pointing at an unpartitioned `metrics.icloud.com`, which +fits — telemetry has no per-account state to be sharded by. + +Which form to prefer is **still unestablished**. Both are offered; the direct one is the more +specific and avoids a hop, while the gateway is what `ckAppInit` itself was addressed to. + +**Keep `cloudKitUserId`.** It is not the same as `adsid` or `DsPrsId`, and it is required to +address record zones in §3 — a private zone is identified by its name *and* its owner, and the +owner is this value. + +**HTTP 401 means the MobileMe token has expired**, not that the account is wrong. Refresh by +repeating Stage 2 — which needs a fresh PET, hence Stage 1 — and retry once. Bound the retry; +a 401 loop against a genuinely rejected account is otherwise indistinguishable from a slow +network. + +### 2.4 Why this is the milestone worth reaching first + +**[observed] All of this held.** Reaching a `cloudKitUserId` proved, in one request and with no +side effects: + +- the tokens from Stage 2 are the right ones and are accepted by a *different* Apple service +- the invented client identity passes CloudKit's checks, not just Grand Slam's +- the container exists and this account may open it +- and, unexpectedly, it yields the service endpoints as well + +And it does so without touching the keychain, without an escrow record, without the user's device +passcode, and without writing anything to the account. Everything after this point is more +expensive; nothing after this point is worth attempting if this fails. + +## 3. Fetching records + +### 3.1 The wire format is protobuf + +CloudKit operations are **Protocol Buffers over HTTP**, not JSON. Requests carry: + +``` +content-type: application/x-protobuf; desc="https://gateway.icloud.com:443/static/protobuf/CloudDB/CloudDBClient.desc"; messageType=RequestOperation; delimited=true +content-encoding: gzip +``` + +Three things follow. The message type is `RequestOperation`. Messages are **length-delimited**, +so a response is a stream of messages rather than one — multiple operations are batched into a +single request and answered in kind. And the `desc` parameter points at Apple's own descriptor +set, served from `gateway.icloud.com`, which is the authoritative schema. + +> **[observed] The descriptor URL is not fetchable.** It is tempting to read that `desc` +> parameter as a published schema that can simply be downloaded — it names a `.desc` file, which +> is what a compiled protobuf `FileDescriptorSet` is called. It returns **HTTP 400 with an empty +> body**, and so does a bare request to `gateway.icloud.com` itself, so the host rejects plain +> requests generally rather than that path being wrong. +> +> Treat the parameter as a *declaration* of which schema the payload conforms to — part of the +> content type, addressed to a client that already has the schema — and not as a way to obtain +> it. Whether it becomes retrievable once a request is properly authenticated is untested and +> worth one attempt from inside a working session, but nothing should be planned around it. + +So the schema is written down here instead. What follows covers only what reading accessory +records requires — the protocol is much larger, and the rest of it writes, deletes, shares and +subscribes, none of which this project does. + +### 3.2 The envelope + +Every call is a `RequestOperation`. Its two universal fields: + +| # | Field | Type | Meaning | +| --- | --- | --- | --- | +| 1 | `header` | `Header` | who is calling — see §3.3 | +| 2 | `request` | `Operation` | which operation this is | + +Then one further field carrying the operation's own request message. **The field number equals +the operation type**, which is the neatest thing in this protocol and worth relying on: + +| # | Field | Operation | +| --- | --- | --- | +| 200 | `zoneSaveRequest` | create a zone | +| **201** | **`zoneRetrieveRequest`** | **list zones** | +| 202 | `zoneDeleteRequest` | delete a zone | +| **203** | **`retrieveZoneChangesRequest`** | **which zones changed** | +| 210 | `recordSaveRequest` | write records | +| **211** | **`recordRetrieveRequest`** | **fetch records by identifier** | +| **213** | **`retrieveChangesRequest`** | **which records changed — the main fetch** | +| 214 | `recordDeleteRequest` | delete records | +| 220 | `queryRetrieveRequest` | query records | + +The `Operation` message declares the same thing again as an enum: + +| # | Field | Type | +| --- | --- | --- | +| 1 | `operationUUID` | string — a fresh uppercase v4 UUID per operation | +| 2 | **`type`** | the enum: `ZONE_RETRIEVE_TYPE = 201`, `RECORD_RETRIEVE_CHANGES_TYPE = 213`, … | +| 3 | `synchronousMode` | bool | +| 4 | `last` | bool — set on the final operation of a batch | + +> **`type` is field 2, not field 1.** Field 1 is a string. Putting the type there encodes a +> varint where the server expects a length-delimited string, and the result is **HTTP 500 with an +> empty body** — no protobuf error, nothing to decode, no hint as to the cause. **[observed.]** +> Any 500 with an empty body from this API should be read as "the request did not parse", and the +> field numbering is the first thing to check. + +So the operation is declared twice — once as this enum, once by which high-numbered field of +`RequestOperation` is populated — and the two must agree. + +Only the four in bold are needed to read accessory data. Everything else in this schema writes, +deletes, shares or subscribes, and this project does none of those. + +### 3.2.1 Where operations are sent + +**Use the `GatewayUrl`, not the plain one.** This settles the question §2.3 leaves open: record +operations are addressed to `cloudKitDatabaseGatewayUrl`, which is `gateway.icloud.com` plus a +`/ckdatabase` path segment. The plain `p-ckdatabase` host is *not* what a client talks to for +operations, despite being the more specific-looking of the two. + +Each operation family then has its own path beneath that. **Note that the "changes" operations +are called `sync`**, which is not obvious from their message names: + +| Operation | Full path | +| --- | --- | +| Zone retrieve | `/ckdatabase/api/client/zone/retrieve` | +| **Zone changes** | `/ckdatabase/api/client/zone/sync` | +| Record retrieve | `/ckdatabase/api/client/record/retrieve` | +| **Record changes** | `/ckdatabase/api/client/record/sync` | +| Record save | `/ckdatabase/api/client/record/save` | +| Record delete | `/ckdatabase/api/client/record/delete` | +| Query retrieve | `/ckdatabase/api/client/query/retrieve` | +| Zone save, zone delete | `/ckdatabase/api/client/zone/{save,delete}` | +| Asset token | `/ckdatabase/api/client/asset/retrieve/token` | +| Subscription create | `/ckdatabase/api/client/subscription/create` | + +**Three services, not one.** `ckdatabase` is only part of it, and the other two matter to other +stages: + +| Service | Base | Used for | +| --- | --- | --- | +| `ckdatabase` | `cloudKitDatabaseGatewayUrl` | records and zones — this stage | +| `ckcoderouter` | `cloudKitCodeGatewayUrl` | `/api/client/code/invoke` — **server-side functions, which is how [Stage 3](./03-keychain-trust.md) reaches Cuttlefish** | +| `ckshare` | `cloudKitShareGatewayUrl` | `/api/client/share/accept`, membership queries — relevant only if shared accessories are ever in scope | + +So a zone listing is `POST https://gateway.icloud.com/ckdatabase/api/client/zone/retrieve`. + +### 3.2.0 Authenticating an operation + +**Operations are authenticated by HTTP headers, not by the protobuf header.** The `userToken` +field exists in the request header message and is **left unset**; sending the CloudKit token +there does not authenticate the call. + +| Header | Value | +| --- | --- | +| `x-cloudkit-userid` | the `cloudKitUserId` from §2.3 | +| `x-cloudkit-authtoken` | the `cloudKitToken` from [Stage 2](./02-mobileme-delegate.md) | + +That answers what `cloudKitToken` is for: not the container-open call, which uses +`mmeAuthToken` over HTTP Basic, but every operation afterwards. Both tokens are needed, at +different points, and neither substitutes for the other. + +Two status codes have specific meanings here: **401** means the token expired — repeat Stage 2 — +and **429** means throttled. + +### 3.2.2 The response envelope + +Responses are `ResponseOperation`, mirroring the request: + +| # | Field | Meaning | +| --- | --- | --- | +| 1 | `operationCost` | what the operation cost against the account's budget | +| 2 | `response` | the `Operation`, echoing the type | +| 3 | `result` | **success or failure — check this first** | +| 4 | `bundled` | bundled sub-requests | +| 200–1101 | per-operation response | at the **same field number** as the request, so `zoneRetrieveResponse` is 201 | + +`Result` carries a `code` at field 1 and an `error` at field 2. The codes are +`SUCCESS = 1`, `PARTIAL = 2`, `FAILURE = 3`, `INDETERMINATE = 4` — note that **`PARTIAL` exists**, +so a batched request can half-succeed and an implementation that treats anything non-`SUCCESS` as +total failure will discard good results. + +`Error` is worth decoding rather than logging as bytes: + +| # | Field | Type | Meaning | +| --- | --- | --- | --- | +| 1 | `clientError` | **message** | a wrapper holding the client-fault code at its own field 1 | +| 2 | `serverError` | **message** | a wrapper holding the server-fault code at its own field 1 | +| 3 | `retryAfterSeconds` | int32 | back-off hint. **Honour it.** | +| 4 | `errorDescription` | string | human-readable | +| 5 | `errorKey` | string | a stable key for the error | +| 6 | `errorInternal` | string | Apple-internal detail | +| 7 | `extensionError` | message | `extensionName`, `typeCode`, `extensionPayload` | + +> **`clientError` and `serverError` are wrapper messages, not bare enums.** Each is a message +> whose field 1 holds the code. Declaring them as plain integers decodes as +> `invalid wire type: LengthDelimited (expected Varint)` — **[observed]** — which is at least a +> diagnostic that names the field, unlike the empty 500 that a malformed *request* produces. + +The client-fault codes are specific enough to act on, and several name situations this project +will hit: `BAD_SYNTAX = 4`, `FORBIDDEN = 5`, `THROTTLED = 6`, `BAD_AUTH_TOKEN = 11`, +`NEEDS_AUTHENTICATION = 12`, `NOT_SUPPORTED = 8`, `EXISTS = 9`. A `BAD_AUTH_TOKEN` means repeat +Stage 2; a `THROTTLED` with `retryAfterSeconds` means wait rather than retry. + +### 3.2.3 Zone retrieve + +The simplest operation, and the one that answers what zones the container holds. + +**Request** — `ZoneRetrieveRequest`, field 201: a single optional `zoneIdentifier` at field 1. +Omitting it asks for everything. + +**[observed] The container holds two zones**, and accessory data is in the first: + +| Zone | `protectionInfo` | `recordProtectionInfo` | `deviceCount` | continuation token | +| --- | --- | --- | --- | --- | +| **`BeaconStore`** | present | **absent** | 10 | present | +| `_defaultZone` | absent | absent | — | absent | + +**`BeaconStore` is the zone**, and `_defaultZone` is CloudKit's empty default. That closes the +open question about the zone name; §3.4's `RecordZoneIdentifier` is built from `BeaconStore` plus +the `cloudKitUserId` as owner. + +`deviceCount` counts the devices participating in the zone — on the observed account, ten, which +is more than its device list showed and is consistent with the zone outliving individual devices +in the same way escrow records do (see [Stage 3 §5](./03-keychain-trust.md)). + +**Response** — `ZoneRetrieveResponse`, field 201: repeated `zoneSummary` at field 1, each: + +| # | Field | Meaning | +| --- | --- | --- | +| 1 | `targetZone` | the `Zone` — its identifier, etag, and **both** protection infos | +| 2 | `currentServerContinuationToken` | the sync token to start from | +| 3 | `clientChangeToken` | the client's own change marker | +| 4 | `deviceCount` | devices participating in the zone | +| 5, 6 | `assetQuotaUsage`, `metadataQuotaUsage` | storage accounting | + +### 3.3 The header + +`RequestOperation.Header` is large and mostly optional. The fields that matter: + +| # | Field | Value for this project | +| --- | --- | --- | +| 1 | `userToken` | **leave unset** — authentication is by HTTP header, see §3.2.0 | +| 2 | `applicationContainer` | `com.apple.icloud.searchparty` | +| 3 | `applicationBundle` | `com.apple.icloud.searchpartyd` | +| 4 | `applicationVersion` | leave unset | +| 7 | `deviceIdentifier` | an `Identifier` for this device | +| 8, 9 | `deviceSoftwareVersion`, `deviceHardwareVersion` | consistent with the identity of Stage 1 §2.2 | +| 10, 11 | `deviceLibraryName`, `deviceLibraryVersion` | the CloudKit client library's own name and version | +| 16 | `deviceProtocolVersion` | leave unset | +| 18 | `mmcsProtocolVersion` | `5.0` | +| 19 | `applicationContainerEnvironment` | `PRODUCTION = 1` | +| 23 | `targetDatabase` | `PRIVATE_DB = 1` | +| 25 | `isolationLevel` | `ZONE = 1` or `OPERATION = 2` | +| 26 | `group` | operation group, matching the correlation headers of §2.2 | +| 10 | `deviceLibraryName` | `com.apple.cloudkit.CloudKitDaemon` | +| 11 | `deviceLibraryVersion` | `1970` | +| 29, 34 | undocumented | `0` | +| 35 | undocumented | `1` | + +Three fields have no meaningful names in any schema and are simply set to constants — `0`, `0` +and `1` at tags 29, 34 and 35. Send them; a server that expects them will not say so. + +**`deviceHardwareID` (22) is required, even for a read.** [observed] Omitting it is rejected with +`BAD_SYNTAX` and the message `deviceHardwareID is required field`. It is the 32-hex-character +device identifier of Stage 1 §2.2 — generated once, persisted, and stable, like everything else +about the identity. + +The other device-identifying fields — `deviceIdentifier` (7), `deviceAssignedName` (21) and +`deviceSerial` (33) — appear to be needed only by operations that *write*. **That is not +confirmed for reads other than zone retrieve**, and the server's validation makes it awkward to +confirm cheaply: it reports **one missing required field at a time**, so discovering the full set +means a request-per-field. Expect to iterate rather than to get a list. + +> **Error reporting here is genuinely good, once the schema is right.** A `FAILURE` result with +> `BAD_SYNTAX` carried `errorDescription` naming the exact field, and an `errorKey` — an +> eight-character stable identifier — that is worth logging for correlation. This is the same API +> that answers a malformed *request* with an empty 500, so the difference is stark: get the +> protobuf syntactically valid and the server becomes helpful. + +The enums are small and their numbering matters: `ContainerEnvironment` is `PRODUCTION = 1`, +`SANDBOX = 2`; `Database` is `PRIVATE_DB = 1`, `PUBLIC_DB = 2`, `SHARED_DB = 3`. Note none of +them use zero, so a defaulted field is not a valid value — an implementation must set these +explicitly rather than relying on protobuf's zero default. + +### 3.4 Identifiers + +Nothing in CloudKit is addressed by a bare string. The base type is: + +**`Identifier`** + +| # | Field | Type | +| --- | --- | --- | +| 1 | `name` | string | +| 2 | `type` | `Type` enum | + +with `RECORD = 1`, `DEVICE = 2`, `SUBSCRIPTION = 3`, `SHARE = 4`, `COMMENT = 5`, +`RECORD_ZONE = 6`, `USER = 7`. + +**`RecordZoneIdentifier`** — a zone is a name plus its owner: + +| # | Field | Value | +| --- | --- | --- | +| 1 | `value` | an `Identifier` naming the zone, type `RECORD_ZONE` | +| 2 | `ownerIdentifier` | an `Identifier` holding the **`cloudKitUserId` from §2.3**, type `USER` | +| 3 | `environment` | `PRODUCTION` | + +**`RecordIdentifier`** — a record is a name plus the zone it lives in: + +| # | Field | Value | +| --- | --- | --- | +| 1 | `value` | an `Identifier` naming the record, type `RECORD` | +| 2 | `zoneIdentifier` | the `RecordZoneIdentifier` above | + +This is why §2.3 insists on keeping `cloudKitUserId`: without it no private zone can be named. + +### 3.5 Fetching changes + +The main fetch is a **changes** operation rather than a listing, because CloudKit's model is +incremental sync. Two levels exist, and both work the same way. + +**Which zones changed** — `RetrieveZoneChangesRequest`, field 203: + +| # | Field | Meaning | +| --- | --- | --- | +| 1 | `syncContinuationToken` | omit on a first run; everything is returned | +| 2 | `maxChangedZones` | page size | + +The response returns repeated `ChangedZone` entries — each with the zone `identifier`, a +`changeType` and a `deleteType` — plus a new `syncContinuationToken` and a `status`. + +**Which records changed** — `RetrieveChangesRequest`, field 213, sent to +`/ckdatabase/api/client/record/sync`: + +| # | Field | Meaning | +| --- | --- | --- | +| 1 | `syncContinuationToken` | omit on a first run | +| 2 | `zoneIdentifier` | the zone — `BeaconStore` plus the owner | +| 3 | `requestedFields` | which fields to return; omit for all | +| 4 | `maxChanges` | page size | +| 5 | `requestedChangesTypes` | which kinds of change | +| 7 | `newestFirst` | ordering | +| 8 | `ignoreCallingDeviceChanges` | skip changes this client made | +| 9 | `includeMergeableDeltas` | include deltas rather than whole records | + +Its response holds repeated `RecordChange`: + +| # | Field | Meaning | +| --- | --- | --- | +| 1 | `identifier` | the `RecordIdentifier` | +| 2 | `etag` | version tag | +| 3 | `recordType` | the record's type name | +| 4 | `type` | the kind of change | +| 5 | `record` | the record itself, when the change carries one | + +plus a `syncContinuationToken` for the next call. + +**Persist that token.** It is both the way to avoid refetching everything and the mechanism by +which this feature would notice a newly-paired accessory on a later run — which, per the +[README](./README.md), is why the account is re-read at all. + +### 3.5.1 What the zone actually contains [observed] + +A first `record/sync` against `BeaconStore` returned **35 records across nine types**, with a +continuation token for the next page: + +| Record type | Count | What it is | +| --- | --- | --- | +| `MasterBeaconRecord` | 6 | **the owned accessories, with their key material** | +| `BeaconNamingRecord` | 5 | user-assigned names, one per accessory | +| `KeyAlignmentRecord` | 4 | last observed key index — see OpenTagViewer's rule 6 | +| `OwnedDeviceKeyRecord` | 8 | key pairs belonging to the user's *devices*, not accessories | +| `SharingCircleSecret` | 5 | secrets for accessories shared with others | +| `OwnerSharingCircle` | 1 | a sharing circle this account owns | +| `OwnerPeerTrust` | 1 | peer trust state | +| `SafeLocation` | 4 | named geofences — "notify when left behind" | +| `LeashRecord` | 1 | accessory grouping | + +Three of these are worth calling out. + +**Six accessories, five name records, four alignment records.** These are not one-to-one. An +accessory may have no name record and no alignment record, so an implementation must join on +`associatedBeacon`/`beaconIdentifier` and tolerate absence rather than zipping lists. + +**`SafeLocation` holds the user's home and work coordinates** — `latitude`, `longitude`, +`radius`, `name`. This project has no use for it. It is in the same zone and will be fetched +whether or not it is wanted, so an implementation should discard it explicitly rather than +letting it flow into storage or logs by default. + +**The sharing machinery is in this zone**, not in a separate shared database. See the +[README](./README.md) on shared accessories — this makes them materially cheaper to reach than a +separate-database design would. + +### 3.6 Records and their fields + +**`Record`** — the fields worth knowing: + +| # | Field | Meaning | +| --- | --- | --- | +| 1 | `etag` | version tag | +| 2 | `recordIdentifier` | which record this is | +| 3 | `type` | the record type, a message wrapping a `name` string | +| 4 | `createdBy` | an `Identifier` | +| 5 | `timeStatistics` | creation and modification times | +| 7 | `recordField` | **repeated `Field`** — the actual contents | +| 9 | `modifiedBy` | an `Identifier` | +| **13** | **`protectionInfo`** | **which PCS key protects this record — the input to Stage 5** | +| 15 | `permission` | access level | +| 24 | `pcsKey` | key material associated with the record | + +**`Record.Field`** is a name and a value: + +| # | Field | Meaning | +| --- | --- | --- | +| 1 | `identifier` | a message wrapping the field's `name` string | +| 2 | `value` | the value | + +**`Record.Field.Value`** is a tagged union — a `type` enum plus whichever typed field is +populated: + +| # | Field | For type | +| --- | --- | --- | +| 1 | `type` | the discriminator | +| 2 | `bytesValue` | `BYTES_TYPE`, and the encrypted types | +| 4 | `signedValue` | `INT64_TYPE` | +| 5 | `doubleValue` | `DOUBLE_TYPE` | +| 6 | `dateValue` | `DATE_TYPE` | +| 7 | `stringValue` | `STRING_TYPE` | +| 8 | `locationValue` | `LOCATION_TYPE` | +| 9 | `referenceValue` | `REFERENCE_TYPE` | +| 10 | `assetValue` | `ASSET_TYPE` | +| 11 | `listValues` | the list types — repeated `Value` | +| 13 | `isEncrypted` | flags an encrypted field | + +The type enum runs `BYTES_TYPE = 1` through `UNKNOWN = 22`. + +> **[observed] `type` describes the *plaintext*, not the ciphertext.** Every field of every +> record in `BeaconStore` came back with `isEncrypted = true`, but their `type` values were +> spread across `STRING_TYPE = 3`, `INT64_TYPE = 7`, `DATE_TYPE = 2`, `DOUBLE_TYPE = 8`, +> `STRING_LIST_TYPE = 15` and `ENCRYPTED_BYTES_TYPE = 20`. +> +> So the type is a promise about what decryption will yield, not a description of what is on the +> wire. `ENCRYPTED_BYTES_TYPE` is what a field whose plaintext is *binary* uses; a field whose +> plaintext is a string is `STRING_TYPE` with `isEncrypted` set. **Branch on `isEncrypted`, and +> use `type` only to decide how to interpret the result.** An implementation that switches on +> `type` alone will try to read ciphertext as a string. + +That also explains the separate `EncryptedValue` message, which holds a signed value, a date or a +string: it is the shape of a field *after* Stage 5 has done its work. + +### 3.7 Protection info + +**`ProtectionInfo`** is small and appears on records, zones and participants alike: + +| # | Field | Type | +| --- | --- | --- | +| 1 | `protectionInfo` | bytes — the encoded protection structure | +| 2 | `protectionInfoTag` | string — a tag identifying it | + +A `Zone` has room for **two**: `protectionInfo` at field 3 and `recordProtectionInfo` at field 6. + +> **[observed] Protection is per-record, and it is universal.** On the `BeaconStore` zone, +> `protectionInfo` was present and `recordProtectionInfo` absent. On the records themselves, +> **every one of the 35 carried its own `protectionInfo` at field 13, and none carried a +> `pcsKey` at field 24.** +> +> So the hierarchy is: the zone's `protectionInfo` gives the zone keys, and each record's own +> `protectionInfo` describes how that record is protected under them. `recordProtectionInfo` on +> the zone and `pcsKey` on the record are both optional and were both absent — an implementation +> must not require either. + +This is the boundary of Stage 4. Everything above yields records whose interesting fields are +ciphertext plus a `ProtectionInfo` describing how to unlock them; turning that into plaintext is +Stage 5 and needs the keychain from Stage 3. + +## 4. Open questions + +**Answered by the run of 2026-08-13:** + +1. ~~Does `ckAppInit` succeed with only Stage 2's tokens?~~ **Yes.** The whole argument for doing + this before Stage 3 rested on it, and it holds: CloudKit access needs no keychain state. +2. ~~Where does the CloudKit endpoint come from?~~ **From `ckAppInit` itself**, as + `cloudKitDatabaseUrl`. Not from the MobileMe delegate, and not hardcoded. +3. ~~Can Apple's protobuf descriptor set be fetched?~~ **No** — see §3.1. + +**Still open:** + +4. ~~What is the zone name holding accessory records?~~ **[observed] `BeaconStore`.** +5. **Plain URL or `GatewayUrl`?** ~~The distinction is unexplained~~ — it is now known to be + direct-to-partition versus routed-through-the-gateway (§2.3). **Which to prefer is still + open**, and it may not matter; trying the direct form first is the obvious default. +6. ~~Can records be listed without any keychain state?~~ **[observed] Yes.** All 35 records were + fetched with no keychain, no trust circle and no device passcode. Fetching and decrypting are + cleanly separable, and Stage 3 is required only to *read* what has already been retrieved. +7. **Is `cloudKitToken` used at all**, given `ckAppInit` authenticates with `mmeAuthToken`? + Presumably it authenticates record operations rather than the setup call, but which requests + take which credential is not established. +8. ~~What is in `values`?~~ **[observed]** An array of two objects, one per environment + (`PRODUCTION` and `SANDBOX`), each carrying the container `name`, its `env`, a database `url` + and a `ckDeviceUrl`. It is a per-environment endpoint table for the container being opened — + useful for confirming the partition, and **not** a general service directory. Notably it does + not carry the escrow host that [Stage 3](./03-keychain-trust.md) needs. diff --git a/docs/findmy-export/05-pcs-decryption.md b/docs/findmy-export/05-pcs-decryption.md new file mode 100644 index 00000000..fdbfd231 --- /dev/null +++ b/docs/findmy-export/05-pcs-decryption.md @@ -0,0 +1,311 @@ +# Stage 5 — PCS decryption + +Specification of how an encrypted CloudKit record field becomes plaintext, using keys held in the +iCloud Keychain. + +Read [README.md](./README.md) first. In particular: **implement from this document alone.** + +> **Unverified.** Nothing here has been run. It is derived by reading implementations whose +> authors report them working, and it is the least-exercised stage in this set. +> +> The ASN.1 structures of §3 are enumerated field by field, and the wire format of §6 is exact. +> What remains unspecified is called out in §8. + +--- + +## 1. What this stage is for + +[Stage 4](./04-cloudkit.md) ends with records in hand whose every field is ciphertext. This stage +turns them into plaintext. It is the last thing between the data and being useful, and it is +where the keys from [Stage 3](./03-keychain-trust.md) are finally spent. + +PCS is **Protected Cloud Storage**, Apple's layer for encrypting CloudKit records under keys the +server never sees. + +## 2. What it needs from the keychain + +The Find My service is identified to PCS by a fixed descriptor: + +| Property | Value | +| --- | --- | +| Service name | `com.apple.icloud.searchparty` | +| Keychain view | **`Manatee`** | +| Service type | `82` | +| Version | v2 | + +**`Manatee` is the keychain view holding these keys**, and it is the concrete answer to what +Stage 3 must deliver. Two views must be synced before decryption can begin: + +``` +Manatee +ProtectedCloudStorage +``` + +Within the `Manatee` view, the service key is the item labelled: + +``` +com.apple.ProtectedCloudStorage-com.apple.icloud.searchparty +``` + +**Keychain items are dictionaries**, and the attribute that matters for key lookup is `acct`, +which holds the **base64 of a compressed elliptic-curve public key**. That is how a protection +structure's key references are resolved against the keychain: match on `acct`. + +## 3. The protection structure + +Every record carries a `protectionInfo` (Stage 4 §3.7), whose `protectionInfo` field is a byte +string. **It is DER-encoded ASN.1**, not protobuf — the one place in this whole protocol where +the encoding changes, and an easy thing to lose an afternoon to. + +### 3.1 The structures + +Written as ASN.1. Tags are **explicit** throughout; anything untagged is in sequence order. + +**`ShareProtection`** — the top level, and what a record's `protectionInfo` bytes decode to: + +```asn1 +ShareProtection ::= [APPLICATION 1] EXPLICIT SEQUENCE { + keyset KeySet, + meta [0] EXPLICIT OCTET STRING, -- encrypted; not needed to decrypt fields + signatureData [1] EXPLICIT SignatureData, + hmac OCTET STRING, + truncatedKeyId[2] EXPLICIT OCTET STRING, -- first 4 bytes of the key id + signature [3] EXPLICIT Signature OPTIONAL, + attributes [4] EXPLICIT SEQUENCE OF Attribute OPTIONAL +} +``` + +Note `hmac` sits **between** two context-tagged fields without a tag of its own — a decoder that +assumes tags ascend monotonically will misparse it. + +**`KeySet`** — the entries, one per party that can unwrap this record: + +```asn1 +KeySet ::= SEQUENCE { + unknown INTEGER, -- observed as 0 + keyset SET OF ShareKey, -- a SET, so unordered + attributes [0] EXPLICIT SEQUENCE OF Attribute OPTIONAL +} + +ShareKey ::= SEQUENCE { + decryptionKey KeyRef, + ciphertext OCTET STRING, -- the RFC 6637 wrapped key + flags INTEGER OPTIONAL -- bit 0 set = read-only +} + +KeyRef ::= SEQUENCE { + keytype INTEGER, + pubKey OCTET STRING -- compressed EC public key +} +``` + +`keyset` is a **SET OF**, not a SEQUENCE OF — the entries are unordered and must be searched by +matching `pubKey`, never indexed. + +**`SignatureData`** — carries the version that §4 step 3 branches on: + +```asn1 +SignatureData ::= SEQUENCE { + version INTEGER, + data OCTET STRING +} +``` + +> The version numbering is **not clearly established**. The reference implementation's own note +> on it is self-contradictory, mapping the same values to different names in one breath. What is +> established is only what §4 step 3 needs: **the share-key derivation is skipped when the +> version is 5**. Treat any other interpretation of this field as unknown. + +**`Signature`** and **`Attribute`** — small and reused: + +```asn1 +Signature ::= SEQUENCE { + keyid OCTET STRING, + digest INTEGER, -- 1 = SHA-256, 2 = SHA-512 + signature OCTET STRING +} + +Attribute ::= SEQUENCE { + key INTEGER, + value OCTET STRING -- itself DER, per key +} +``` + +Known attribute keys, each whose `value` is a further DER structure: + +| Key | Contents | +| --- | --- | +| 1 | `SEQUENCE { build [0] UTF8String, time [1] GeneralizedTime }` | +| 3 | `SEQUENCE { flags INTEGER }` — the Manatee flags | +| 7 | a `Signature` — the owner signature checked in §4 step 4 | + +### 3.2 Keys as they appear in the keychain + +The keychain items of §2 hold these: + +```asn1 +PublicKey ::= [APPLICATION 1] EXPLICIT SEQUENCE { + pcsService INTEGER, -- 82 for Find My + unknown INTEGER, + pubKey OCTET STRING, + attributes [0] EXPLICIT SEQUENCE OF Attribute OPTIONAL, + signature [1] EXPLICIT Signature OPTIONAL +} + +PrivateKey ::= CHOICE { + v1 SEQUENCE { key OCTET STRING, public PublicKey OPTIONAL }, + v2 [APPLICATION 5] SEQUENCE { data OCTET STRING } +} +``` + +> **`[APPLICATION 1]` is used by both `ShareProtection` and `PublicKey`.** They are told apart by +> where they appear, not by their tag, so a decoder must be told which one to expect rather than +> dispatching on the tag alone. + +The Find My service is **v2** (§2), so its keys are the `[APPLICATION 5]` form. + +There is also a keyset wrapper used when keys are carried together: + +```asn1 +ShareProtectionKeySet ::= [APPLICATION 2] EXPLICIT SEQUENCE { + unknown UTF8String, + keys SET OF PrivateKey, + unknown2 SET OF ANY, + hash OCTET STRING OPTIONAL -- SHA-256 over this structure's DER, +} -- computed with `hash` itself absent +``` + +## 4. Unwrapping, step by step + +**Step 1 — find the entry that belongs to us.** Compress our private key's public part and scan +the keyset for an entry whose public key matches those bytes. That entry's ciphertext is the +wrapped key intended for us. If none matches, we do not hold a key for this record and no amount +of retrying will change that — report it as a missing key, not as a decryption failure. + +**Step 2 — unwrap it.** The wrapping is **RFC 6637** — the ECDH key-wrap construction from +OpenPGP — with the fixed parameter string `fingerprint`. The result is the PCS **master key**, +16 bytes. + +**Step 3 — derive the share key, conditionally.** + +``` +if signature_data.version != 5 and not read_only: + master_key = KDF(master_key, "MsaeEooevaX fooo 012") +``` + +Otherwise the master key is used directly. **This branch is easy to miss and produces a key that +fails every subsequent check**, so verify against the checksum in step 5 rather than pressing on. + +**Step 4 — verify the signatures.** The structure is signed, and unless the read-only flag is set +the signature is verified against a key derived from the master key itself (§5). A second, +"past" signature may be present and should be tried if the first fails — that is key rotation, +not corruption. An owner signature may additionally be present as attribute `7`. + +**Step 5 — check you have the right key.** Two independent checks, and both should be performed +before anything is decrypted: + +- the first 4 bytes of the derived key's **key id** must equal the structure's truncated key id +- the structure's **HMAC** must verify under the derived key + +A mismatch means the wrong key, not corrupt data. + +**Step 6 — decrypt the fields.** + +## 5. Key derivation + +Every derived key comes from the master key by the same construction: a **counter-mode KDF with +HMAC** (the NIST SP 800-108 shape), with a fixed label and an output the same length as the input +key. + +| Derived key | Label | +| --- | --- | +| Share key | `MsaeEooevaX fooo 012` | +| HMAC key | `hmackey-of-masterkey` | +| Key-id label key | `master key id labell` | +| Encryption key | `encryption key key m` | + +**Every label is exactly 20 characters.** That is not a coincidence and not something to +normalise away — treat the labels as opaque fixed byte strings and copy them exactly, spaces +included. + +**The key id** is a two-stage construction rather than a plain digest: + +``` +label_key = KDF(master_key, "master key id labell") +key_id = HMAC-SHA256(label_key, "M key input data 2 u") +``` + +**The master EC key** — used to verify the structure's own signature — is derived from the master +key by a route that is unusual enough to be worth spelling out: + +``` +out = PBKDF2-HMAC-SHA256(master_key, salt = "full master key", iterations = 10, length = 128) +out = reverse(out) # the output is little-endian; big-endian is needed +n = mask_to_bit_length(out, order_bits(P-256)) +if n > order: n = n - order +private_scalar = n +``` + +Three traps in five lines: **ten** iterations, not a realistic PBKDF2 count; the output is +**reversed** because it is produced little-endian and consumed big-endian; and the reduction is a +single conditional subtraction rather than a modulo. The curve is **P-256** +(`prime256v1`/`secp256r1`). + +## 6. Field decryption + +Encrypted field values are **AES-128-GCM**. + +The ciphertext is not bare. It carries a **variable-length header** that must be parsed, checked, +and then used as authenticated data: + +| Offset | Size | Meaning | +| --- | --- | --- | +| 0 | 1 | **encryption version. Must be 3** — anything else is a format this document does not describe, and should be refused rather than guessed at. | +| 1 | 2 | first part of the key id | +| 3 | 1 | *N*, the length of the second part | +| 4 | *N* | second part of the key id | + +so the header is `4 + N` bytes, and the key id to compare against is the two byte ranges +**concatenated** — bytes 1–2 followed by bytes 4 to 4+*N* — skipping the length byte at offset 3. +That reconstructed value must equal the leading bytes of the derived key's key id (§5). It is the +third and cheapest opportunity to detect the wrong key, and it costs nothing. + +After the header: + +| Part | Size | +| --- | --- | +| IV | 12 bytes | +| Tag | 12 bytes — **not the usual 16** | +| Ciphertext | the remainder | + +**The AAD is the entire header** — bytes 0 to `4 + N`, including the version byte and the length +byte, not merely the key id. + +The key is `KDF(master_key, "encryption key key m")`, 16 bytes for AES-128. + +> Note the tag length: **12 bytes, not GCM's default 16.** A library configured with the default +> will reject every message, and the failure looks like corruption rather than misconfiguration. + +## 7. Interpreting the plaintext + +Stage 4 §3.6 establishes that a field's declared `type` describes its **plaintext**, so once +decrypted the bytes are interpreted according to it — a `STRING_TYPE` field yields a string, an +`INT64_TYPE` field an integer, and so on. The protocol also defines a small `EncryptedValue` +message holding a signed value, a date or a string, which is the shape a decrypted field takes. + +Which of those two forms applies to which field type is **not established** — see §8. + +## 8. Open questions + +1. **Is a decrypted field raw bytes, or the `EncryptedValue` message?** §7 cannot say, and the + answer decides whether decryption yields a value or another parse step. +2. **What are the `SharingCircleSecret` records for?** [Stage 4 §3.5.1](./04-cloudkit.md) + observes five of them in the zone, each with a `secretType`, a `sharingCircleIdentifier` and a + `secretData` blob. The plausible reading is that they protect accessories shared with others, + as a parallel key hierarchy to this one — but nothing here establishes it. +3. **Does the zone's `protectionInfo` play a role, given every record carries its own?** +4. **What is the `meta` field, and the two `unknown` integers?** All three are carried and none + is needed to decrypt a field, so they are recorded rather than explained. Stage 4 + observed both present at the zone and universal at the record. Whether a record's structure is + unwrapped under a zone key or directly under the keychain service key is not established, and + it changes what §4 step 1 is matching against. diff --git a/docs/findmy-export/06-output.md b/docs/findmy-export/06-output.md new file mode 100644 index 00000000..e2ef009b --- /dev/null +++ b/docs/findmy-export/06-output.md @@ -0,0 +1,151 @@ +# Stage 6 — Output + +Specification of how decrypted accessory records become something FindMy.py can use. + +Read [README.md](./README.md) first. + +> **The mapping is confirmed.** A real `MasterBeaconRecord` fetched from CloudKit was compared +> field by field against the plists OpenTagViewer already imports, and they are the same data +> under nearly the same names. Values marked **[observed]** come from that comparison. +> +> This is the shortest stage in the set, and deliberately so: if it turns out to be long, +> something has gone wrong upstream. + +--- + +## 1. Why this stage is nearly nothing + +The macOS export route reads a cache directory. **[observed] That cache is a local copy of the +CloudKit records** — the plists even carry a `cloudKitMetadata` field holding CloudKit's own +system fields, which is the giveaway. + +So there is no format to design. The records fetched in [Stage 4](./04-cloudkit.md) and decrypted +in [Stage 5](./05-pcs-decryption.md) *are* the export, modulo naming. + +## 2. The accessory mapping + +`MasterBeaconRecord` against the `OwnedBeacons/.plist` layout: + +| CloudKit field | Plist key | Note | +| --- | --- | --- | +| `privateKey` | `privateKey` | **wrapped** — see §2.1 | +| `publicKey` | `publicKey` | wrapped | +| `sharedSecret` | `sharedSecret` | wrapped | +| `sharedSecret2` | `secondarySharedSecret` | **the one rename**, and also wrapped | +| `productId` | `productId` | | +| `vendorId` | `vendorId` | | +| `model` | `model` | | +| `isZeus` | `isZeus` | integer in CloudKit, **boolean** in the plist | +| `systemVersion` | `systemVersion` | | +| `pairingDate` | `pairingDate` | | +| `batteryLevel` | `batteryLevel` | | +| `stableIdentifier` | `stableIdentifier` | string in CloudKit, **a list** in the plist | +| — | `identifier` | the record's own identifier, from `recordIdentifier` | +| — | `cloudKitMetadata` | CloudKit system fields; see §2.2 | +| `secureLocationsSharedSecret` | — | **[observed] absent** on the account examined | +| `groupIdentifier` | — | no home in this format; see §4 | + +### 2.1 Key material is wrapped + +Every key and secret in the plist is nested two levels deep rather than stored as bare bytes: + +``` +privateKey → { "key": { "data": } } +``` + +CloudKit returns the bytes directly, so writing the plist means wrapping them and reading it +means unwrapping. It carries no information — it is a serialisation artefact of the macOS +framework that produced the cache — but omitting it produces a file that fails to import for a +reason nothing will explain. + +### 2.2 `cloudKitMetadata` + +The plists carry CloudKit's own record metadata as an opaque blob. **Whether anything reads it is +unestablished**; the fixtures in OpenTagViewer's test resources contain a 3-byte stub, which +suggests it is at least tolerated as a placeholder. Prefer emitting a placeholder over omitting +the key, and do not attempt to synthesise real CloudKit metadata. + +## 3. The naming mapping + +`BeaconNamingRecord` maps **exactly** — same field names, same camelCase: + +| CloudKit field | Plist key | +| --- | --- | +| `name` | `name` | +| `associatedBeacon` | `associatedBeacon` | +| `roleId` | `roleId` | +| `emoji` | `emoji` — **[observed] absent** on the account examined | + +Plus `identifier` and `cloudKitMetadata` as above. + +The directory layout OpenTagViewer imports mirrors the association: + +``` +OwnedBeacons/.plist +BeaconNamingRecord//.plist +``` + +> **[observed] These are not one-to-one.** Six accessories, five naming records and four key +> alignment records were returned for one account. Join on `associatedBeacon` and tolerate +> absence — an accessory with no naming record is normal, not an error, and must still be +> exported. + +## 4. Prefer FindMy.py's own format + +The plist layout above exists because it is what a Mac happened to write. It predates fields the +protocol now carries, and `groupIdentifier` is the clearest case: it identifies accessories that +belong to a set, such as the two halves of a pair of AirPods, and there is **nowhere to put it**. + +FindMy.py gained support for exactly that in 0.10.0, along with serial-number extraction and +group naming. Its native JSON has somewhere for these to live. + +**So emit FindMy.py's format as the primary output**, and treat the plist layout as a +compatibility path for existing tooling rather than the target. A generator that writes only the +plist form is discarding data it successfully decrypted. + +## 5. What not to export + +[Stage 4 §3.5.1](./04-cloudkit.md) records that the zone contains more than accessories. Of the +nine record types returned, most have no place in an export: + +| Type | Export? | +| --- | --- | +| `MasterBeaconRecord`, `BeaconNamingRecord` | **yes** — this is the payload | +| `KeyAlignmentRecord` | **yes** — see §6 | +| `SafeLocation` | **no.** Holds the user's home and work coordinates. | +| `OwnedDeviceKeyRecord` | no — the user's devices' keys, not accessories' | +| `OwnerPeerTrust`, `OwnerSharingCircle`, `SharingCircleSecret` | no — trust and sharing state | +| `LeashRecord` | no | + +**`SafeLocation` deserves an explicit decision rather than a default.** It arrives whether wanted +or not, it decrypts with the same keys as everything else, and it contains named geofences around +the user's home. Discard it at the point of decryption. Do not log it, do not persist it, and do +not let it reach an export file because nobody wrote a filter. + +## 6. Key alignment + +`KeyAlignmentRecord` carries `beaconIdentifier`, `lastIndexObserved` and +`lastIndexObservationDate`. + +This is not incidental. OpenTagViewer's rule 6 explains why: without an alignment record, a +freshly imported accessory starts its key search at index 0 from its pairing date, which for an +eighteen-month-old tag means deriving on the order of fifty thousand keys and issuing hundreds of +requests against Apple. That is an account-flagging risk, not merely slow. + +**So export the alignment record whenever one exists.** The format OpenTagViewer imports has +carried it since export format `0.0.2`, and the importer falls back to a probe when it is absent +— but the fallback is the expensive path, and this route can avoid it. + +**[observed] Not every accessory has one** — four records for six accessories. Absence is normal +and must not be treated as an error. + +## 7. Open questions + +1. **Does anything read `cloudKitMetadata`?** §2.2. If nothing does, the key can hold a + placeholder indefinitely; if something does, this stage is not as thin as it looks. +2. **What is `stableIdentifier` a list of** in the plist, given CloudKit returns a single string? + A one-element list is the obvious guess and is not confirmed. +3. **What does `isZeus` mean?** It is carried faithfully by both formats and understood by + neither. +4. **Is `secureLocationsSharedSecret` needed for anything?** It was absent on the account + examined, and no consumer of it has been identified. diff --git a/docs/findmy-export/README.md b/docs/findmy-export/README.md new file mode 100644 index 00000000..0d81f088 --- /dev/null +++ b/docs/findmy-export/README.md @@ -0,0 +1,287 @@ +# Exporting FindMy accessory keys without a Mac — protocol specification + +This directory specifies the network protocol an application must speak to Apple in order to +read a user's FindMy accessory (AirTag) key material directly from iCloud, with no Mac and no +macOS export step. + +It exists so that the flow can be implemented under a permissive licence, without deriving from +any existing implementation. + +**The implementation target is [FindMy.py](https://github.com/malmeloo/FindMy.py), in Python, not +Java.** OpenTagViewer already embeds FindMy.py through Chaquopy, so Python written there *is* app +code — and FindMy.py already performs Stages 1 and 2 and persists their output, meaning the +hardest prerequisites are met before this feature starts. Building it there rather than in the +app's Java means the work lands where the session already lives, and everyone else using the +library gets it too. + +--- + +## Why a specification and not a port + +The one working implementation of this flow is +[OpenBubbles/rustpush](https://github.com/OpenBubbles/rustpush), together with +[stek29/export-findmy](https://github.com/stek29/export-findmy), a headless command-line +exporter built on a fork of it. + +Their licensing rules out reuse: + +| Component | Licence | Consequence | +| --- | --- | --- | +| `rustpush` — CloudKit, keychain, escrow, FindMy record handling | **SSPL v1**, with a written exception granting unrestricted rights to OpenBubbles alone | Copyleft. Deriving from it would relicense OpenTagViewer. | +| `apple-private-apis` — GSA authentication, Anisette (`icloud-auth`, `omnisette`) | **MPL 2.0** | File-level copyleft. A direct port would have to stay MPL, putting a second licence inside an MIT app. | +| `apple-private-apis/icloud-auth/rustcrypto-srp` — forked SRP | MIT / Apache-2.0 | Permissive, but describing it costs nothing. | +| `stek29/export-findmy` — the CLI that glues the above together | **No licence file** — all rights reserved by default | Not usable as source at all. | + +What those licences cover is *expression*: code layout, identifiers, comments, structure. They +do not and cannot cover **Apple's wire protocol** — the endpoints, header names, payload +layouts, cryptographic constructions and state transitions written down here. Those are facts +about a third party's service, and facts are not copyrightable. + +## The clean-room split + +These documents are the "specification" half of a clean-room reimplementation. + +- **The specification is written by someone who has read the reference implementations.** That + is deliberate: it is the only way to learn the protocol without a packet capture of a real + Apple device. +- **The implementation must be written by someone who has not.** The implementer works from + these documents alone. The language is Python, targeting FindMy.py; the discipline would be + identical in any language, because what must not cross the boundary is the *expression* of the + reference implementations, not any particular syntax. + +So, for anyone implementing: + +> **Do not open `rustpush`, `apple-private-apis`, or `export-findmy` while writing this code.** +> If something here is ambiguous, incomplete or wrong, fix *this document* — raise the gap, get +> the specification amended, and implement from the amendment. Reading the reference to resolve +> a question destroys the separation that makes the reimplementation defensible. +> +> This matters more, not less, for an upstream contribution: FindMy.py is MIT, and a patch +> carrying SSPL-derived expression into it would be a problem for its maintainer as well as for +> this project. + +Nothing below quotes, paraphrases at the level of expression, or names any type, function or +file from those projects. + +--- + +## The stages + +The full flow from an Apple ID to accessory private keys is six stages. They are specified +separately because they are separately testable and separately hard. **All six are now written.** + +| # | Stage | Document | Status | +| --- | --- | --- | --- | +| 1 | **Apple ID authentication** — Anisette headers, SRP handshake against GSA, two-factor, session tokens | [01-authentication.md](./01-authentication.md) | **Specified and verified** | +| 2 | **MobileMe delegate** — exchange the short-lived PET for iCloud service tokens | [02-mobileme-delegate.md](./02-mobileme-delegate.md) | **Specified and verified** | +| 3 | **iCloud Keychain trust circle** — join via escrow recovery, using a device passcode | [03-keychain-trust.md](./03-keychain-trust.md) | Listing **verified**; recovery and join unverified | +| 4 | **CloudKit** — open the container, fetch encrypted `BeaconStore` records | [04-cloudkit.md](./04-cloudkit.md) | §2 **verified**; §3 specified, unverified | +| 5 | **PCS decryption** — decrypt those records with keychain-held keys | [05-pcs-decryption.md](./05-pcs-decryption.md) | Specified, unverified | +| 6 | **Output** — write the layout OpenTagViewer already imports | [06-output.md](./06-output.md) | Mapping **confirmed**; unexercised | + +Stage 1 is specified first because it is self-contained, because it is the stage that the app's +existing native Anisette work already feeds, and because finishing one stage is the only honest +way to estimate the other five. + +## Shared accessories: reachable, and new + +The macOS export route this replaces reads a directory named `OwnedBeacons`, and that is exactly +what it gets — accessories the user owns. An AirTag shared *with* the user, via Apple's sharing +feature, does not appear. + +**The iCloud route can reach them**, and that would be capability the current app does not have. +The container holds a distinct record type for them, alongside the owned one. But the two are not +symmetrical, and the asymmetry is the whole story: + +| Owned accessory record | Shared accessory record | +| --- | --- | +| private key, public key | — | +| shared secret, secondary shared secret | — | +| secure-locations shared secret | — | +| pairing date, stable identifier | share date, correlation identifier | +| — | owner handle, accepted flag, role, share type | + +**A shared record carries no key material.** It describes the share — who owns the accessory, +whether the user accepted it, what role they hold — and nothing that would let anyone locate +anything. The keys live behind a separate structure: the container also holds owner and member +*sharing circle* records and a *sharing circle secret*, and the plausible reading is that the +circle secret is what unlocks a shared accessory's keys. + +So supporting shared accessories means **a second key hierarchy on top of the PCS one**, not a +handful of extra fields. It is a deliberate scope decision rather than something that falls out +of the work, and it is not part of Stages 1 to 6 as specified. + +**[observed] The sharing machinery is in the same zone as everything else.** A fetch of +`BeaconStore` returned `SharingCircleSecret`, `OwnerSharingCircle` and `OwnerPeerTrust` records +alongside the owned accessories — not in CloudKit's separate shared database, and reachable with +no extra credential. `SharingCircleSecret` carries a `secretType`, a `sharingCircleIdentifier` +and a `secretData` blob, which is very likely the key material a shared accessory's records are +protected under. + +That makes shared accessories **substantially cheaper than a separate-database design would**: +the records are already in hand after the ordinary fetch, and the remaining work is understanding +how a circle secret unwraps a shared accessory's keys. + +One question is still open, and it is the one that decides the cost: **whether any of it requires +Apple's identity service.** Some sharing machinery in the reference reaches into per-participant +messaging targets, which would be a far larger dependency. The plausible reading is that identity +is needed to *establish* or *accept* a share and not to read one already accepted — but that is +an inference from naming, not something established. + +## Stage 6 is a rename, not a translation + +The macOS export route this replaces reads a cache directory. **[observed] That cache is a local +copy of the CloudKit records** — the plists even carry a `cloudKitMetadata` field holding +CloudKit's own system fields. + +Comparing a real `MasterBeaconRecord` fetched from CloudKit against the `OwnedBeacons` plist +OpenTagViewer already imports: + +| | CloudKit record | macOS cache plist | +| --- | --- | --- | +| Key material | `privateKey`, `publicKey`, `sharedSecret` | same names, but wrapped in `{key: {data: …}}` | +| Secondary secret | `sharedSecret2` | `secondarySharedSecret` — **the one rename** | +| Everything else | `productId`, `vendorId`, `model`, `isZeus`, `systemVersion`, `pairingDate`, `batteryLevel`, `stableIdentifier` | identical names | +| Record identity | in `recordIdentifier` | an `identifier` field | +| CloudKit system fields | in the envelope | a `cloudKitMetadata` blob | + +`BeaconNamingRecord` matches exactly: `name`, `associatedBeacon`, `roleId`, same camelCase. + +So Stage 6 is: unwrap the key dictionaries, rename one field, synthesise or omit +`cloudKitMetadata`, and decide what to do about `stableIdentifier` — a string in CloudKit, a list +in the plist. **It is not a format redesign**, which is why the layout parawanderer specified for +the community export matched what the Rust exporter produces. + +Two caveats. **Optional fields really are optional**: `emoji` on the naming record, and +`secureLocationsSharedSecret` and `groupIdentifier` on the beacon record, were all absent on the +observed account despite appearing in the schema. And `groupIdentifier` has no home in the plist +format at all — it is what FindMy.py 0.10.0 added support for — which is an argument for emitting +FindMy.py's native JSON rather than the older plist layout. + +## Two constraints worth knowing before planning around this + +**It removes the Mac, not the Apple device.** Stage 3 joins the iCloud Keychain trust circle by +escrow recovery, which requires the screen-lock passcode of a device already in the user's +circle — an iPhone PIN or a Mac login password. The device itself is not needed and is never +contacted, but the user has to know that passcode. An Apple ID that has never had an Apple +device in its keychain circle has nothing to recover from. + +**It writes to the user's Apple account**, in more than one way. See below. + +## Account-side residue is a first-class design problem + +This feature does not only read. It leaves artefacts behind in the user's Apple account, and +each one persists until something deletes it. Two are known so far: + +| Artefact | Created by | Persists as | Lifetime | +| --- | --- | --- | --- | +| A **registered device** | signing in with a machine identity Apple has not seen before | an entry in the account's device list, at appleid.apple.com and in iOS Settings | **kept while signed in** — see below | +| An **escrow record** ("bottle") | joining the keychain trust circle in Stage 3 | a recoverable keychain escrow entry, **not shown in any Apple UI** | **permanent unless explicitly deleted** — see below | + +**[observed] Escrow records outlive the devices that made them, and nothing cleans them up.** +The account these documents were written against showed *one* iMac Pro in its device list while +holding *eight* iMac Pro escrow records — leftovers from the macOS-VM route, whose device entries +had long since been removed. Deleting a device does not delete its escrow record. + +So of the two artefacts, the **escrow record is the one that actually accumulates**, and it does +so invisibly. The twenty phantom iMacs were the visible symptom; their escrow records are still +there. + +**The escrow record has no user-facing surface at all.** Apple's internal name for the mechanism +is *secure backup* — the last time it was directly visible to users was the iCloud Security Code +flow, since folded into iCloud Keychain and Account Recovery. There is a toggle and a +recovery-key screen; there is nothing enumerating records. + +What a bottle carries is essentially a copy of a device's identity: `device_name`, +`device_model`, `device_model_class`, `serial`, `build`, an escrow timestamp and a bottle id. So +it mirrors a device-list entry, which is why there is no separate screen for it — and which has +two consequences worth designing around: + +- **The labelling of Stage 1 §2.2 carries over for free.** A bottle created with the same + identity inherits the same recognisable serial, so it is identifiable by the same means as the + device — provided the identity is consistent across stages, which it must be anyway. +- **Deleting one is correspondingly dangerous.** A bottle belonging to a real iPhone looks like a + bottle belonging to us, and destroying the wrong one destroys that device's ability to recover + its keychain. Hence the enumerate-before-deleting rule below, and hence any deletion must + confirm against the serial rather than a list position. + +Neither is hypothetical. The macOS-VM export route this feature replaces accumulated **around +twenty phantom iMacs** in one maintainer's account, because every VM run minted a fresh identity +and every fresh identity registered as a new device. Nothing broke and nothing was banned — it +was simply irreversible clutter in a place the user cannot easily reason about. + +So the standing rule for every stage: + +> **Anything this feature creates in a user's Apple account must be created at most once, +> named so a human can recognise it among their real devices, and removable from inside the +> app.** + +Which means concretely: + +- **Never regenerate a persisted identity silently.** Identity state that is missing or + unreadable is an error to report, not a cue to mint a new one. Silent regeneration is the + precise mechanism by which twenty iMacs appear. +- **Reuse before creating.** One identity, one escrow record, reused across every run. +- **Label for the human, not the protocol.** An entry indistinguishable from real hardware is one + the user dare not delete; an obviously-ours one they can remove without a second thought. Use + whichever field the stage actually controls — in Stage 1 that is the serial number, not the + name. +- **Removal belongs in sign-out**, not in a hidden maintenance command that only someone reading + the source would find. +- **Enumerate before deleting.** The escrow list in particular contains the user's *real* + devices. Anything that deletes must show what it is about to delete and require the user to + confirm against identifying detail. + +**Settled, and the answer is the unwelcome one: signing in is enough.** Authentication alone +registers the device; no separate announce call is needed, so there is no read-only flow that +avoids it. See [01-authentication.md](./01-authentication.md) §13. + +The mitigation is cheap and confirmed to work: **supply a self-describing serial number**, which +Apple displays verbatim on the device's page. An entry reading `0PENTAGVIEWR` is one the user can +recognise and remove; one showing no serial at all is indistinguishable from real hardware. The +device also stays untrusted for verification codes, and should be kept that way. + +### The registration is what suppresses two-factor, so it stays + +**[observed]** A second sign-in from the same machine identity needed **no second factor at +all** — Apple remembered the machine and did not challenge it. + +That settles a design question that looked open. This feature is not a one-shot export: to notice +a newly-paired AirTag, the app has to re-read the account periodically. Recurring access means +recurring authentication, and authentication is only silent because a device registration exists +for Apple to recognise. + +**So the device is not residue. It is the application's identity on the account**, and it should +be treated like the identity of any other device the user owns: + +- **Created once**, and reused for the life of the installation. +- **Never churned.** Registering and removing around each operation would demand a two-factor + code every time, which is fatal for a background refresh and worse for the user than the entry + it was trying to avoid. +- **Removed at sign-out** — when the user deliberately disconnects the account — and at no other + time. +- **Legible while it exists**, via the serial number and, if the naming call is implemented, the + name. + +Residue is what an application leaves behind *after* it is done. A registration that is actively +in use is not that. The failure the twenty phantom iMacs represent was never "a device exists" — +it was *twenty* of them, unlabelled, from an identity regenerated on every run. + +## Verification status + +**Stages 1 and 2 are verified** against a real Apple account on 2026-08-13. Stage 1: +SRP handshake, server proof verified, session payload decrypted, trusted-device two-factor, 25 +service tokens. Stage 2: delegate exchange accepted, 11 iCloud service tokens returned — +including `searchPartyToken`, the one this project exists for. + +Stage 2 is verified in the stronger sense: the probe that exercised it was written **from the +specification**, not by calling any reference implementation. It worked on the first attempt, so +the document is one somebody has successfully built from. + +Findings are marked `[observed]` in each, and several corrected what reading the references had +suggested. Paths that remain untested: SMS two-factor, the 403 attributed to Advanced Data +Protection, the terms-of-service flow, and the device lifecycle of Stage 2 §7. + +**Stages 2 to 6 are not written and nothing about them has been executed.** They will be derived +by reading implementations that their authors report working. Each needs the same validation +against a real account before any claim that it works. From 4ba437577acd9f37770eb99bd23a865df4ce9aad Mon Sep 17 00:00:00 2001 From: "Shane B." Date: Thu, 13 Aug 2026 18:42:35 +0200 Subject: [PATCH 002/131] Amend specification from clean-room implementation feedback The implementation side raised sixteen gaps in docs/findmy-export/GAPS.md. All but one are now closed in the stage documents, by the specification side reading the references -- which is that side's job, and is the whole reason the split exists. Nothing was communicated to the implementer directly; the amendments live in the specifications. The most important change is a defect in the specification rather than a gap: Stage 5 documented the AES-GCM additional authenticated data as the ciphertext header alone. It is the header concatenated with a per-field context string of zone name, record name and field name. As written, every field decryption would have failed authentication with no diagnostic beyond "GCM error". Two consequences are now stated: decrypting a field needs the record's identity and not just its bytes, and each ciphertext is bound to its position. Also closed: the Cuttlefish function-invoke envelope and fetchViableBottles (Stage 3); the SP 800-108 KDF input and the RFC 6637 parameters, ciphertext layout and plaintext framing (Stage 5); which bits the master EC scalar keeps; what the structure HMAC covers; and every assumed protobuf field number (Stage 4). One assumption the implementation had made was wrong -- the RFC 6637 ciphertext is an MPI bit-count prefix and a compact point, not an X9.62 point. Five corrections accepted as raised: the numeric account identifier is available from Stage 2 and should be taken there; the PET is spent by Stage 2, which is an ordering constraint on the whole flow; the private key's v2 application tag is explicit; one persisted value may serve as both device UUID and UDID; and the iPhone identity is a worked example rather than a requirement. Still open: joining the trust circle after escrow recovery, Stage 3 6.7. Co-Authored-By: Claude Opus 5 --- docs/findmy-export/01-authentication.md | 11 +- docs/findmy-export/02-mobileme-delegate.md | 8 + docs/findmy-export/03-keychain-trust.md | 62 ++++- docs/findmy-export/04-cloudkit.md | 53 +++- docs/findmy-export/05-pcs-decryption.md | 122 +++++++-- docs/findmy-export/GAPS.md | 288 +++++++++++++++++++++ 6 files changed, 513 insertions(+), 31 deletions(-) create mode 100644 docs/findmy-export/GAPS.md diff --git a/docs/findmy-export/01-authentication.md b/docs/findmy-export/01-authentication.md index b3f50f75..1d9709c8 100644 --- a/docs/findmy-export/01-authentication.md +++ b/docs/findmy-export/01-authentication.md @@ -87,9 +87,16 @@ in the sense that it is time-derived, not that Apple rejects a repeat within its ### 2.2 Client identity -The client presents itself as an iPhone. The values are invented and are not validated against +The client presents itself as a device. The values are invented and are not validated against real hardware, but they must be **internally consistent** and **stable across logins**. +> **The iPhone below is a worked example, not a requirement.** A client presenting as a Mac works +> equally well, including against the `iosbuddy` endpoint of [Stage 2](./02-mobileme-delegate.md) +> despite the name — FindMy.py has long presented as a MacBook Pro and authenticates fine. What +> matters is that one client is consistent with itself. An implementation that already has an +> identity should **keep it**: introducing a second invalidates every existing session, and +> Stage 1 §13 explains what a changed identity costs. + A worked example, using an iPhone 14 Pro on iOS 17.4: | Field | Example value | @@ -102,7 +109,7 @@ A worked example, using an iPhone 14 Pro on iOS 17.4: | Device serial | **User-visible — choose it as a label.** Uppercase alphanumeric, 10–12 characters. `0PENTAGVIEWR` is confirmed accepted and displayed. See §13. | | Device name | any user-visible string | | Device UUID | a random v4 UUID, **uppercase**, generated once and persisted | -| UDID | 32 uppercase hex characters, generated once and persisted | +| UDID | 32 uppercase hex characters, generated once and persisted. **May be the same value as the device UUID**, with its hyphens removed — a UUID's hex form is exactly 32 uppercase hex characters. Nothing observed requires them to differ, and one persisted value is one fewer thing that can drift out of step with itself. | The OS version, build, CFNetwork version and Darwin version must correspond to one another — they describe one real iOS release, and Apple's own clients never disagree with themselves. diff --git a/docs/findmy-export/02-mobileme-delegate.md b/docs/findmy-export/02-mobileme-delegate.md index 6f52e8ef..b5368d7c 100644 --- a/docs/findmy-export/02-mobileme-delegate.md +++ b/docs/findmy-export/02-mobileme-delegate.md @@ -147,6 +147,14 @@ produce far better diagnostics than the third. **2. `status`.** An integer, and `0` means success. Anything else is a failure and the whole response is the diagnostic. +**`dsid`.** The response also carries a top-level `dsid`: the **numeric** account identifier. +This is the same value as `DsPrsId` in the Stage 1 session payload, and it is what +[Stage 4 §2.1](./04-cloudkit.md) needs as its HTTP Basic username. + +**Prefer this one.** Stage 1 §6 records that the session payload's keys vary between logins, so +depending on it for a value that is also available here is needless fragility. Taking `dsid` from +this response means Stage 4 needs nothing retained from Stage 1 at all. + **3. `delegates`.** A dictionary keyed by the same service identifiers that were requested. Each value is: diff --git a/docs/findmy-export/03-keychain-trust.md b/docs/findmy-export/03-keychain-trust.md index d44a7512..a2e7bbe7 100644 --- a/docs/findmy-export/03-keychain-trust.md +++ b/docs/findmy-export/03-keychain-trust.md @@ -68,6 +68,57 @@ Calls are made with the function-invoke operation — field `1101` of `RequestOp [Stage 4 §3.2](./04-cloudkit.md) — naming the target `Cuttlefish`, the method, and a protobuf body. The response body is protobuf too. +### 2.2 The invoke message + +Field `1101` of `RequestOperation` carries a `FunctionInvokeRequest`: + +| # | Field | Type | Value | +| --- | --- | --- | --- | +| 1 | `service` | string | `Cuttlefish` | +| 2 | `name` | string | the method, e.g. `fetchViableBottles` | +| 3 | `parameters` | bytes | **the method's own request message, serialised** | + +The response arrives at field `1101` of `ResponseOperation` as a `FunctionInvokeResponse`: + +| # | Field | Type | +| --- | --- | --- | +| 1 | `serializedResult` | bytes — **the method's own response message, serialised** | + +So there are **two layers of protobuf**: the CloudKit envelope, and inside `parameters` and +`serializedResult` an entirely separate message that CloudKit neither parses nor validates. A +malformed inner message therefore produces a successful CloudKit operation carrying a payload that +fails to decode, not a CloudKit error. + +### 2.3 `fetchViableBottles` + +The method [§5](#5-listing-what-the-account-has--read-only) step 2 needs. + +**Request:** + +| # | Field | Type | Value | +| --- | --- | --- | --- | +| 1 | `filter` | uint32 | `1` | +| 2 | `metrics` | bytes | empty | + +**Response:** + +| # | Field | Type | Meaning | +| --- | --- | --- | --- | +| 1 | `valid` | repeated `EscrowData` | **the usable bottles** | +| 2 | — | — | a legacy repeat of the same type; ignore | +| 3 | `partial` | repeated `EscrowMeta` | bottles that are not fully usable | + +**`EscrowData`:** + +| # | Field | Type | Meaning | +| --- | --- | --- | --- | +| 1 | `id` | string | **the join key** — matches an escrow record's `label` (§5 step 3) | +| 2 | `bottle` | message | the sealed material | +| 4 | `meta` | message | metadata | + +> Note field 3 is skipped, and that `EscrowMeta` is an **empty message** in every schema examined +> — it carries nothing, so a `partial` entry conveys only that a bottle exists in that state. + **The endpoint is a different service from the record database**: ``` @@ -90,13 +141,22 @@ id, exists for keychain item operations proper. | Value | From | | --- | --- | -| **PET** | Stage 1. Used as the escrow proxy's HTTP Basic password — so the five-minute expiry applies here too. | +| **PET** | Stage 1. Used as the escrow proxy's HTTP Basic password — see the warning below. | | The account's email | Stage 1 `acname`, the Basic username | | A working CloudKit container | Stage 4 §2, for every Cuttlefish call | | `com.apple.gs.icloud.escrow.auth` | Stage 1 issues this token; whether this stage needs it, or only the PET, is unestablished | | **The user's device passcode** | the user, at the moment of recovery. Never stored. | | Anisette headers | the same identity as every other stage | +> **The PET is spent by Stage 2, and this is an ordering constraint on the whole flow.** Stage 2 +> uses it as its HTTP Basic password and then replaces it with the service tokens it returns, so +> an implementation that follows Stage 2 as written has **no PET left** by the time this stage +> begins — and it expires in about five minutes regardless. +> +> So Stage 3 needs either a **second, fresh authentication** whose PET is retained rather than +> spent, or Stage 2 amended to keep a copy. This is not a detail of Stage 3; it changes how +> Stages 1 to 3 fit together, and an implementation that discovers it late will have to restructure. + ## 4. The escrow proxy ``` diff --git a/docs/findmy-export/04-cloudkit.md b/docs/findmy-export/04-cloudkit.md index 9bce0486..44c9990f 100644 --- a/docs/findmy-export/04-cloudkit.md +++ b/docs/findmy-export/04-cloudkit.md @@ -61,8 +61,10 @@ POST https://gateway.icloud.com/setup/setup/ck/v1/ckAppInit?container=com.apple. **Authentication is HTTP Basic**, and the credential pair is worth stating precisely because both halves are easy to get wrong: -- **username** — `DsPrsId`, the *numeric* account identifier from the Stage 1 session payload. - Not `adsid`, which is a different identifier in the same dictionary. +- **username** — the *numeric* account identifier. Not `adsid`, which is a different identifier + entirely. It is available as `DsPrsId` in the Stage 1 session payload **and** as the top-level + `dsid` of the Stage 2 delegate response; **prefer the latter**, since Stage 1's payload keys + vary between logins and Stage 2's do not. - **password** — `mmeAuthToken`. Not `cloudKitToken`, despite this being CloudKit. Headers are the CloudKit set of §2.2. @@ -208,7 +210,7 @@ the operation type**, which is the neatest thing in this protocol and worth rely | 202 | `zoneDeleteRequest` | delete a zone | | **203** | **`retrieveZoneChangesRequest`** | **which zones changed** | | 210 | `recordSaveRequest` | write records | -| **211** | **`recordRetrieveRequest`** | **fetch records by identifier** | +| 211 | `recordRetrieveRequest` | fetch records by identifier — **not needed**, see below | | **213** | **`retrieveChangesRequest`** | **which records changed — the main fetch** | | 214 | `recordDeleteRequest` | delete records | | 220 | `queryRetrieveRequest` | query records | @@ -231,8 +233,12 @@ The `Operation` message declares the same thing again as an enum: So the operation is declared twice — once as this enum, once by which high-numbered field of `RequestOperation` is populated — and the two must agree. -Only the four in bold are needed to read accessory data. Everything else in this schema writes, -deletes, shares or subscribes, and this project does none of those. +Only the three in bold are needed to read accessory data: list the zones, find what changed, and +fetch the changes. `recordRetrieveRequest` (211) fetches records by identifier and is **not +required** — `record/sync` returns everything — so its messages are not specified here. + +Everything else in this schema writes, deletes, shares or subscribes, and this project does none +of those. ### 3.2.1 Where operations are sent @@ -292,7 +298,7 @@ Responses are `ResponseOperation`, mirroring the request: | # | Field | Meaning | | --- | --- | --- | -| 1 | `operationCost` | what the operation cost against the account's budget | +| 1 | `operationCost` | uint32 — what the operation cost against the account's budget | | 2 | `response` | the `Operation`, echoing the type | | 3 | `result` | **success or failure — check this first** | | 4 | `bundled` | bundled sub-requests | @@ -313,7 +319,7 @@ total failure will discard good results. | 4 | `errorDescription` | string | human-readable | | 5 | `errorKey` | string | a stable key for the error | | 6 | `errorInternal` | string | Apple-internal detail | -| 7 | `extensionError` | message | `extensionName`, `typeCode`, `extensionPayload` | +| 7 | `extensionError` | message | 1 `extensionName` (string), 2 `typeCode` (uint32), 3 `extensionPayload` (bytes) | > **`clientError` and `serverError` are wrapper messages, not bare enums.** Each is a message > whose field 1 holds the code. Declaring them as plain integers decodes as @@ -351,7 +357,7 @@ in the same way escrow records do (see [Stage 3 §5](./03-keychain-trust.md)). | # | Field | Meaning | | --- | --- | --- | -| 1 | `targetZone` | the `Zone` — its identifier, etag, and **both** protection infos | +| 1 | `targetZone` | the `Zone`: 1 `zoneIdentifier`, 2 `etag` (string), 3 `protectionInfo`, 6 `recordProtectionInfo` | | 2 | `currentServerContinuationToken` | the sync token to start from | | 3 | `clientChangeToken` | the client's own change marker | | 4 | `deviceCount` | devices participating in the zone | @@ -377,7 +383,7 @@ in the same way escrow records do (see [Stage 3 §5](./03-keychain-trust.md)). | 25 | `isolationLevel` | `ZONE = 1` or `OPERATION = 2` | | 26 | `group` | operation group, matching the correlation headers of §2.2 | | 10 | `deviceLibraryName` | `com.apple.cloudkit.CloudKitDaemon` | -| 11 | `deviceLibraryVersion` | `1970` | +| 11 | `deviceLibraryVersion` | `1970`, as a **string** | | 29, 34 | undocumented | `0` | | 35 | undocumented | `1` | @@ -449,8 +455,16 @@ incremental sync. Two levels exist, and both work the same way. | 1 | `syncContinuationToken` | omit on a first run; everything is returned | | 2 | `maxChangedZones` | page size | -The response returns repeated `ChangedZone` entries — each with the zone `identifier`, a -`changeType` and a `deleteType` — plus a new `syncContinuationToken` and a `status`. +**`RetrieveZoneChangesResponse`:** + +| # | Field | Type | +| --- | --- | --- | +| 1 | `changes` | repeated `ChangedZone` | +| 2 | `syncContinuationToken` | bytes | +| 3 | `status` | int32 | + +**`ChangedZone`:** 1 `identifier`, 2 `changeType`, 3 `deleteType`, 4 `capabilities` (bytes), +5 `isAnonymous` (bool), 6 `anonymousZoneInfo` (bytes), 7 `zoneParentIdentifier`. **Which records changed** — `RetrieveChangesRequest`, field 213, sent to `/ckdatabase/api/client/record/sync`: @@ -466,7 +480,14 @@ The response returns repeated `ChangedZone` entries — each with the zone `iden | 8 | `ignoreCallingDeviceChanges` | skip changes this client made | | 9 | `includeMergeableDeltas` | include deltas rather than whole records | -Its response holds repeated `RecordChange`: +**`RetrieveChangesResponse`:** + +| # | Field | Type | +| --- | --- | --- | +| 1 | `change` | repeated `RecordChange` | +| 2 | `syncContinuationToken` | bytes | + +each `RecordChange` being: | # | Field | Meaning | | --- | --- | --- | @@ -535,9 +556,15 @@ separate-database design would. | # | Field | Meaning | | --- | --- | --- | -| 1 | `identifier` | a message wrapping the field's `name` string | +| 1 | `identifier` | **a message** whose field 1 is the `name` string | | 2 | `value` | the value | +> **Three places wrap a name in a message rather than using a bare string**, and they are easy to +> get wrong because both encode as length-delimited on the wire: a record's `type` (field 3 of +> `Record`), a change's `recordType` (field 3 of `RecordChange`), and a field's `identifier` +> here. **All three are messages with the string at field 1.** Declaring one as a bare string does +> not degrade gracefully — it throws, and takes the whole response with it. + **`Record.Field.Value`** is a tagged union — a `type` enum plus whichever typed field is populated: diff --git a/docs/findmy-export/05-pcs-decryption.md b/docs/findmy-export/05-pcs-decryption.md index fdbfd231..ea97c949 100644 --- a/docs/findmy-export/05-pcs-decryption.md +++ b/docs/findmy-export/05-pcs-decryption.md @@ -154,7 +154,7 @@ PublicKey ::= [APPLICATION 1] EXPLICIT SEQUENCE { PrivateKey ::= CHOICE { v1 SEQUENCE { key OCTET STRING, public PublicKey OPTIONAL }, - v2 [APPLICATION 5] SEQUENCE { data OCTET STRING } + v2 [APPLICATION 5] EXPLICIT SEQUENCE { data OCTET STRING } } ``` @@ -162,7 +162,9 @@ PrivateKey ::= CHOICE { > where they appear, not by their tag, so a decoder must be told which one to expect rather than > dispatching on the tag alone. -The Find My service is **v2** (§2), so its keys are the `[APPLICATION 5]` form. +The Find My service is **v2** (§2), so its keys are the `[APPLICATION 5]` form. That tag is +**explicit**, like every other application tag here — it wraps the SEQUENCE rather than replacing +its tag, so the contents are one level deeper than an implicit reading would place them. There is also a keyset wrapper used when keys are carried together: @@ -182,9 +184,49 @@ the keyset for an entry whose public key matches those bytes. That entry's ciphe wrapped key intended for us. If none matches, we do not hold a key for this record and no amount of retrying will change that — report it as a missing key, not as a decryption failure. -**Step 2 — unwrap it.** The wrapping is **RFC 6637** — the ECDH key-wrap construction from -OpenPGP — with the fixed parameter string `fingerprint`. The result is the PCS **master key**, -16 bytes. +**Step 2 — unwrap it.** The wrapping is **RFC 6637**, the ECDH key-wrap construction from +OpenPGP. It has three parts, and each has a detail worth stating. + +*The ciphertext layout* is OpenPGP's MPI encoding, **not** an X9.62 point: + +| Part | Size | +| --- | --- | +| Bit length of the ephemeral point | 2 bytes, big-endian — a count of **bits**, not bytes | +| Ephemeral public key | that many bits. **[observed] 256 bits — a 32-byte compact point**, not a 33- or 65-byte X9.62 encoding | +| Wrapped key length | 1 byte | +| Wrapped key | that many bytes | + +*The key derivation* is ECDH against that ephemeral point, then: + +``` +KEK = SHA-256( be32(1) ‖ shared_secret ‖ 08 2A 86 48 CE 3D 03 01 07 + ‖ 0x12 ‖ 0x03 0x01 ‖ 0x08 ‖ 0x07 + ‖ "Anonymous Sender " ‖ fingerprint ) +``` + +where the OID bytes are length-prefixed P-256, `0x12` is ECDH, `0x08` names SHA-256 as the KDF +hash, `0x07` names **AES-128** as the key-wrap cipher, the sender string is exactly twenty +characters including its four trailing spaces, and: + +> **The "fingerprint" is the literal ASCII word `fingerprint`, zero-padded to 20 bytes.** It is +> not a key fingerprint, not a hash of anything, and not eleven bytes — it occupies the 20-byte +> fingerprint slot of an otherwise ordinary RFC 6637 parameter block, right-padded with zeros. + +Only the **first 16 bytes** of that 32-byte digest are used, as an AES-128 key-encryption key. +The unwrapping itself is RFC 3394 AES key unwrap, whose integrity check is the first signal that +everything above was assembled correctly. + +*The plaintext* is framed, not bare: + +| Part | Content | +| --- | --- | +| byte 0 | algorithm identifier, **must be `1`** | +| bytes 1 … *L* | the key — the PCS master key, 16 bytes | +| next 2 bytes | big-endian checksum: the sum of the key bytes, modulo 65536 | +| trailing *p* bytes | padding, every byte equal to *p* | + +*L* is `len - p - 3`. Verify the checksum and the padding; both are cheap and both catch a wrong +key before it propagates. **Step 3 — derive the share key, conditionally.** @@ -205,7 +247,11 @@ not corruption. An owner signature may additionally be present as attribute `7`. before anything is decrypted: - the first 4 bytes of the derived key's **key id** must equal the structure's truncated key id -- the structure's **HMAC** must verify under the derived key +- the structure's **HMAC** must verify under the derived key, over the concatenation of three + things: the **DER of `keyset`**, then the raw bytes of **`meta`**, then the **DER of + `signatureData`** — in that order, and nothing else. Note it is a re-encoding of two sub- + structures rather than a span of the original bytes, so a decoder that discards the input after + parsing must be able to re-encode faithfully. A mismatch means the wrong key, not corrupt data. @@ -213,9 +259,27 @@ A mismatch means the wrong key, not corrupt data. ## 5. Key derivation -Every derived key comes from the master key by the same construction: a **counter-mode KDF with -HMAC** (the NIST SP 800-108 shape), with a fixed label and an output the same length as the input -key. +Every derived key comes from the master key by the same construction: **NIST SP 800-108 +counter-mode KDF with HMAC-SHA256**, with a fixed label and an output the same length as the +input key. + +The PRF input is, exactly: + +``` +HMAC-SHA256( master_key, be32(i) ‖ label ‖ 0x00 ‖ context ‖ be32(L) ) +``` + +| Part | Value | +| --- | --- | +| `i` | the block counter, **starting at 1**, big-endian 32-bit | +| `label` | the fixed label from the table below, as raw ASCII bytes | +| `0x00` | the separator SP 800-108 specifies | +| `context` | **empty** — no context is used | +| `L` | the output length **in bits**, big-endian 32-bit | + +Blocks are concatenated and truncated to the requested length. Since every use here requests 16 +bytes and SHA-256 produces 32, the counter never advances past 1 in practice — but implement the +loop anyway rather than assuming a single block. | Derived key | Label | | --- | --- | @@ -241,14 +305,16 @@ key by a route that is unusual enough to be worth spelling out: ``` out = PBKDF2-HMAC-SHA256(master_key, salt = "full master key", iterations = 10, length = 128) out = reverse(out) # the output is little-endian; big-endian is needed -n = mask_to_bit_length(out, order_bits(P-256)) +n = keep_low_bits(out, order_bits(P-256)) # the LOW 256 bits, not the high ones if n > order: n = n - order private_scalar = n ``` -Three traps in five lines: **ten** iterations, not a realistic PBKDF2 count; the output is -**reversed** because it is produced little-endian and consumed big-endian; and the reduction is a -single conditional subtraction rather than a modulo. The curve is **P-256** +Four traps in five lines. **Ten** iterations, not a realistic PBKDF2 count. The output is +**reversed**, because it is produced little-endian and consumed big-endian. The truncation keeps +the **low** 256 bits — the operation is a bit mask, not the `bits2int` convention that keeps the +high bits, and the two are indistinguishable by the arithmetic that follows. And the reduction is +a single **conditional subtraction**, not a modulo. The curve is **P-256** (`prime256v1`/`secp256r1`). ## 6. Field decryption @@ -278,8 +344,34 @@ After the header: | Tag | 12 bytes — **not the usual 16** | | Ciphertext | the remainder | -**The AAD is the entire header** — bytes 0 to `4 + N`, including the version byte and the length -byte, not merely the key id. +### The AAD is the header **and** a field context string + +> **Correction — this section previously said the AAD was the header alone, and that was wrong.** +> Decryption with only the header as AAD fails authentication on every field, with no diagnostic +> beyond "GCM error". The AAD is the concatenation of **two** parts: + +``` +AAD = header ‖ "--" +``` + +| Part | Content | +| --- | --- | +| First | the entire header — bytes 0 to `4 + N`, version and length byte included | +| Second | the context string, ASCII, no separator between it and the header | + +where the three names are the **zone's name** (`BeaconStore`), the **record's own name** from its +`recordIdentifier`, and the **field's name** from its `identifier` — joined by single hyphens. + +Two consequences worth stating: + +- **Decryption needs the record's identity, not just its bytes.** A function taking only a + ciphertext and a key cannot decrypt a PCS field. It needs the zone name, the record name and the + field name too, so the plumbing has to carry them down. +- **It binds each field to its position.** A ciphertext moved to another field, another record or + another zone will not authenticate, which is the point. + +For reference, a header as actually produced is six bytes: `03`, two key-id bytes, `02`, two more +key-id bytes — version 3, then the two-byte-plus-two-byte split of §6's layout. The key is `KDF(master_key, "encryption key key m")`, 16 bytes for AES-128. diff --git a/docs/findmy-export/GAPS.md b/docs/findmy-export/GAPS.md new file mode 100644 index 00000000..bf421ea5 --- /dev/null +++ b/docs/findmy-export/GAPS.md @@ -0,0 +1,288 @@ +# Gaps found while implementing + +Written by the implementation side of the clean-room split, addressed to the specification. + +> **Answered 2026-08-13.** Every gap below has been resolved in the specification documents, by +> the specification side reading the references — which is that side's job and is why the split +> exists. Nothing was told to the implementer directly; the amendments are in the stage documents +> and should be read from there. +> +> **Read Stage 5 §6 first.** Answering B2 turned up a defect in the specification serious enough +> that it would have failed every decryption: the AAD was documented as the header alone, and it +> is the header **concatenated with a per-field context string**. That was the specification's +> error, not the implementation's, and it is exactly the class of thing this document existed to +> surface. + +## Disposition + +| Gap | Outcome | +| --- | --- | +| **A1** Cuttlefish invoke message | **Closed.** Stage 3 §2.2 and §2.3 now specify the invoke envelope and `fetchViableBottles` with field numbers. | +| **A2** joining after recovery | **Still open.** Stage 3 §6.7 remains unwritten; it is the last unspecified section in the set. | +| **B1** the KDF's fixed input | **Closed.** Stage 5 §5. The default reading — `be32(i) ‖ label ‖ 0x00 ‖ context ‖ be32(bits)`, context empty, counter from 1 — was correct. | +| **B2** RFC 6637 parameters | **Closed.** Stage 5 §4 step 2, in full: MPI ciphertext layout, the KDF input, and the framed plaintext. **The ciphertext-layout assumption was wrong** — it is a 2-byte big-endian *bit* count then a 32-byte compact point, not an X9.62 point with a self-describing length. | +| **B3** a decrypted field's form | **Partly closed.** The AAD correction above is the load-bearing part. Date and integer encodings remain unstated; the runtime search is still the right approach. | +| **B4** which bits the EC scalar keeps | **Closed.** Stage 5 §5: the **low** bits. It is a bit mask, not `bits2int`. | +| **B5** what the HMAC covers | **Closed.** Stage 5 §4 step 5: DER of `keyset` ‖ raw `meta` ‖ DER of `signatureData`. | +| **C** assumed field numbers | **Closed.** Stage 4 §3.2.2, §3.2.3, §3.5, §3.6. Every assumption listed was correct. The three name-wrappers are now called out explicitly, since that was the one that throws rather than degrades. | +| **C1** `recordRetrieveRequest` | **Closed.** The bold was wrong; it is not required. Stage 4 §3.2 now says so. | +| **D1** `dsid` from Stage 2 | **Accepted.** Stage 2 §4 and Stage 4 §2.1 now prefer it, for the reason given. | +| **D2** the PET is spent by Stage 2 | **Accepted.** Stage 3 §3 now states it as an ordering constraint on the whole flow. | +| **D3** the v2 application tag | **Closed.** It is explicit, like the others. Stage 5 §3.2. | +| **D4** UDID from the device UUID | **Accepted.** Stage 1 §2.2 now permits it. | +| **D5** the client need not be an iPhone | **Accepted.** Stage 1 §2.2 now frames the iPhone as a worked example and says an existing identity should be kept. | + +**On the TLS note.** Agreed, and it is not merely a Stage 3 problem — a library-wide `ssl=False` +undermines the Apple Root CA pinning of Stage 1 §2.3 as well. Fixing it upstream is the right +move; until then no stage of this should be considered to have the transport security its +documents describe. + +--- + + +Per the [README](./README.md): *"If something here is ambiguous, incomplete or wrong, fix this +document — raise the gap, get the specification amended, and implement from the amendment."* +This is that raise. Nothing here was resolved by reading `rustpush`, `apple-private-apis` or +`export-findmy`; the questions below are exactly the ones that reading them would have answered, +and they are open because it was not done. + +**What was built:** Stages 4, 5 and 6 in [FindMy.py](https://github.com/malmeloo/FindMy.py), plus +the read-only half of Stage 3. + +> **Stage 4 §3 is now verified against a live account, on 2026-08-13**, by an implementation +> written from this document alone. Container opened, both zones listed, **35 records fetched +> across the same nine types §3.5.1 records, in the same counts.** Most of §C below is therefore +> answered; what remains open is marked. Stage 5 has still never touched a real record. + +Gaps are graded by what they cost: + +| Grade | Meaning | +| --- | --- | +| **A** | Blocks the stage. Cannot be implemented from the document as written. | +| **B** | Implemented by searching a small space of readings at runtime. Works, but the answer should be written down. | +| **C** | Assumed. A wrong assumption fails loudly and cheaply. | +| **D** | A correction or clarification, not a gap. | + +--- + +## A — blocking + +### A1. Stage 3 §2.1: the Cuttlefish function-invoke message is not specified + +§2.1 says Cuttlefish is called "with the function-invoke operation — field `1101` of +`RequestOperation` — naming the target `Cuttlefish`, the method, and a protobuf body", and §5 step +2 requires `fetchViableBottles`. + +**The message shape is not given.** Field 1101's request message, its field numbers, how the +target and method names are carried, and the shape of `fetchViableBottles`'s own request and +response are all absent. Stage 4 §3.2 tabulates every other operation's message in this much +detail; 1101 is the one that only gets a number. + +**Consequence:** Stage 3 §5 step 2 and step 3 are unimplementable, so the read-only listing cannot +tell viable bottles from stale ones. Only step 1 — the escrow proxy's own metadata — was built. It +also blocks §6.7 entirely, and with it every route to the `Manatee` keys that Stage 5 needs. + +**What would close it:** the same treatment §3.2 gives operations 201 and 213 — the request and +response message layouts with field numbers, for the invoke envelope and for `fetchViableBottles`. + +### A2. Stage 3 §6.7: joining the circle after recovery is named, not specified + +The document says so itself, and open question 3 records it. Worth restating here only because it +is what makes A1 load-bearing rather than merely annoying: even given A1, recovery ends with +"the escrowed keychain material" in a format nothing describes, and no route from there to an item +in the `Manatee` view whose `acct` matches a key in a record's protection structure. + +**Consequence:** the output contract Stage 3 sets itself — "we can enumerate `Manatee` and find an +item whose `acct` matches" — cannot be reached. Stage 5 is implemented against keys supplied by the +caller instead. + +--- + +## B — ambiguous, resolved by searching at runtime + +Each of these has a small space of plausible readings and a cheap way to tell which is right, +because Stage 5 §4 step 5 and §6 provide integrity checks. The implementation tries the candidates +and reports which one worked, at INFO level. **The first successful decryption of a real record +answers all three at once**, and the answers belong in the document. + +### B1. Stage 5 §5: the KDF's fixed input is not specified + +> "a **counter-mode KDF with HMAC** (the NIST SP 800-108 shape), with a fixed label and an output +> the same length as the input key" + +The construction is named and the labels are exact, but not *how* counter, label and length are +laid out in the PRF's input. SP 800-108 counter mode is +`PRF(K, [i] || Label || 0x00 || Context || [L])`, and the document supplies a label but no context +and no encoding widths. Five readings are tried: + +| Name | Fixed input | +| --- | --- | +| `sp800_108` (default) | `be32(counter) ‖ label ‖ 0x00 ‖ be32(bits)` | +| `sp800_108_no_separator` | `be32(counter) ‖ label ‖ be32(bits)` | +| `counter_label` | `be32(counter) ‖ label` | +| `label_counter` | `label ‖ be32(counter)` | +| `label_only` | `label` | + +This one matters most: it is used four times over, including for the encryption key, so getting it +wrong fails everything downstream with no clue as to why. + +### B2. Stage 5 §4 step 2: the RFC 6637 parameters are not specified + +> "The wrapping is **RFC 6637** … with the fixed parameter string `fingerprint`." + +RFC 6637's `Param` is a structure, not a string: curve OID, algorithm id, KDF hash id, KEK +algorithm id, the constant `"Anonymous Sender "`, and a 20-byte recipient fingerprint. Two +readings of the sentence are possible — that `fingerprint` fills the fingerprint slot of an +otherwise ordinary `Param`, or that `Param` is literally the eleven bytes `fingerprint` — and +neither the hash nor the KEK algorithm is stated. Six combinations are tried; AES key unwrap's own +integrity check says which fits. + +Also unstated, and **not** guessed: the ciphertext's layout. It is read as an X9.62 point followed +by the wrapped key, with the point's length taken from its own leading byte, which is +self-describing and so needs no assumption. Whether the unwrapped material is the 16-byte master +key or carries a leading algorithm byte, as OpenPGP's use of this construction does, is tried both +ways. + +### B3. Stage 5 §8 Q1: a decrypted field's form + +Open question 1 asks whether a decrypted field is raw bytes or an `EncryptedValue` message, and +says the answer "decides whether decryption yields a value or another parse step". Both are +accepted: the plain reading is tried first, and the wrapped one when the plain reading does not +look like the declared type. A string is the awkward case — a small protobuf message holding a +short string decodes as valid UTF-8, its tag and length bytes arriving as control characters — so +"it decoded" is not sufficient and "it decoded and reads like text" is used instead. + +Two further encodings are unstated and had to be chosen: + +- **Dates.** Read as seconds from 2001-01-01 rather than from the Unix epoch, as an IEEE double, + big- or little-endian. Told apart by plausibility: a pairing date after 2019 and not in the + future. A wrong reading is rejected rather than exported. +- **Integers.** Read big-endian. + +### B4. Stage 5 §5: which bits the master EC scalar keeps + +> `n = mask_to_bit_length(out, order_bits(P-256))` + +Masking 1024 bits down to 256 can keep the low bits or the high ones, and "mask" suggests the +former while the conventional `bits2int` keeps the latter. Both satisfy the conditional +subtraction that follows, so the arithmetic does not disambiguate it. + +**This one is cheap**, because the master EC key is used only to verify a protection structure's +signature (§4 step 4) and not to decrypt anything. It is exposed as a parameter and signature +verification is off by default. Worth resolving before anyone turns verification on. + +### B5. Stage 5 §4 step 5: what the structure's HMAC covers + +> "the structure's **HMAC** must verify under the derived key" + +The HMAC key's derivation is given (`hmackey-of-masterkey`) but not what it is computed over — the +DER of the whole structure, of the keyset, of the structure with the HMAC field itself absent, or +something else. Compare §3.2's `ShareProtectionKeySet`, where the document is careful to say the +hash is "computed with `hash` itself absent"; the equivalent sentence is missing here. + +**Consequence:** only one of §4 step 5's two checks is implemented. The truncated key id is +sufficient to select a key, so nothing is blocked, but one of the two independent confirmations is +absent. + +--- + +## C — assumed field numbers + +Stage 4 §3 gives field numbers for most messages and omits them for these. Each was assumed; a +wrong assumption means protobuf silently routes the data to the unknown-field set and the caller +sees an empty result, so the implementation logs which field numbers actually arrived whenever a +response decodes as empty. **One live `record/sync` settles all of them.** + +| Message | Assumed | Where the document stops | +| --- | --- | --- | +| `RetrieveChangesResponse` | 1 = repeated `recordChange`, 2 = `syncContinuationToken` | §3.5 gives `RecordChange`'s own layout but not its envelope's | +| `RetrieveZoneChangesResponse` | 1 = repeated `changedZone`, 2 = token, 3 = status | §3.5 names the members only | +| `ChangedZone` | 1 = `identifier`, 2 = `changeType`, 3 = `deleteType` | as above | +| `Zone` | 1 = `zoneIdentifier`, 2 = `etag` | §3.2.3 pins only `protectionInfo` (3) and `recordProtectionInfo` (6) | +| `ExtensionError` | 1, 2, 3 | §3.2.2 names the three members | +| the name wrapper | `name` at 1 | §3.6 says "a message wrapping a `name` string" | +| `Header.deviceLibraryVersion` | a string | §3.3 gives the value `1970`, which could be either | +| `ResponseOperation.operationCost` | int64 | §3.2.2 names it without a type | +| `RetrieveChangesRequest.requestedFields` | repeated string | §3.5 names it without a type | + +**One of these is riskier than the rest.** §3.6 describes a record's `type` and a field's +`identifier` as "a message wrapping a `name` string", while §3.5 describes `RecordChange.recordType` +as "the record's type name" — which reads like a bare string. A bare string and a wrapper message +are both length-delimited on the wire, so declaring the wrong one does not degrade, it throws and +takes the whole response with it. All three are declared as `bytes` and resolved at read time +instead. **Worth stating explicitly in the document which they are**, because a reader will +otherwise reasonably declare a message and lose an afternoon. + +### C1. Stage 4 §3.2: `recordRetrieveRequest` is listed as needed but never specified + +Field 211 is one of the four operations marked in bold as required to read accessory data, but +unlike 201, 203 and 213 its request and response messages are never given. It turns out not to be +needed — `record/sync` returns everything — so it is not implemented. Either specify it or drop the +bold. + +--- + +## D — corrections and clarifications + +### D1. Stage 4 §2.1: `DsPrsId` is also returned by Stage 2 + +§2.1 says the HTTP Basic username is "the **numeric** account identifier from the Stage 1 session +payload — not `adsid`". True, but it implies the Stage 1 payload is the only source, and Stage 1 +§6's own note that the payload's keys vary makes depending on it awkward. + +**The MobileMe delegate response of Stage 2 carries the same numeric identifier as a top-level +`dsid`.** FindMy.py already persists it and already uses it as the Basic username against the +search-party service with `searchPartyToken`, which works — so the two are the same value. The +implementation uses that, which means Stage 4 needs nothing retained from Stage 1 at all. + +Suggested amendment to Stage 2 §5: state that the response's top-level `dsid` is `DsPrsId`. + +### D2. Stage 3 §3: the PET is gone by the time Stage 3 runs + +§3 lists the PET as a prerequisite and notes the five-minute expiry. What it does not say is that +the PET is *consumed* by Stage 2 — it is the Basic password of the delegate exchange — and any +implementation that follows Stage 2 as written will have thrown it away by the time Stage 3 starts. +FindMy.py does exactly this: `idms_pet` is replaced by the service tokens at the end of login. + +So Stage 3 does not merely need a PET "within five minutes"; it needs a **second, fresh** login +whose PET is retained rather than spent, or Stage 2 must be told to keep it. Worth a sentence, +because it is an ordering constraint on the whole flow rather than a detail of Stage 3. + +### D3. Stage 5 §3.2: `PrivateKey` v2 is the only application tag written without `EXPLICIT` + +Every other application tag in the document is written `[APPLICATION n] EXPLICIT SEQUENCE`. +`PrivateKey`'s v2 arm is written `[APPLICATION 5] SEQUENCE`. If that is deliberate the tag replaces +the SEQUENCE's own tag and the scalar is one level shallower; if it is a typo it does not. Both +nestings are accepted; the document should say which is meant. + +### D4. Stage 1 §2.2: the UDID can be derived from the device UUID + +§2.2 asks for both a "Device UUID" (v4, uppercase, persisted) and a "UDID" (32 uppercase hex, +persisted) as separate values. Stage 4 §3.3 then requires the UDID as `deviceHardwareID`. + +Two independently persisted identity values are two things that can drift apart, and the README's +standing rule is that identity state must never be silently regenerated. A UUID's hex form is +exactly 32 uppercase hex characters, so **one persisted value can serve both**, which is what the +implementation does. Worth noting as permitted, unless something requires them to differ. + +### D5. Stage 4 §2.2 and §3.3: this client is a Mac, not an iPhone + +The specification's identity is an iPhone throughout, and Stage 2 §3 notes the `iosbuddy` endpoint +matches it. FindMy.py has long presented as a `MacBookPro18,3` on `Mac OS X 13.4.1` — including to +that same `iosbuddy` endpoint — and it works. + +The implementation keeps FindMy.py's existing identity rather than introducing a second one, +swapping only the bundle for CloudKit's. Consistency within one client matters more than which +platform it claims, and changing it would invalidate every existing user's session. Not a defect in +the document, but its worked example should not be read as a requirement. + +--- + +## Not a specification gap, but worth knowing + +**FindMy.py disables TLS certificate verification.** `findmy/util/http.py` passes `ssl=False` to +every request, which turns verification off for the whole library. Stage 3 §6.6 requires pinned +certificates specifically because the material crossing that exchange is the user's entire +keychain, protected by a passcode short enough to brute-force offline. Those two facts cannot both +stand. Fixing it is out of scope for a protocol implementation and is upstream's call, but Stage 3 +must not be built on that session as it is. From 53e23b5d16257348945a451797a3976f6a4aef98 Mon Sep 17 00:00:00 2001 From: "Shane B." Date: Thu, 13 Aug 2026 18:52:21 +0200 Subject: [PATCH 003/131] Answer the decrypted-field form from ciphertext sizes Stage 5 open question 1 asked whether a decrypted field is raw bytes or a wrapped EncryptedValue. It is both, chosen by declared type, and the answer falls out of arithmetic rather than needing a key. Ciphertext overhead is fixed at 30 bytes, so plaintext length is the observed length minus 30. For the four ENCRYPTED_BYTES fields of a MasterBeaconRecord that gives 85, 57, 32 and 32 -- exactly the key material sizes in the committed macOS cache fixture. Those decrypt to raw bytes. The scalar fields fit none of that and fit an EncryptedValue wrapper precisely: 36 bytes for a UUID string, 6 for "AirTag", a 9-byte Date holding a double, and two bytes for a varint that needs no length prefix. Also documents four response fields an earlier draft omitted, two of which arrive in practice: status at 4 and zoneAttributesChanges at 12. Still open: whether the date counts from 2001 or 1970. Co-Authored-By: Claude Opus 5 --- docs/findmy-export/04-cloudkit.md | 20 ++++++-- docs/findmy-export/05-pcs-decryption.md | 47 +++++++++++++++--- docs/findmy-export/GAPS.md | 63 ++++++++++++++++++++----- 3 files changed, 106 insertions(+), 24 deletions(-) diff --git a/docs/findmy-export/04-cloudkit.md b/docs/findmy-export/04-cloudkit.md index 44c9990f..a4aa709b 100644 --- a/docs/findmy-export/04-cloudkit.md +++ b/docs/findmy-export/04-cloudkit.md @@ -482,10 +482,22 @@ incremental sync. Two levels exist, and both work the same way. **`RetrieveChangesResponse`:** -| # | Field | Type | -| --- | --- | --- | -| 1 | `change` | repeated `RecordChange` | -| 2 | `syncContinuationToken` | bytes | +| # | Field | Type | Notes | +| --- | --- | --- | --- | +| 1 | `change` | repeated `RecordChange` | | +| 2 | `syncContinuationToken` | bytes | | +| 3 | `clientChangeToken` | bytes | | +| 4 | `status` | int32 | **[observed] present, value 3**, on every page | +| 5 | `changedShares` | repeated bytes | sharing; not needed here | +| 6 | `pendingArchivedRecords` | bool | | +| 7 | `changedDeltas` | repeated bytes | mergeable deltas | +| 8 | `syncObligations` | repeated bytes | | +| 12 | `zoneAttributesChanges` | bytes | **[observed] ~900 bytes on the first page only** | + +Fields 4 and 12 arrive in practice and an earlier draft of this document omitted both, so a +decoder built from it routed them to the unknown-field set. They are not needed to read +accessories — but a client that logs unmodelled fields will see them, and should not treat their +presence as a sign the schema is wrong. each `RecordChange` being: diff --git a/docs/findmy-export/05-pcs-decryption.md b/docs/findmy-export/05-pcs-decryption.md index ea97c949..107d9d38 100644 --- a/docs/findmy-export/05-pcs-decryption.md +++ b/docs/findmy-export/05-pcs-decryption.md @@ -380,17 +380,50 @@ The key is `KDF(master_key, "encryption key key m")`, 16 bytes for AES-128. ## 7. Interpreting the plaintext -Stage 4 §3.6 establishes that a field's declared `type` describes its **plaintext**, so once -decrypted the bytes are interpreted according to it — a `STRING_TYPE` field yields a string, an -`INT64_TYPE` field an integer, and so on. The protocol also defines a small `EncryptedValue` -message holding a signed value, a date or a string, which is the shape a decrypted field takes. +Stage 4 §3.6 establishes that a field's declared `type` describes its **plaintext**. What the +plaintext *is* depends on that type, and the two cases differ: -Which of those two forms applies to which field type is **not established** — see §8. +| Declared type | Decrypted plaintext | +| --- | --- | +| `ENCRYPTED_BYTES_TYPE` (20) | **raw bytes.** No wrapper. | +| `STRING_TYPE`, `INT64_TYPE`, `DATE_TYPE`, and the other scalars | an **`EncryptedValue` message**, with the value in the field matching its type | + +`EncryptedValue` carries `signedValue` at field 3, `dateValue` at 5 and `stringValue` at 6. + +> **[observed] This is established by arithmetic, without decrypting anything.** Ciphertext +> overhead is fixed at 30 bytes — a 6-byte header, a 12-byte IV and a 12-byte tag (§6) — so +> plaintext length is the ciphertext length minus 30. Comparing that against the accessory data +> in the macOS cache format ([Stage 6](./06-output.md)): +> +> | Field | Ciphertext | Plaintext | macOS cache | +> | --- | --- | --- | --- | +> | `privateKey` | 115 | **85** | **85** | +> | `publicKey` | 87 | **57** | **57** | +> | `sharedSecret` | 62 | **32** | **32** | +> | `sharedSecret2` | 62 | **32** | **32** | +> +> Four exact matches: the `ENCRYPTED_BYTES_TYPE` fields decrypt to raw key material with nothing +> around it. The scalars do not fit that pattern and fit a wrapper exactly: +> +> | Field | Plaintext | As an `EncryptedValue` | +> | --- | --- | --- | +> | `stableIdentifier` | 38 | tag + length + **36** = a UUID string | +> | `model` | 8 | tag + length + **6** = `AirTag` | +> | `pairingDate` | 11 | tag + length + a 9-byte `Date` (tag + 8-byte double) | +> | `batteryLevel`, `isZeus`, `vendorId` | 2 | tag + a 1-byte varint | +> +> Every one lands exactly. Note the varint cases carry **no length prefix** — protobuf varint +> fields are not length-delimited — which is why they are two bytes rather than three. + +**The date's epoch is still unsettled.** The `Date` message holds a double, and whether it counts +from 2001-01-01 as Apple's frameworks do or from the Unix epoch is not established here. Reject +an implausible result — a pairing date in the future, or before AirTags existed — rather than +exporting it. ## 8. Open questions -1. **Is a decrypted field raw bytes, or the `EncryptedValue` message?** §7 cannot say, and the - answer decides whether decryption yields a value or another parse step. +1. ~~Is a decrypted field raw bytes, or the `EncryptedValue` message?~~ **[observed] Both, by + type** — see §7. What remains open is only the **date epoch**: 2001 or 1970. 2. **What are the `SharingCircleSecret` records for?** [Stage 4 §3.5.1](./04-cloudkit.md) observes five of them in the zone, each with a `secretType`, a `sharingCircleIdentifier` and a `secretData` blob. The plausible reading is that they protect accessories shared with others, diff --git a/docs/findmy-export/GAPS.md b/docs/findmy-export/GAPS.md index bf421ea5..1af68718 100644 --- a/docs/findmy-export/GAPS.md +++ b/docs/findmy-export/GAPS.md @@ -186,32 +186,69 @@ absent. --- -## C — assumed field numbers +## C — assumed field numbers, now mostly confirmed Stage 4 §3 gives field numbers for most messages and omits them for these. Each was assumed; a wrong assumption means protobuf silently routes the data to the unknown-field set and the caller -sees an empty result, so the implementation logs which field numbers actually arrived whenever a -response decodes as empty. **One live `record/sync` settles all of them.** +sees an empty result rather than an error. + +**The live run of 2026-08-13 confirmed every assumption it exercised**, so these are no longer +guesses and the document can simply state them: + +| Message | Assumed, and **[confirmed]** | Evidence | +| --- | --- | --- | +| `RetrieveChangesResponse` | 1 = repeated `recordChange`, 2 = `syncContinuationToken` | 35 records decoded, and paging advanced | +| `Zone` | 1 = `zoneIdentifier`, 2 = `etag` | both zones named themselves correctly | +| the name wrapper | `name` at 1 | record types decoded without the fallback path firing | +| `ZoneSummary` | 1 = `targetZone`, 4 = `deviceCount` | `BeaconStore` reported 11 devices, `_defaultZone` none | + +Still assumed, because nothing exercised them: | Message | Assumed | Where the document stops | | --- | --- | --- | -| `RetrieveChangesResponse` | 1 = repeated `recordChange`, 2 = `syncContinuationToken` | §3.5 gives `RecordChange`'s own layout but not its envelope's | | `RetrieveZoneChangesResponse` | 1 = repeated `changedZone`, 2 = token, 3 = status | §3.5 names the members only | | `ChangedZone` | 1 = `identifier`, 2 = `changeType`, 3 = `deleteType` | as above | -| `Zone` | 1 = `zoneIdentifier`, 2 = `etag` | §3.2.3 pins only `protectionInfo` (3) and `recordProtectionInfo` (6) | | `ExtensionError` | 1, 2, 3 | §3.2.2 names the three members | -| the name wrapper | `name` at 1 | §3.6 says "a message wrapping a `name` string" | | `Header.deviceLibraryVersion` | a string | §3.3 gives the value `1970`, which could be either | | `ResponseOperation.operationCost` | int64 | §3.2.2 names it without a type | | `RetrieveChangesRequest.requestedFields` | repeated string | §3.5 names it without a type | -**One of these is riskier than the rest.** §3.6 describes a record's `type` and a field's -`identifier` as "a message wrapping a `name` string", while §3.5 describes `RecordChange.recordType` -as "the record's type name" — which reads like a bare string. A bare string and a wrapper message -are both length-delimited on the wire, so declaring the wrong one does not degrade, it throws and -takes the whole response with it. All three are declared as `bytes` and resolved at read time -instead. **Worth stating explicitly in the document which they are**, because a reader will -otherwise reasonably declare a message and lose an afternoon. +**One was riskier than the rest, and the risk did not materialise.** §3.6 describes a record's +`type` and a field's `identifier` as "a message wrapping a `name` string", while §3.5 describes +`RecordChange.recordType` as "the record's type name" — which reads like a bare string. A bare +string and a wrapper message are both length-delimited on the wire, so declaring the wrong one does +not degrade, it throws and takes the whole response with it. All three were declared as `bytes` and +resolved at read time, and the wrapper reading turned out to be correct. **Worth stating explicitly +in the document that these are wrappers with the name at field 1**, since a reader who guesses the +other way loses the whole response rather than one field. + +### C2. `RetrieveChangesResponse` carries a field 4 the document does not mention [observed] + +The first page returned 35 records and a continuation token. A second call with that token returned +**no records, no token, and a field 4** the schema does not model — so the paging loop makes one +request whose only content is that field. + +Two things follow, and the second matters more than the first: + +- The document should record field 4's existence. By analogy with `RetrieveZoneChangesResponse`, + which §3.5 does say carries a `status`, it is most likely a status or completion flag. +- **If it is a "no more changes" indicator, reading it saves a request per fetch.** The README's + own rule 6 treats unnecessary requests against Apple as an account-flagging risk rather than mere + slowness, and this feature is meant to poll periodically, so one wasted round trip per poll is + worth removing. Right now the only way to know a fetch is complete is to ask again and be told + nothing. + +The implementation now logs the wire type and value of any unmodelled field, so the next run +identifies it. Its earlier diagnostic overstated this as "the schema's assumed field numbers are +likely wrong", which was misleading — the numbers were right; the field was simply extra. That has +been fixed to distinguish "decoded nothing" from "decoded fine, plus something unknown". + +### C3. Not yet exercised: the parts Stage 5 actually needs + +Fetching proved the envelope, but the run printed only record types and counts. **Whether +`Record.recordField` (7) and `Record.protectionInfo` (13) decoded is still unknown**, and those two +are precisely what Stage 5 consumes — a record with zero fields would look identical in that +output. The probe now reports both, so the next run settles it. ### C1. Stage 4 §3.2: `recordRetrieveRequest` is listed as needed but never specified From 79a2a52dccfb4533e7a7a029633498dab8c14192 Mon Sep 17 00:00:00 2001 From: "Shane B." Date: Thu, 13 Aug 2026 19:00:43 +0200 Subject: [PATCH 004/131] Specify joining the trust circle after escrow recovery Stage 3 6.7 was the last unwritten section and the blocker between recovered escrow material and usable keys. It now specifies the sequence and its cryptography. The shape is not obvious and explains why the step is five operations rather than one: the recovered blob is a bottled peer, the sealed identity of a device already in the circle. Recovering it lets the client borrow that device's identity only long enough for it to vouch for a new identity of its own, and it is the new identity that joins. Documents the HKDF-SHA384 derivation of the bottle's three keys with the account identifier as salt, the P-384 scalar conversion, the four verifications to perform before decrypting anything, the AES-256-GCM unsealing, and the vouch-then-join sequence. Calls out one hazard prominently: establish and joinWithVoucher differ by a single branch, and an implementation that falls back from the second to the first on error will silently destroy the user's existing trust circle. This project should never call establish. Still open, and now the largest gap in the set: the protobuf layouts of the peer, voucher and key-share messages that joinWithVoucher carries. Co-Authored-By: Claude Opus 5 --- docs/findmy-export/03-keychain-trust.md | 83 ++++++++++++++++++++++--- 1 file changed, 74 insertions(+), 9 deletions(-) diff --git a/docs/findmy-export/03-keychain-trust.md b/docs/findmy-export/03-keychain-trust.md index a2e7bbe7..1273dae0 100644 --- a/docs/findmy-export/03-keychain-trust.md +++ b/docs/findmy-export/03-keychain-trust.md @@ -12,7 +12,7 @@ Read [README.md](./README.md) first. In particular: **implement from this docume > the reference had suggested. > > **Everything from §6 onward is unverified**: recovery, joining the circle, enrolling and -> deleting. Those are derived by reading implementations whose authors report them working. +> deleting. §6.7's message layouts are moreover incomplete — see §8. Those are derived by reading implementations whose authors report them working. > > It is also the stage that **needs the user's device passcode** and **writes to their account**. > Everything before it was read-only; this is not. Treat the residue rules of §7 as part of the @@ -453,18 +453,82 @@ entire keychain, and the passcode protecting it is short enough to brute-force o attacker obtains the blob. Do not disable verification, and do not fall back to the system trust store because pinning was inconvenient. -### 6.7 After recovery +### 6.7 From recovered material to a trusted peer -The recovered material lets the client reconstitute itself as a peer in the trust circle via -Cuttlefish (§2.1). **That sequence is not specified here** — establishing or joining via a -voucher, and syncing trust changes, are named in the protocol and remain unwritten. See §8. +The §6.5 plaintext is not the keychain. It is a **bottled peer** — the sealed identity of a +device that was already in the circle. Recovering it lets this client impersonate that device +just long enough for it to vouch for a *new* identity of our own, which is what actually joins. + +That indirection is the whole design, and it is why the sequence has five steps rather than one. + +**Step 1 — parse the recovered blob.** It is a property list with one field that matters: a +**bottled peer entropy** blob. Everything below derives from it plus the account's `adsid`. + +**Step 2 — derive the bottle's three keys.** All by **HKDF-SHA384**, with the **`adsid` as salt** +and the entropy as the input keying material: + +| Key | HKDF info string | Output | +| --- | --- | --- | +| Symmetric | `Escrow Symmetric Key` | 32 bytes | +| Signing | `Escrow Signing Private Key` | 56 bytes → an EC scalar | +| Encryption | `Escrow Encryption Private Key` | 56 bytes → an EC scalar | + +The two EC keys are on **P-384**, and the 56-byte output is converted to a scalar by **FIPS 186-4 +B.5.1**, the extra-random-bits method — generate more bits than the order needs and reduce, rather +than retrying. + +> Note the salt: the account identifier, not a random value. Two accounts recovering the same +> entropy would derive different keys, which is the point. + +**Step 3 — verify before trusting.** The bottle carries public keys and signatures, and all four +checks should pass before anything is decrypted: + +- the derived encryption key's public part equals the bottle's escrowed encryption key +- the derived signing key's public part equals the bottle's escrowed signing key +- the bottle's own signature verifies under the escrowed signing key, **SHA-384** +- the sponsoring peer's signature over the bottle verifies under that peer's known key + +A mismatch on either of the first two means the passcode produced the wrong entropy. The third and +fourth mean the bottle is not what it claims. + +**Step 4 — open the bottle.** **AES-256-GCM**, keyed by the symmetric key from step 2, with the +IV and authentication tag carried alongside the ciphertext in the bottle structure. The plaintext +holds the sponsoring peer's **signing and encryption private keys**, again P-384. + +At this point the client holds a second peer's identity. It has not joined anything. + +**Step 5 — vouch, then join.** + +1. Generate a **new identity of our own** — a fresh peer, with its own keys. +2. Using the recovered peer, **sign a voucher** for the new identity. This is the recovered + device saying "I trust this one". +3. **Fetch the recovered peer's key shares** — the TLK shares that carry the actual keychain view + keys. **If it has none, stop**: joining would succeed and yield no keys, which is a worse + outcome than failing. +4. **Create a bottle for the new identity**, sealed under the *device passcode*, so that this + client is itself recoverable later. This is the escrow record §7 discusses. +5. Call Cuttlefish **`joinWithVoucher`** with the new peer, its bottle, the voucher's sponsor, and + the re-shared keys. + +The response carries trust changes to apply to local state. A client with no voucher would call +**`establish`** instead, creating a new circle rather than joining one — **not what this project +wants**, and calling it by mistake resets the account's trust rather than joining it. + +> **`establish` and `joinWithVoucher` differ by one branch and are catastrophically different.** +> The first is for a device forming a circle where none exists; the second for joining one that +> does. An implementation that falls back from the second to the first on error will silently +> destroy the user's existing trust circle. There is no reason this project should ever call +> `establish`. + +**Step 6 — sync the views.** With trust established, sync the `Manatee` and `ProtectedCloudStorage` +views (§6.8). Only then does [Stage 5](./05-pcs-decryption.md) have keys. > **Passcode handling.** Read it, use it twice, discard it. Never written to disk, never logged, > never in diagnostics, and not retained in memory past the exchange. It is the most sensitive > value this project touches: it is the key to the user's whole keychain, and unlike an Apple ID > password it cannot be rotated without physical access to the device. -### What this stage must actually deliver +### 6.8 What this stage must actually deliver [Stage 5 §2](./05-pcs-decryption.md) makes the output contract concrete, and it is narrower than "join the circle": @@ -562,9 +626,10 @@ This stage has more of them than the rest of the set combined, and they are load 2. **What is the exact SRP exchange for recovery?** §6 is a sketch. The parameters, the derivation from the passcode, and the response format all need specifying before anything can be implemented. -3. **What does joining the circle actually involve after recovery?** Reconstituting a peer, - establishing or joining via a voucher, and syncing trust changes are all named in the protocol - and none are specified here. +3. **What are the peer, voucher and TLK-share message layouts?** §6.7 specifies the *sequence* and + its cryptography, but the protobuf messages it passes to `joinWithVoucher` — the peer's + permanent and dynamic info, the voucher, and the key shares — are not enumerated. This is the + largest remaining gap in the set, and it is the same kind of work Stage 4 §3 did for CloudKit. 4. **Which token authenticates what?** Stage 1 issues `com.apple.gs.icloud.escrow.auth`, but the escrow proxy is authenticated with the PET. Whether that token is needed at all is unknown. 5. **Can bottles be listed without any prior trust state?** §5 is presented as read-only and From 1ed2ce9414c903f6e398fb48f5c3bedff3a687ae Mon Sep 17 00:00:00 2001 From: "Shane B." Date: Thu, 13 Aug 2026 19:04:10 +0200 Subject: [PATCH 005/131] Tell implementers not to look for coherence that is not there The stage documents record a number of inconsistencies -- four wire formats in one flow, two SRP variants in one login, commands spelled differently in the URL and the body, hand-padded fixed strings, unnamed constants. Read as a designed protocol these look like errors in the documentation, and a reader will reasonably go looking for the rule that explains them. There is no rule. This is strata, not architecture, and the inconsistency is the specification. Saying so up front saves the search. Co-Authored-By: Claude Opus 5 --- docs/findmy-export/README.md | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/docs/findmy-export/README.md b/docs/findmy-export/README.md index 0d81f088..6a5b9f93 100644 --- a/docs/findmy-export/README.md +++ b/docs/findmy-export/README.md @@ -65,6 +65,36 @@ file from those projects. --- +## Do not look for the underlying logic + +This is called a protocol specification throughout, and for an implementer it is one: every layer +must be spoken correctly or nothing comes out. But it was plainly never *designed* as one. It is +strata — work by different teams in different years, each building on whatever was already there. + +That matters because it changes how to read these documents. **Where they record an inconsistency, +the inconsistency is the specification.** There is no rule underneath it waiting to be found, and +looking for one costs time. Some examples, all documented in the stages below: + +- **Four wire formats in one flow** — XML property lists for Grand Slam and escrow, JSON for the + two-factor endpoints, protobuf for CloudKit, DER ASN.1 for the protection structures. +- **Two different SRP variants in one login.** Grand Slam deviates from RFC 5054 in three specific + ways; escrow recovery deviates in none. The same code serves both only if parameterised. +- **Commands named twice, differently.** `get_records` in the URL is `GETRECORDS` in the body, and + `get_club_cert` is `GETCLUB`. There is no transformation connecting the pairs; they are tables. +- **The operation type declared twice** in a CloudKit request, as an enum and as a field number, + and required to agree. +- **A keychain trust protocol tunnelled through a storage API.** Octagon's server side is invoked + as a CloudKit server-side function, its protobuf inside a `bytes` field CloudKit never parses. +- **Fixed strings padded to length by hand** — key-derivation labels of exactly twenty characters, + `Anonymous Sender ` with its four trailing spaces, and an RFC 6637 fingerprint slot holding + the literal ASCII word `fingerprint`. +- **Constants nobody can name.** Three integer header fields set to 0, 0 and 1 on every request. + +None of this is a criticism of anyone. It is what a system looks like after fifteen years of +being extended by people solving the problem in front of them. But an implementer who expects +coherence will keep searching for a pattern that is not there, so: **expect none, and copy the +tables exactly.** + ## The stages The full flow from an Apple ID to accessory private keys is six stages. They are specified From a25558fab96c1429a2f4e22c1528917391ad003d Mon Sep 17 00:00:00 2001 From: "Shane B." Date: Thu, 13 Aug 2026 19:34:49 +0200 Subject: [PATCH 006/131] Enumerate the trust-circle join messages Stage 3 6.9 gives field numbers for everything joinWithVoucher carries: the peer and its four signed blobs, the voucher, the bottle and its inner structures, and the key shares. This was the last outstanding message enumeration in the set. Several details are the kind that fail silently. SignedInfo's signature covers the serialised bytes rather than the parsed message, so a client must send back exactly what it signed. OTBottle reserves fields 3 to 7, and its inner bottle starts at field 3 with no fields 1 or 2. TlkShare declares wrapped keys, signatures and public keys as strings rather than bytes. And CuttlefishEstablishRequest carries the same four fields as the join request in a different order, which is one more reason the two must not be confused. The whole flow now has a written specification. What remains is that nothing from Stage 3 6 onward has been run. Co-Authored-By: Claude Opus 5 --- docs/findmy-export/03-keychain-trust.md | 143 +++++++++++++++++++++++- docs/findmy-export/GAPS.md | 96 ++++++++++++++++ 2 files changed, 234 insertions(+), 5 deletions(-) diff --git a/docs/findmy-export/03-keychain-trust.md b/docs/findmy-export/03-keychain-trust.md index 1273dae0..7093c525 100644 --- a/docs/findmy-export/03-keychain-trust.md +++ b/docs/findmy-export/03-keychain-trust.md @@ -12,7 +12,7 @@ Read [README.md](./README.md) first. In particular: **implement from this docume > the reference had suggested. > > **Everything from §6 onward is unverified**: recovery, joining the circle, enrolling and -> deleting. §6.7's message layouts are moreover incomplete — see §8. Those are derived by reading implementations whose authors report them working. +> deleting. The sequence, cryptography and message layouts are all specified; none has been run. Those are derived by reading implementations whose authors report them working. > > It is also the stage that **needs the user's device passcode** and **writes to their account**. > Everything before it was read-only; this is not. Treat the residue rules of §7 as part of the @@ -528,6 +528,139 @@ views (§6.8). Only then does [Stage 5](./05-pcs-decryption.md) have keys. > value this project touches: it is the key to the user's whole keychain, and unlike an Apple ID > password it cannot be rotated without physical access to the device. +### 6.9 The messages `joinWithVoucher` carries + +All are protobuf, and all travel inside the invoke envelope of §2.2 — so they are serialised into +a `bytes` field that CloudKit does not inspect. + +**`CuttlefishJoinWithVoucherRequest`** — the request itself: + +| # | Field | Type | +| --- | --- | --- | +| 1 | `restorePoint` | string — the client's sync token, if it has one | +| 2 | `peer` | `CuttlefishPeer` | +| 3 | `bottle` | `Bottle` — the escrow record being created for the new identity | +| 4 | `shares` | repeated `TlkShare` | +| 5 | `keys` | repeated `ViewKeys` | + +`CuttlefishEstablishRequest` is the same message with the fields in a **different order** — peer 1, +bottle 2, keys 3, shares 4 — and no restore point. Another reason not to confuse the two (§6.7). + +**`CuttlefishPeer`** — everything except `hash` is a signed blob: + +| # | Field | Type | +| --- | --- | --- | +| 1 | `hash` | string — the peer's identifier | +| 2 | `permanentInfo` | `SignedInfo` | +| 3 | `stableInfo` | `SignedInfo` | +| 4 | `dynamicInfo` | `SignedInfo` | +| 5 | `voucher` | `SignedInfo` | + +**`SignedInfo`** is the wrapper that makes this work — field 1 `info` (bytes, a serialised message) +and field 2 `signature` (bytes). **The signature covers the serialised bytes, not the parsed +message**, so a client must keep the exact bytes it signed and send those; re-encoding before +sending invalidates the signature even if the content is identical. + +**`PeerPermanentInfo`** — inside `permanentInfo.info`: + +| # | Field | +| --- | --- | +| 1 | `epoch` (uint64) | +| 2 | `signingKey` (bytes) | +| 3 | `encryptionKey` (bytes) | +| 4 | `machineId` (string) | +| 5 | `modelId` (string) | +| 6 | `creationTime` (uint64) | + +**`PeerStableInfo`** — inside `stableInfo.info`. Larger, and mostly optional: + +| # | Field | | # | Field | +| --- | --- | --- | --- | --- | +| 1 | `clock` | | 10 | `flexiblePolicyVersion` | +| 2 | `frozenPolicyVersion` | | 11 | `flexiblePolicyHash` | +| 3 | `frozenPolicyHash` | | 12 | `userControllableViewStatus` | +| 4 | `secrets` (repeated bytes) | | 13 | `custodianRecoveryKeys` (repeated) | +| 5 | `osVersion` | | 14 | `secureElementIdentity` | +| 6 | `deviceName` | | 15 | `walrus` | +| 7 | `recoverySigningPublicKey` | | 16 | `webAccess` | +| 8 | `recoveryEncryptionPublicKey` | | 18 | `isInheritedAccount` (bool) | +| 9 | `serialNumber` | | | | + +Note field 17 is absent. `deviceName` and `serialNumber` are how this peer appears to the user, so +the labelling rules of the [README](./README.md) apply here as they do to the device registration. + +**`PeerDynamicInfo`** — inside `dynamicInfo.info`, and the actual statement of who trusts whom: + +| # | Field | Type | +| --- | --- | --- | +| 1 | `clock` | uint64 | +| 2 | `includeds` | repeated string — peer ids this peer trusts | +| 3 | `excludeds` | repeated string — peer ids it has removed | +| 4 | `dispositions` | repeated `PeerDisposition` | +| 5 | `preapprovals` | repeated string | + +**`Voucher`** — inside `voucher.info`, and remarkably small for what it does: + +| # | Field | Type | +| --- | --- | --- | +| 1 | `reason` | uint32 | +| 2 | `beneficiary` | string — the peer being vouched **for** | +| 3 | `sponsor` | string — the peer vouching | + +Three fields. The signature on the enclosing `SignedInfo`, made with the recovered peer's signing +key, is the entire weight of the claim. + +**`Bottle`** — the escrow record created for the new identity: + +| # | Field | Type | +| --- | --- | --- | +| 2 | `bottle` | bytes — a serialised `OTBottle` | +| 3 | `escrowedSigningKey` | bytes | +| 4 | `escrowedKeySignature` | bytes | +| 5 | `peerKeySignature` | bytes | +| 6 | `peerID` | string | +| 7 | `bottleID` | string | + +Field 1 is absent. `escrowedKeySignature` and `peerKeySignature` are the two signatures §6.7 step 3 +verifies when recovering. + +**`OTBottle`** — inside `bottle.bottle`: + +| # | Field | Type | +| --- | --- | --- | +| 1 | `peerID` | string | +| 2 | `bottleID` | string — a UUID | +| 8 | `escrowedSigningKey` | bytes | +| 9 | `escrowedEncryptionKey` | bytes | +| 10 | `peerSigningKey` | bytes | +| 11 | `peerEncryptionKey` | bytes | +| 12 | `ciphertext` | message: 1 `ciphertext`, 2 `authenticationCode`, 3 `initializationVector` | + +**Fields 3 to 7 are reserved and must not be used.** The AES-256-GCM unsealing of §6.7 step 4 +takes its three inputs from field 12; note the tag is carried separately from the ciphertext and +must be appended before decryption, or supplied to a detached-tag interface. + +The plaintext is an **`OTInternalBottle`**, which is two fields and both are at surprising numbers: +`signingKey` at **3** and `encryptionKey` at **4**, each an `OTPrivateKey` of `keyType` (1) and +`keyData` (2). There is no field 1 or 2. + +**`TlkShare`** — one per view key being handed to the new peer. Note that almost everything is a +**string**, including things that look like binary: + +| # | Field | Type | | # | Field | Type | +| --- | --- | --- | --- | --- | --- | --- | +| 1 | `service` | string | | 7 | `receiverPublicEncryptionKey` | **string** | +| 2 | `curve` | uint64 | | 8 | `sender` | string | +| 3 | `epoch` | uint64 | | 9 | `signature` | **string** | +| 4 | `keyId` | string | | 10 | `version` | uint32 | +| 5 | `poisoned` | uint32 | | 11 | `wrappedKey` | **string** | +| 6 | `receiver` | string | | | | | + +**`ViewKeys`** — sent when *establishing* keys rather than receiving them, so this project sends an +empty list: field 1 `service`, then `topLevelKey` (2), `classA` (3), `classC` (4) and +`oldTopLevelKey` (5), each a `ViewKey` of `keyId` (1), `topLevelKeyId` (2), `keyNumber` (3), +`key` (4) and a fifth field whose name is misspelled in every schema examined. + ### 6.8 What this stage must actually deliver [Stage 5 §2](./05-pcs-decryption.md) makes the output contract concrete, and it is narrower than @@ -626,10 +759,10 @@ This stage has more of them than the rest of the set combined, and they are load 2. **What is the exact SRP exchange for recovery?** §6 is a sketch. The parameters, the derivation from the passcode, and the response format all need specifying before anything can be implemented. -3. **What are the peer, voucher and TLK-share message layouts?** §6.7 specifies the *sequence* and - its cryptography, but the protobuf messages it passes to `joinWithVoucher` — the peer's - permanent and dynamic info, the voucher, and the key shares — are not enumerated. This is the - largest remaining gap in the set, and it is the same kind of work Stage 4 §3 did for CloudKit. +3. **How are the peer's signed blobs constructed, exactly?** §6.9 enumerates every message, but + two things it does not settle: which fields of `PeerStableInfo` a client is *required* to + populate, and what the policy version and hash fields must contain. A peer that signs an + incomplete stable info may be admitted and then behave oddly rather than being rejected. 4. **Which token authenticates what?** Stage 1 issues `com.apple.gs.icloud.escrow.auth`, but the escrow proxy is authenticated with the PET. Whether that token is needed at all is unknown. 5. **Can bottles be listed without any prior trust state?** §5 is presented as read-only and diff --git a/docs/findmy-export/GAPS.md b/docs/findmy-export/GAPS.md index 1af68718..da326b42 100644 --- a/docs/findmy-export/GAPS.md +++ b/docs/findmy-export/GAPS.md @@ -323,3 +323,99 @@ certificates specifically because the material crossing that exchange is the use keychain, protected by a passcode short enough to brute-force offline. Those two facts cannot both stand. Fixing it is out of scope for a protocol implementation and is upstream's call, but Stage 3 must not be built on that session as it is. + +--- + +# Second round — gaps found implementing the amendments + +Written after implementing every closed gap above. Stage 5 is now exact throughout, Stage 4's +schema states rather than assumes, and Stage 3's Cuttlefish invoke path is built. + +Same grading. Nothing here was resolved by reading the references. + +## E — blocking + +### E1. Stage 3 §6.7 gives the sequence and the cryptography, but not the message layouts + +§6.7 is a large step forward and closes the "what actually happens" question. It does not close +the implementation question, because **not one of the structures it manipulates is specified**. +Compare §2.3, which gives `fetchViableBottles` a field-numbered request and response; §6.7 names +six steps and no messages. + +Concretely, what is missing, step by step: + +| Step | Specified | Missing | +| --- | --- | --- | +| 1 — parse the recovered blob | it is a plist holding a bottled peer entropy blob | **the key it is under**, and what the plist's other fields are | +| 2 — derive three keys | exactly: HKDF-SHA384, `adsid` as salt, three info strings, P-384, FIPS 186-4 B.5.1 | nothing. **This step is implementable today** | +| 3 — verify | which four checks, and that the digest is SHA-384 | the **bottle structure**: where the escrowed encryption key, escrowed signing key, bottle signature and sponsor signature live, and how the sponsoring peer's key is obtained | +| 4 — open the bottle | AES-256-GCM under the symmetric key | where the **IV and tag** sit in the bottle structure, the tag length, and what the AAD is — Stage 5 §6 is a standing warning that both are easy to get wrong and fail identically | +| 5 — vouch and join | the order of operations, and that `establish` must never be called | **what a peer is** (its fields, how an identity is generated and serialised), **what a voucher is and what it signs over**, **what a TLK share is**, what "re-shared keys" means concretely, and `joinWithVoucher`'s request and response messages | +| 6 — sync the views | that it must happen | **the whole of it.** No operation, container, message or view-sync protocol is named | + +So the honest status is that §6.7 makes the *shape* of the work clear and leaves it unbuildable. +**Step 2 is the exception** and is worth implementing immediately as a testable primitive, because +it is stated precisely enough to be right or wrong on its own. + +**What would close it:** the treatment §2.3 already gives one method — field-numbered request and +response messages — extended to `joinWithVoucher`, to the peer and voucher structures, to the +bottle structure of steps 3 and 4, and to whatever step 6 turns out to be. + +> **A note on where the danger is.** §6.7's warning about `establish` is well taken and the +> implementation will never call it. But the more likely failure is quieter: step 5.3 says to stop +> if the recovered peer has no key shares, because joining would otherwise succeed and yield no +> keys. Without a specified TLK share structure there is no way to *perform* that check, so an +> implementation built from §6.7 as it stands would join the circle, write an escrow record, and +> then discover it has nothing — having left two permanent artefacts on the account for nothing. +> That is the outcome the README's residue rules exist to prevent, and it is currently reachable. + +## F — smaller, from the same pass + +### F1. Stage 5 §4 step 4: what the structure's *signature* covers is still unstated + +B5 closed what the **HMAC** covers, and that is implemented. The **signature** of the same step is +still described only as "the structure is signed" — there is no statement of what data is signed. +`SignatureData` carries a `version` and a `data`, and §3.1's note says the version numbering is not +established, so even the obvious guess is not safe. + +Consequence: the master EC key derivation is implemented and tested — B4 closed which bits it keeps +— but nothing uses it, because there is nothing to verify against. It is dead code until this is +answered, which is a strange place for a correctly-specified derivation to sit. + +### F2. Stage 3 §5 step 3: what joins to what is ambiguous + +The join key is described as an escrow record's `label` matching a bottle's `id`. §5.1 separately +records that the decoded metadata carries its own `bottleID`. On the observed account those are +presumably equal, but the document does not say so, and an implementation has to pick. + +This one is handled rather than blocked: the implementation matches on **either**, and reports both +mismatch directions rather than dropping them. Worth one sentence stating whether they are always +the same value. + +### F3. Stage 4 §3.5: the extra response fields are documented, and one of them is useful + +The amendment documenting fields 3 through 12 is confirmed correct — declaring them silences the +diagnostic, and a live run now reports nothing unmodelled. + +**Field 4, `status`, came back as the varint `3` on both a full page and an empty one.** If it +distinguishes "more to come" from "complete", reading it would remove one request per fetch: the +loop currently learns a page was the last only by asking again and being told nothing. Given this +feature polls periodically and the README treats surplus requests to Apple as an account-flagging +risk, that is worth knowing. What its values mean is not stated. + +## G — confirmed by implementing + +Recorded so the next reader knows these were exercised rather than assumed: + +- **The KDF, verbatim.** `be32(i) ‖ label ‖ 0x00 ‖ be32(bits)` with an empty context reproduces + independently. The counter loop is implemented even though 16 bytes from SHA-256 never advances it. +- **The RFC 6637 rewrite.** MPI bit count, 32-byte compact point, AES-128 KEK from the first half of + the digest, and the framed plaintext with its checksum and self-describing padding all round-trip. + The compact point needs the y coordinate recovered by solving the curve equation; either root + works, since a point and its negation share an x and ECDH takes only x. +- **The AAD correction.** Implemented as `header ‖ "--"`. The consequence the + document predicts is real: decryption now takes a field's identity, and the plumbing to carry a + zone name from the fetch down to a field decrypt was the largest single change of this round. +- **EncryptedValue's field numbers.** Round-tripping the wrapper reproduces the exact plaintext + sizes the amendment derived from ciphertext sizes — 8 bytes for `AirTag`, 11 for a date, 2 for a + small integer, 38 for a UUID string. Independent confirmation that 3, 5 and 6 are right. From 379b007547bf66ae54f885b9b5edeb20f7dd752b Mon Sep 17 00:00:00 2001 From: "Shane B." Date: Thu, 13 Aug 2026 19:48:07 +0200 Subject: [PATCH 007/131] Specify what the protection structure's signature covers, and the paging terminator Two gaps raised by the implementation side. Stage 5 4 step 4 said only that "the structure is signed". It now gives the nested ObjectSignature that SignatureData.data decodes to, and the exact byte sequence the signature covers -- a hand-built concatenation rather than a re-encoded structure, with one asymmetry worth the warning: an absent symmKeyCount contributes four zero bytes while absent attributes or key lists contribute nothing. Also records which key verifies which of the three possible signatures, so the master EC key derivation now has something to verify against. Stage 4 3.5's status field is the paging terminator: 3 means the zone is fully synced. The implementation side guessed this from observing it on both a full and an empty page, and the guess was right. Worth more than saving a request -- a response can carry status 3 and still contain changes, so a client that stops on an empty page always pages once more than it needs to. Co-Authored-By: Claude Opus 5 --- docs/findmy-export/04-cloudkit.md | 14 ++++-- docs/findmy-export/05-pcs-decryption.md | 64 ++++++++++++++++++++++--- docs/findmy-export/GAPS.md | 61 ++++++++++++----------- 3 files changed, 98 insertions(+), 41 deletions(-) diff --git a/docs/findmy-export/04-cloudkit.md b/docs/findmy-export/04-cloudkit.md index a4aa709b..6ce7c2e0 100644 --- a/docs/findmy-export/04-cloudkit.md +++ b/docs/findmy-export/04-cloudkit.md @@ -487,7 +487,7 @@ incremental sync. Two levels exist, and both work the same way. | 1 | `change` | repeated `RecordChange` | | | 2 | `syncContinuationToken` | bytes | | | 3 | `clientChangeToken` | bytes | | -| 4 | `status` | int32 | **[observed] present, value 3**, on every page | +| 4 | `status` | int32 | **the paging terminator — see below** | | 5 | `changedShares` | repeated bytes | sharing; not needed here | | 6 | `pendingArchivedRecords` | bool | | | 7 | `changedDeltas` | repeated bytes | mergeable deltas | @@ -495,9 +495,15 @@ incremental sync. Two levels exist, and both work the same way. | 12 | `zoneAttributesChanges` | bytes | **[observed] ~900 bytes on the first page only** | Fields 4 and 12 arrive in practice and an earlier draft of this document omitted both, so a -decoder built from it routed them to the unknown-field set. They are not needed to read -accessories — but a client that logs unmodelled fields will see them, and should not treat their -presence as a sign the schema is wrong. +decoder built from it routed them to the unknown-field set. + +> **`status == 3` means the zone is fully synced.** It is how a client knows to stop paging, and +> it is more reliable than inferring completion from an empty page or an unchanged token: **a +> response can carry `status = 3` and still contain changes**, so a client that stops only when a +> page comes back empty always spends one request more than it needs to. Loop while `status != 3`, +> carrying the continuation token forward each time. +> +> Other values of this field have not been observed and their meanings are unestablished. each `RecordChange` being: diff --git a/docs/findmy-export/05-pcs-decryption.md b/docs/findmy-export/05-pcs-decryption.md index 107d9d38..66a73f32 100644 --- a/docs/findmy-export/05-pcs-decryption.md +++ b/docs/findmy-export/05-pcs-decryption.md @@ -102,15 +102,33 @@ KeyRef ::= SEQUENCE { `keyset` is a **SET OF**, not a SEQUENCE OF — the entries are unordered and must be searched by matching `pubKey`, never indexed. -**`SignatureData`** — carries the version that §4 step 3 branches on: +**`SignatureData`** — carries the version that §4 step 3 branches on, and a nested structure: ```asn1 SignatureData ::= SEQUENCE { version INTEGER, - data OCTET STRING + data OCTET STRING -- itself DER: an ObjectSignature } ``` +**`ObjectSignature`**, found by decoding `SignatureData.data` — this is what §4 step 4 verifies +against: + +```asn1 +ObjectSignature ::= SEQUENCE { + rollCount INTEGER, + outerSignKeyType INTEGER, + public KeyRef, + signature Signature, + symmKeyCount [0] EXPLICIT INTEGER OPTIONAL, + signature2 [1] EXPLICIT Signature OPTIONAL, + ecKeyList [2] EXPLICIT SEQUENCE OF KeyRef OPTIONAL, + attributes [3] EXPLICIT SEQUENCE OF Attribute OPTIONAL +} +``` + +`signature2` is the "past" signature §4 step 4 falls back to — it is key rotation, not corruption. + > The version numbering is **not clearly established**. The reference implementation's own note > on it is self-contradictory, mapping the same values to different names in one breath. What is > established is only what §4 step 3 needs: **the share-key derivation is skipped when the @@ -238,10 +256,44 @@ if signature_data.version != 5 and not read_only: Otherwise the master key is used directly. **This branch is easy to miss and produces a key that fails every subsequent check**, so verify against the checksum in step 5 rather than pressing on. -**Step 4 — verify the signatures.** The structure is signed, and unless the read-only flag is set -the signature is verified against a key derived from the master key itself (§5). A second, -"past" signature may be present and should be tried if the first fails — that is key rotation, -not corruption. An owner signature may additionally be present as attribute `7`. +**Step 4 — verify the signatures.** + +The signed data is a **hand-built concatenation, not a DER structure** — nothing re-encodes it as +one message, so it must be assembled in exactly this order: + +| Order | Bytes | +| --- | --- | +| 1 | DER of `keyset` | +| 2 | raw `meta` | +| 3 | `outerSignKeyType`, big-endian 32-bit | +| 4 | `rollCount`, big-endian 32-bit | +| 5 | `symmKeyCount`, big-endian 32-bit — **`0` when absent**, not omitted | +| 6 | `public.keytype`, big-endian 32-bit | +| 7 | `public.pubKey`, raw | +| 8 | DER of `attributes`, **only if present** | +| 9 | DER of `ecKeyList`, **only if present** | + +Note the asymmetry: an absent `symmKeyCount` contributes four zero bytes, while absent +`attributes` or `ecKeyList` contribute nothing at all. Getting that backwards is a four-byte +difference that fails verification with no other symptom. + +Signatures are **ECDSA over SHA-256**. `Signature.digest` is `1` for SHA-256; nothing observed +uses `2`. + +**Which key verifies what:** + +| Signature | Verified with | +| --- | --- | +| `ObjectSignature.signature` | the **master EC key** derived per §5 — this is what B4's bit-ordering answer is for | +| `ShareProtection.signature`, when present | the private key that unwrapped the entry, or a caller-supplied signing key | +| Attribute `7`, when present | a caller-supplied signing key | + +**The master-key check is skipped when the read-only flag is set**, and falls back to `signature2` +if the first fails. + +> **`Signature.keyid`, when non-empty, is the compressed public key of the signer** — not a hash +> or an identifier. Check it before verifying: a mismatch names the wrong key immediately, where a +> failed verification does not. An **empty** `keyid` means self-signed. **Step 5 — check you have the right key.** Two independent checks, and both should be performed before anything is decrypted: diff --git a/docs/findmy-export/GAPS.md b/docs/findmy-export/GAPS.md index da326b42..13f5e2fb 100644 --- a/docs/findmy-export/GAPS.md +++ b/docs/findmy-export/GAPS.md @@ -18,7 +18,7 @@ Written by the implementation side of the clean-room split, addressed to the spe | Gap | Outcome | | --- | --- | | **A1** Cuttlefish invoke message | **Closed.** Stage 3 §2.2 and §2.3 now specify the invoke envelope and `fetchViableBottles` with field numbers. | -| **A2** joining after recovery | **Still open.** Stage 3 §6.7 remains unwritten; it is the last unspecified section in the set. | +| **A2** joining after recovery | **Closed.** Stage 3 §6.7 specifies the sequence and the cryptography, §6.9 the messages. See [E1](#e1-closed-by-69) below for what was built from them. | | **B1** the KDF's fixed input | **Closed.** Stage 5 §5. The default reading — `be32(i) ‖ label ‖ 0x00 ‖ context ‖ be32(bits)`, context empty, counter from 1 — was correct. | | **B2** RFC 6637 parameters | **Closed.** Stage 5 §4 step 2, in full: MPI ciphertext layout, the KDF input, and the framed plaintext. **The ciphertext-layout assumption was wrong** — it is a 2-byte big-endian *bit* count then a 32-byte compact point, not an X9.62 point with a self-describing length. | | **B3** a decrypted field's form | **Partly closed.** The AAD correction above is the load-bearing part. Date and integer encodings remain unstated; the runtime search is still the right approach. | @@ -335,39 +335,38 @@ Same grading. Nothing here was resolved by reading the references. ## E — blocking -### E1. Stage 3 §6.7 gives the sequence and the cryptography, but not the message layouts +### E1 — closed by §6.9 -§6.7 is a large step forward and closes the "what actually happens" question. It does not close -the implementation question, because **not one of the structures it manipulates is specified**. -Compare §2.3, which gives `fetchViableBottles` a field-numbered request and response; §6.7 names -six steps and no messages. +*Raised when §6.7 landed without message layouts; closed by §6.9, which supplies them.* -Concretely, what is missing, step by step: +§6.9 enumerates every message the join carries, and all of it is now modelled in +`findmy/cloudkit/proto/cuttlefish.proto`. The four members that fail silently rather than loudly +each have a test rather than a comment: -| Step | Specified | Missing | -| --- | --- | --- | -| 1 — parse the recovered blob | it is a plist holding a bottled peer entropy blob | **the key it is under**, and what the plist's other fields are | -| 2 — derive three keys | exactly: HKDF-SHA384, `adsid` as salt, three info strings, P-384, FIPS 186-4 B.5.1 | nothing. **This step is implementable today** | -| 3 — verify | which four checks, and that the digest is SHA-384 | the **bottle structure**: where the escrowed encryption key, escrowed signing key, bottle signature and sponsor signature live, and how the sponsoring peer's key is obtained | -| 4 — open the bottle | AES-256-GCM under the symmetric key | where the **IV and tag** sit in the bottle structure, the tag length, and what the AAD is — Stage 5 §6 is a standing warning that both are easy to get wrong and fail identically | -| 5 — vouch and join | the order of operations, and that `establish` must never be called | **what a peer is** (its fields, how an identity is generated and serialised), **what a voucher is and what it signs over**, **what a TLK share is**, what "re-shared keys" means concretely, and `joinWithVoucher`'s request and response messages | -| 6 — sync the views | that it must happen | **the whole of it.** No operation, container, message or view-sync protocol is named | - -So the honest status is that §6.7 makes the *shape* of the work clear and leaves it unbuildable. -**Step 2 is the exception** and is worth implementing immediately as a testable primitive, because -it is stated precisely enough to be right or wrong on its own. - -**What would close it:** the treatment §2.3 already gives one method — field-numbered request and -response messages — extended to `joinWithVoucher`, to the peer and voucher structures, to the -bottle structure of steps 3 and 4, and to whatever step 6 turns out to be. - -> **A note on where the danger is.** §6.7's warning about `establish` is well taken and the -> implementation will never call it. But the more likely failure is quieter: step 5.3 says to stop -> if the recovered peer has no key shares, because joining would otherwise succeed and yield no -> keys. Without a specified TLK share structure there is no way to *perform* that check, so an -> implementation built from §6.7 as it stands would join the circle, write an escrow record, and -> then discover it has nothing — having left two permanent artefacts on the account for nothing. -> That is the outcome the README's residue rules exist to prevent, and it is currently reachable. +| Trap | How it is held | +| --- | --- | +| `SignedInfo`'s signature covers the **serialised bytes**, not the parsed message | `SignedBlob` in `findmy/keychain/join.py` holds bytes and a signature, never a message. There is nothing in it to re-encode, so a signature cannot be invalidated by a re-serialisation that changes bytes without changing content. | +| `OTBottle` reserves 3–7 | `reserved 3, 4, 5, 6, 7;` in the schema, which makes reuse a compile error rather than a convention | +| `OTInternalBottle` starts at field 3 | modelled as-is, with a test asserting its field set is exactly `{3, 4}` | +| `TlkShare` declares binary-looking members as `string` | modelled as `string`, with a test asserting the wire type of all three | + +**`CuttlefishEstablishRequest` is deliberately not defined at all.** The document's warning is +sound, but a comment saying "never call this" is weaker than not being able to. With the message +undefined, no code in the package can construct one, and a test asserts the name appears nowhere. +That the two requests carry the same four members in a *different order* is the argument for +structural prevention rather than discipline: a confusion between them would serialise perfectly +cleanly and mean something else entirely. + +**Step 5.3's check is implemented and is a refusal, not a warning.** `require_key_shares` raises +rather than logging, because the failure it prevents is the expensive kind: joining without shares +*succeeds*, and leaves a peer in the user's trust circle and an escrow record on their account — +both permanent, both invisible in Apple's own interfaces — in exchange for no keys at all. + +**What is still not built, and why.** The join is assembled but never sent. Sending it needs the +passcode-authenticated recovery of §6.2–§6.5, which is specified and not yet implemented, and it +writes two permanent artefacts to a real account. Those are worth building deliberately rather than +as the tail of a schema change. The message layer is the part that can be written and tested +without an account, and that is what exists. ## F — smaller, from the same pass From 480f8787f8c74f209c3b3c254cc1ed338ec17286 Mon Sep 17 00:00:00 2001 From: "Shane B." Date: Thu, 13 Aug 2026 19:56:26 +0200 Subject: [PATCH 008/131] Correct the escrow label, and keep unknown-field reporting An escrow record's label and its bottleID are different values: the label is com.apple.icdp.record. and identifies the record, while bottleID is a UUID identifying the bottle inside it. The escrow proxy takes the label everywhere, and the label is what joins against a Cuttlefish bottle's id. Stage 3 7.1 said deletion addresses . That was wrong -- deleting by bottleID addresses nothing. Also documents that label is overloaded: a record class for get_records and get_club_cert, one specific record for srp_init, recover, enroll and delete. Separately, the README now asks implementations to report unmodelled fields permanently rather than removing the diagnostic once the known fields are covered. It has already found two gaps, one of which was a correctness bug rather than an omission: the paging loop stopped on an empty page, which coincided with completion only by luck. Co-Authored-By: Claude Opus 5 --- docs/findmy-export/03-keychain-trust.md | 38 +++++++++++++++----- docs/findmy-export/GAPS.md | 48 ++++++++++++++++--------- docs/findmy-export/README.md | 18 ++++++++++ 3 files changed, 79 insertions(+), 25 deletions(-) diff --git a/docs/findmy-export/03-keychain-trust.md b/docs/findmy-export/03-keychain-trust.md index 7093c525..d04b91ea 100644 --- a/docs/findmy-export/03-keychain-trust.md +++ b/docs/findmy-export/03-keychain-trust.md @@ -260,12 +260,21 @@ A dictionary. Fields vary by command; the ones common to all: and, per command: `blob`, `blobDigest`, `metadata`, `dsid`, `silentAttempt`, `baseRootCertVersions`, `trustedRootCertVersions`. -Two labels appear: +**`label` is overloaded**, and which meaning applies depends on the command: -| Label | Used for | +| Command | `label` means | | --- | --- | -| `com.apple.securebackup.record` | escrow records themselves — the label for `get_records` | -| `com.apple.icdp.record` | the club certificate, when enrolling | +| `get_records` | a **record class** — `com.apple.securebackup.record` | +| `get_club_cert` | a record class — `com.apple.icdp.record` | +| `srp_init`, `recover`, `enroll`, `delete` | **one specific record**, identified individually | + +A specific record's label has the form `com.apple.icdp.record.` — the class, then the +identifier of the peer the record belongs to. + +> **A record's label is not its `bottleID`, and the two must not be substituted for each other.** +> The label identifies the *escrow record*; `bottleID`, found inside the record's metadata, is a +> separate **UUID** identifying the bottle. They are different strings with different shapes. +> Everything addressed to the escrow proxy takes the **label**. > **`userActionLabel` is a free-text string that Apple keeps.** The reference sends descriptions > of the operation being performed. It is not authentication and nothing checks it, but it is @@ -285,9 +294,15 @@ an array of: | Key | Meaning | | --- | --- | -| `label` | **the bottle identifier** — the join key | +| `label` | **the record's identifier, and the join key** — see below | | `metadata` | base64 of an XML plist: the record's descriptive fields | +> **Join on `label`, not on `bottleID`.** The escrow proxy's `label` is the same value as the +> Cuttlefish bottle's `id` (§2.3), and that pairing is what step 3 matches. The `bottleID` inside +> the decoded metadata is a **different** value — a UUID for the bottle itself — and nothing +> requires it to equal the label. Reporting a mismatch between them is reasonable +> diagnostics; treating either as a substitute for the other is not. + **Step 2 — viable bottles.** Cuttlefish `fetchViableBottles`, with a filter value of `1` and an empty metrics list. The response carries a list of valid bottles, each with an id. @@ -375,8 +390,9 @@ big-endian length followed by that many bytes. ### 6.2 Begin the exchange -Send `srp_init` (§4.3) with `label` set to the chosen record's `bottleID`, and `blob` set to the -base64 of the SRP client's public value **A**. +Send `srp_init` (§4.3) with `label` set to the chosen record's **label** — its own identifier from +the `metadataList` entry, *not* its `bottleID` — and `blob` set to the base64 of the SRP client's +public value **A**. SRP parameters match [Stage 1 §4.1](./01-authentication.md): the RFC 5054 2048-bit group, SHA-256, and a 32-byte client secret. @@ -711,8 +727,12 @@ Deletion is **two calls**, both `delete`, in this order: | Order | `label` | | --- | --- | -| 1 | `.double` — a companion record | -| 2 | `` | +| 1 | `