Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
# Handshake transport audit

[Report](report.md) · [Artifacts](artifacts.md)

The table lists the actual handshake task, identified by its request ID. In ON trials, either the handshake GET or the concurrent early POST may create the fresh connection; the other request can reuse it within that trial.

The early POST HTTP status comes from the saved Xcode response metadata for `AID=-1`. HTTP 400 confirms rejection of that POST, but does not identify whether soft stickiness failed, requests arrived out of order, or another server-side condition caused it. OFF has no early POST.

A fresh HTTP/2 connection was verified for each trial's first transport request, not inferred from creating a new service or process. In ON trials, the early POST could create the connection first and the handshake GET could reuse it. DNS, radio, and server state may remain warm. Repeated response-metadata callbacks for one request are counted once.

Connection setup is `connectEndDate - connectStartDate` and includes the TLS handshake. TLS handshake is `secureConnectionEndDate - secureConnectionStartDate`, a portion of connection setup; do not add the two values.

| Scenario | Pair | Setting | Test status | Early POST HTTP | Cold source | Handshake protocol | Handshake reused | Connection setup (incl. TLS, ms) | TLS handshake (ms) | Remote address |
|---|---:|:---:|---|:---:|---|---|:---:|---:|---:|---|
| omnient | 1 | OFF | completed | — | handshake | h2 | false | 27.000 | 17.000 | 2404:6800:400b:c015::451 |
| omnient | 1 | ON | completed | 200 | handshake | h2 | false | 84.000 | 14.000 | 2404:6800:400b:c015::451 |
| omnient | 2 | OFF | completed | — | handshake | h2 | false | 25.000 | 16.000 | 2404:6800:400b:c015::451 |
| omnient | 2 | ON | failed | 200 | handshake | h2 | false | 24.000 | 14.000 | 2404:6800:400b:c015::451 |
| omnient | 3 | OFF | completed | — | handshake | h2 | false | 25.000 | 15.000 | 2404:6800:400b:c015::451 |
| omnient | 3 | ON | completed | 200 | handshake | h2 | false | 19.000 | 12.000 | 2404:6800:400b:c015::451 |
| omnient | 4 | OFF | completed | — | handshake | h2 | false | 33.000 | 14.000 | 2404:6800:400b:c015::451 |
| omnient | 4 | ON | completed | 200 | handshake | h2 | false | 27.000 | 14.000 | 2404:6800:400b:c015::451 |
| omnient | 5 | OFF | completed | — | handshake | h2 | false | 27.000 | 17.000 | 2404:6800:400b:c015::451 |
| omnient | 5 | ON | completed | 200 | handshake | h2 | false | 34.000 | 23.000 | 2404:6800:400b:c015::451 |
| omnient | 6 | OFF | completed | — | handshake | h2 | false | 19.000 | 13.000 | 2404:6800:400b:c015::451 |
| omnient | 6 | ON | completed | 200 | handshake | h2 | false | 21.000 | 15.000 | 2404:6800:400b:c015::451 |
| omnient | 7 | OFF | completed | — | handshake | h2 | false | 28.000 | 18.000 | 2404:6800:400b:c015::451 |
| omnient | 7 | ON | completed | 200 | handshake | h2 | false | 55.000 | 13.000 | 2404:6800:400b:c015::451 |
| omnient | 8 | OFF | completed | — | handshake | h2 | false | 61.000 | 17.000 | 2404:6800:400b:c015::451 |
| omnient | 8 | ON | timeout | 400 | handshake | h2 | false | 62.000 | 17.000 | 2404:6800:400b:c015::451 |
| omnient | 9 | OFF | completed | — | handshake | h2 | false | 24.000 | 15.000 | 2404:6800:400b:c015::451 |
| omnient | 9 | ON | completed | 200 | handshake | h2 | false | 29.000 | 19.000 | 2404:6800:400b:c015::451 |
| omnient | 10 | OFF | completed | — | handshake | h2 | false | 27.000 | 17.000 | 2404:6800:400b:c015::451 |
| omnient | 10 | ON | completed | 200 | handshake | h2 | false | 32.000 | 18.000 | 2404:6800:400b:c015::451 |
| viewfinder | 1 | OFF | failed | — | handshake | h2 | false | 28.000 | 17.000 | 2404:6800:400b:c015::451 |
| viewfinder | 1 | ON | completed | 200 | handshake | h2 | false | 33.000 | 14.000 | 2404:6800:400b:c015::451 |
| viewfinder | 2 | OFF | completed | — | handshake | h2 | false | 24.000 | 12.000 | 2404:6800:400b:c015::451 |
| viewfinder | 2 | ON | completed | 200 | handshake | h2 | false | 26.000 | 15.000 | 2404:6800:400b:c015::451 |
| viewfinder | 3 | OFF | completed | — | handshake | h2 | false | 23.000 | 16.000 | 2404:6800:400b:c015::451 |
| viewfinder | 3 | ON | completed | 400 | handshake | h2 | false | 26.000 | 16.000 | 2404:6800:400b:c015::451 |
| viewfinder | 4 | OFF | failed | — | handshake | h2 | false | 28.000 | 15.000 | 2404:6800:400b:c015::451 |
| viewfinder | 4 | ON | completed | 400 | handshake | h2 | false | 57.000 | 16.000 | 2404:6800:400b:c015::451 |
| viewfinder | 5 | OFF | completed | — | handshake | h2 | false | 22.000 | 16.000 | 2404:6800:400b:c015::451 |
| viewfinder | 5 | ON | completed | 200 | handshake | h2 | false | 40.000 | 15.000 | 2404:6800:400b:c015::451 |
| viewfinder | 6 | OFF | completed | — | handshake | h2 | false | 21.000 | 12.000 | 2404:6800:400b:c015::451 |
| viewfinder | 6 | ON | completed | 200 | handshake | h2 | false | 51.000 | 13.000 | 2404:6800:400b:c015::451 |
| viewfinder | 7 | OFF | completed | — | handshake | h2 | false | 31.000 | 13.000 | 2404:6800:400b:c015::451 |
| viewfinder | 7 | ON | completed | 400 | handshake | h2 | false | 22.000 | 13.000 | 2404:6800:400b:c015::451 |
| viewfinder | 8 | OFF | completed | — | handshake | h2 | false | 55.000 | 17.000 | 2404:6800:400b:c015::451 |
| viewfinder | 8 | ON | failed | 400 | handshake | h2 | false | 303.000 | 293.000 | 2404:6800:400b:c015::451 |
| viewfinder | 9 | OFF | completed | — | handshake | h2 | false | 30.000 | 14.000 | 2404:6800:400b:c015::451 |
| viewfinder | 9 | ON | completed | 200 | handshake | h2 | false | 26.000 | 14.000 | 2404:6800:400b:c015::451 |
| viewfinder | 10 | OFF | completed | — | handshake | h2 | false | 25.000 | 14.000 | 2404:6800:400b:c015::451 |
| viewfinder | 10 | ON | completed | 400 | handshake | h2 | false | 24.000 | 13.000 | 2404:6800:400b:c015::451 |

## M1 and M3 dispatch

The handshake payload column is the request's encoded `count`. `count=0` means neither M1 nor M3 was in the handshake request itself. `early POST` means the first request containing that message was an `AID=-1` POST logged before WebChannel opened, concurrent with the handshake; it does not prove server arrival order or acceptance. `post-open POST` means its first request was logged after the channel opened.

M1 is message offset 0. The demo sends M3 at offset 1 in Omnient and offset 2 in Viewfinder (after M2 Heartbeat). Request body prefixes expose `count` and `ofs`, which identify the first HTTP request containing each queued message. A rejected early POST may be retried after opening; the table records the first dispatch.

The observed M1 delay is measured from the client connect timestamp to the M1 `send()` call; scheduling can vary slightly from the requested delay.

| Scenario | Pair | Setting | M1 after connect (ms) | Handshake payload | M1 first request | M3 first request |
|---|---:|:---:|---:|:---:|---|---|
| omnient | 1 | OFF | 50.304 | count=0 | post-open POST | post-open POST |
| omnient | 1 | ON | 52.865 | count=0 | early POST | early POST |
| omnient | 2 | OFF | 52.478 | count=0 | post-open POST | post-open POST |
| omnient | 2 | ON | 51.285 | count=0 | early POST | early POST |
| omnient | 3 | OFF | 51.087 | count=0 | post-open POST | post-open POST |
| omnient | 3 | ON | 52.562 | count=0 | early POST | early POST |
| omnient | 4 | OFF | 51.808 | count=0 | post-open POST | post-open POST |
| omnient | 4 | ON | 50.489 | count=0 | early POST | early POST |
| omnient | 5 | OFF | 50.592 | count=0 | post-open POST | post-open POST |
| omnient | 5 | ON | 51.444 | count=0 | early POST | early POST |
| omnient | 6 | OFF | 50.449 | count=0 | post-open POST | post-open POST |
| omnient | 6 | ON | 51.799 | count=0 | early POST | early POST |
| omnient | 7 | OFF | 51.580 | count=0 | post-open POST | post-open POST |
| omnient | 7 | ON | 52.847 | count=0 | early POST | early POST |
| omnient | 8 | OFF | 52.240 | count=0 | post-open POST | post-open POST |
| omnient | 8 | ON | 52.907 | count=0 | early POST | early POST |
| omnient | 9 | OFF | 51.205 | count=0 | post-open POST | post-open POST |
| omnient | 9 | ON | 52.866 | count=0 | early POST | early POST |
| omnient | 10 | OFF | 52.473 | count=0 | post-open POST | post-open POST |
| omnient | 10 | ON | 50.740 | count=0 | early POST | early POST |
| viewfinder | 1 | OFF | 52.201 | count=0 | post-open POST | post-open POST |
| viewfinder | 1 | ON | 50.372 | count=0 | early POST | post-open POST |
| viewfinder | 2 | OFF | 51.271 | count=0 | post-open POST | post-open POST |
| viewfinder | 2 | ON | 50.958 | count=0 | early POST | post-open POST |
| viewfinder | 3 | OFF | 50.601 | count=0 | post-open POST | post-open POST |
| viewfinder | 3 | ON | 51.225 | count=0 | early POST | post-open POST |
| viewfinder | 4 | OFF | 51.768 | count=0 | post-open POST | post-open POST |
| viewfinder | 4 | ON | 52.828 | count=0 | early POST | post-open POST |
| viewfinder | 5 | OFF | 52.048 | count=0 | post-open POST | post-open POST |
| viewfinder | 5 | ON | 51.827 | count=0 | early POST | post-open POST |
| viewfinder | 6 | OFF | 51.553 | count=0 | post-open POST | post-open POST |
| viewfinder | 6 | ON | 52.863 | count=0 | early POST | post-open POST |
| viewfinder | 7 | OFF | 51.563 | count=0 | post-open POST | post-open POST |
| viewfinder | 7 | ON | 51.818 | count=0 | early POST | post-open POST |
| viewfinder | 8 | OFF | 50.862 | count=0 | post-open POST | post-open POST |
| viewfinder | 8 | ON | 50.518 | count=0 | early POST | post-open POST |
| viewfinder | 9 | OFF | 52.851 | count=0 | post-open POST | post-open POST |
| viewfinder | 9 | ON | 52.908 | count=0 | early POST | post-open POST |
| viewfinder | 10 | OFF | 51.151 | count=0 | post-open POST | post-open POST |
| viewfinder | 10 | ON | 50.377 | count=0 | early POST | post-open POST |

## Observations and limits

Omnient sends M3 during connection establishment; Viewfinder schedules M3 about 500 ms later, after the channel opened in every completed Viewfinder trial. Fresh connection verification is to the reached edge, not a cold DNS lookup or cold backend application state.

The initial early POST returned HTTP 400 in 6/20 ON trials. Of those trials, 4 completed; inspect the logs to distinguish successful retries from incomplete runs.

The Xcode logs contain retry diagnostics in ON trials for Omnient pairs 2, 8 and Viewfinder pairs 3, 4, 7, 8, 10. Those retries can affect TTFA and TTFD; the raw logs should be consulted before attributing a timing difference solely to the handshake option.

Incomplete trials: omnient pair 2 ON; omnient pair 8 ON; viewfinder pair 1 OFF (Xcode log includes network connection lost, -1005); viewfinder pair 4 OFF; viewfinder pair 8 ON (Xcode log includes network connection lost, -1005).

Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Lens benchmark — 2026-10-08 physical-device cold-connection run (binary encoding)

## Experiment specification

[Google net_http: Lens Latency Mimic & Shutter Benchmark Mode](https://github.com/google/net_http/blob/main/webchannel/objc/demo/WebChanneliOSDemo/DESIGN.md#d-lens-latency-mimic--shutter-benchmark-mode).

## Environment

| Field | Recorded value |
|---|---|
| Device | iPhone SE (3rd generation), hardware model `iPhone14,6` |
| iOS | 26.3.1, from XCTest |
| Network switches | Not independently recorded for this run |
| Endpoint | `https://webchannel.sandbox.google.com/staging/channel/lens` |
| Mac | macOS 26.5.1; Xcode 26.6; Build version 17F113 |
| Source | Git HEAD `aa71e660c4d409919bec313fccc9d1376baa103c` plus uncommitted benchmark instrumentation |
| Test period | 2026-10-08 15:06–15:11 local time, from artifact timestamps |
| Test execution | 10 alternating OFF/ON pairs per scenario; a separate `xcodebuild test-without-building` invocation for each trial |
| Test result | 35 completed trials; 40 passed fresh-transport verification |
| Transport requirement | First transport: `h2`, a new TCP/TLS connection on the handshake GET/POST or early POST; if the GET reused the early POST connection, matching address and local port |

## Procedure and parameters

1. Build the XCTest bundle once, then launch exactly one cold-trial test method per `xcodebuild test-without-building` invocation. Keep parallel testing disabled and retain each `.xcresult` bundle.
2. Run Omnient first, then Camera Viewfinder. Each pair runs `fastHandshake2=false` (OFF), then `fastHandshake2=true` (ON). The XCTest creates a new `WebChannelService` and WebChannel client per trial.
3. Use `imageSizeKb=150`, `detectionDelayMs=50`, `initialMessageDelayMs=50`, `enableBinaryEncoding=true`, `sendRawJson=true`, `fastHandshake=false`, `nonBlockingSend=false`, `blockingHandshake=false`, `forceLongPolling=false`, and `detectBufferingProxy=false`. The 150 KB value is metadata; no 150 KB image is uploaded.
4. Schedule M1 Sticky Cluster Info 50 ms after `connect()` returns. Omnient sends M3 Prefetch immediately after M1; Camera Viewfinder schedules M2 Heartbeat at +200 ms and M3 Prefetch at +500 ms after `connect()` returns. M4 Final Capture follows preliminary detections. Each trial has a 60-second completion deadline.
5. Record Handshake (`webChannelOpened` minus connect), TTFA (M3 send to stream ACK), TTFD (M3 send to preliminary detections), and M4 RTT (M4 send to interaction response).
6. Include a pair in the raw measurement tables only if both runs completed, both trials meet the transport requirement, and the absolute ON/OFF handshake-duration difference is ≤20 ms. Calculate that difference from full-precision values. Keep every trial in the raw attachments and audit.

## Omnient: valid pair raw measurements

Values shown in the table are rounded to three decimal places.

| Run # | fastHandshake2 | Handshake (ms) | TTFA (ms) | TTFD (ms) | M4 RTT (ms) |
|---:|:---:|---:|---:|---:|---:|
| 3 | OFF | 183.940 | 274.629 | 321.384 | 135.920 |
| 3 | ON | 179.153 | 150.608 | 197.459 | 155.047 |
| 4 | OFF | 188.329 | 301.885 | 322.818 | 140.845 |
| 4 | ON | 193.633 | 517.116 | 540.622 | 145.079 |
| 6 | OFF | 166.148 | 267.794 | 317.084 | 149.240 |
| 6 | ON | 164.204 | 206.971 | 255.013 | 161.305 |

## Camera Viewfinder: valid pair raw measurements

Values shown in the table are rounded to three decimal places.

| Run # | fastHandshake2 | Handshake (ms) | TTFA (ms) | TTFD (ms) | M4 RTT (ms) |
|---:|:---:|---:|---:|---:|---:|
| 2 | OFF | 173.476 | 155.322 | 209.601 | 169.366 |
| 2 | ON | 162.429 | 135.548 | 184.422 | 140.504 |
| 5 | OFF | 173.993 | 139.603 | 190.365 | 147.950 |
| 5 | ON | 191.875 | 145.243 | 195.553 | 139.722 |
| 9 | OFF | 165.460 | 137.316 | 187.138 | 143.419 |
| 9 | ON | 180.641 | 478.588 | 478.829 | 134.641 |
2 changes: 1 addition & 1 deletion webchannel/benchmark/lens/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ When changing or generating files under this directory, follow [README.md](READM
- Write reports in English. Each run's `report.md` must link the specification and record the physical-device environment, network switch states when known, experiment procedure, parameter values, and separate Omnient and Camera Viewfinder raw measurement tables with `Run #` as the first column.
- Run 10 OFF/ON pairs per scenario unless the user changes the count. State the inclusion rule: both runs completed and their full-precision handshake durations differ by at most 10 ms by default, or the run-specific threshold explicitly requested by the user. For cold-connection runs, also require verified fresh HTTP/2 transport for both trials. Apply any explicitly requested run-specific outlier omission without changing the raw files.
- Do not put median or other summary statistics, comparisons, interpretation, an excluded-pair list, or artifact links in `report.md`. Keep the table values rounded to three decimals and preserve every original measurement in raw artifacts.
- Put per-trial transport evidence, early POST HTTP status, and available evidence of whether M1/M3 were in the handshake request, a concurrent early POST, or a later POST in `audit.md`. Keep analysis of transport behavior and limitations there, separate from `report.md`.
- Put per-trial transport evidence, early POST HTTP status, and available evidence of whether M1/M3 were in the handshake request, a concurrent early POST, or a later POST in `connection-reuse-audit.md`. Keep analysis of transport behavior and limitations there, separate from `report.md`.
- Put descriptions and direct links to raw CSV, JSON, logs, and available result bundles in `artifacts.md`. A result bundle need not be checked in if its exported evidence is retained.
- For new cold-connection runs, store per-trial Xcode and attachment-export logs in `logs/`, result bundles in `xcresults/`, and exported attachments in `attachments/`. Preserve earlier run layouts and keep the summarizer compatible with both.
- Describe an early POST HTTP 400 as a rejection. Client logs alone do not establish whether soft stickiness failed or requests arrived out of order.
Expand Down
Loading
Loading