Meet the remaining TAPI HTTP agreements: Authorization header, 511, generic Retry-After (529) - #148
Open
MichaelGHSeg wants to merge 6 commits into
Open
MichaelGHSeg wants to merge 6 commits into
MichaelGHSeg wants to merge 6 commits into
Conversation
Route any retryable response carrying a valid Retry-After header through the rate-limit path (no retry-budget cost) instead of special-casing 429. Retryable statuses without Retry-After continue to use counted exponential backoff. Adds 529 to the retryable set and covers both paths with tests. Matches the behaviour already shipped in analytics-java 3.5.5 and the generic-retry-after conformance suite in sdk-e2e-tests.
2 tasks
Routing any retryable status with Retry-After to the rate-limit path left
ShouldDeleteBatch inconsistent with HandleResponse. With rate limiting on and
backoff off, a 503 or 529 carrying Retry-After would rate-limit the pipeline
(WaitUntilTime set, uploads blocked) while ShouldDeleteBatch still reported
true, so the batch file was deleted and the pipeline then stalled waiting to
retry events that no longer existed.
That configuration is reachable from CDN settings and directly from
Configuration.HttpConfig — it is the config ConfigurationHttpConfigTest builds.
ShouldDeleteBatch now keeps a retryable batch whenever rate limiting is
enabled, matching swift's shouldDropBatch ("Rate limit config handles retryable
codes that carry Retry-After — don't drop"). Non-retryable statuses are still
dropped, and a retryable status with neither rate limiting nor backoff enabled
is still dropped since nothing would retry it.
The previous commit kept a retryable batch whenever rate limiting was enabled, which was too broad: a 500 with no Retry-After and backoff disabled was also kept, so the file was re-uploaded even though nothing had scheduled a retry. The sdk-e2e-tests "backoffConfig.enabled: false" case caught this — it expects exactly one request and saw two. ShouldDeleteBatch now takes the same retryAfterSeconds value handed to HandleResponse, so the two agree on whether the response actually took the rate-limit path. A retryable status keeps its batch only when it carries a usable Retry-After and rate limiting is on; otherwise only backoff can retry it, and with backoff off the batch is dropped as before. The single-argument overload is retained. 232 tests pass.
Analytics-CSharp-plan.md states 'Spec item 1: 2xx and 3xx are success', but IsSuccessStatusCode and the two status checks in RetryStateMachine were 2xx-only, so a 3xx fell through to the retry classifier. analytics-go, analytics-python and analytics-php already follow the spec here; this brings C# into line with them and with its own plan. 236 tests pass, including new cases covering 200, 201, 301 and 304.
Two Key Agreements from the HTTP response design doc that this SDK did not meet.
The doc requires every SDK to send the write key in the Authorization header,
and TAPI authenticates and routes on it instead of parsing the payload — which
is the performance reason the header exists. This SDK sent no Authorization at
all; it relied solely on the writeKey embedded in the batch body by Storage.
Upload requests now carry Basic credentials built from the write key with an
empty password, matching analytics-python, -go, -ruby, -php and -java, all of
which send base64("<writeKey>:").
The value is exposed as a protected BasicAuthorization on HTTPClient rather
than by widening _apiKey, so a custom IHTTPClientProvider can send the same
header; the Unity sample, which overrides DoPost, now does. The writeKey stays
in the payload, so nothing depends on the header alone yet.
Separately, 511 Network Authentication Required was retryable here. The doc
makes it conditional — "Authenticate, then retry if library supports OAuth" —
and this SDK has no OAuth, so a 511 could never be satisfied and retrying only
spent the budget. It joins 501 and 505 as an explicit Drop. analytics-python,
the one SDK with OAuth, correctly retries 511 only when an OauthManager is
configured; go, ruby, php and java exclude it as this now does.
240 tests pass, including new coverage of the header value, and all 79 e2e
tests still pass.
The header assertion in sdk-e2e-tests is opt-in per SDK, since analytics-kotlin and analytics-swift do not send it yet. This SDK does, so it runs the check.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #147 — review that first. This PR has grown past its original scope: it now closes the three remaining TAPI Key Agreements this SDK did not meet, not just 529.
Authorization header — the headline change
The design doc's first Key Agreement is that every SDK sends the write key in the
Authorizationheader. TAPI authenticates and routes on that header rather than parsing the payload, which is the server-side performance reason the header exists.This SDK sent no
Authorizationat all. It relied solely on thewriteKeythatStorageembeds in the batch body, so it was giving up the benefit the whole initiative was for.Upload requests now carry
Basic base64("<writeKey>:")— an empty password, matching analytics-python, -go, -ruby, -php and -java. The value is exposed as aprotected HTTPClient.BasicAuthorizationrather than by widening_apiKey, so a customIHTTPClientProvidercan send it too; the Unity sample, which overridesDoPost, does. ThewriteKeystays in the payload, so nothing depends on the header alone yet.511 Network Authentication Required
The doc makes 511 conditional — "Authenticate, then retry if library supports OAuth." This SDK has no OAuth, so a 511 could never be satisfied and retrying only spent the budget. It joins 501 and 505 as an explicit
Drop. (analytics-python is the only SDK with OAuth and correctly retries 511 only when anOauthManageris configured.)Generic Retry-After, including 529
Any retryable response carrying a valid
Retry-Afterroutes through the rate-limit path, which does not consume retry budget; retryable statuses without the header continue on counted exponential backoff. AddsRetryAfterParserhandling delta-seconds and RFC 1123 HTTP-date (null for past dates). Matches analytics-java 3.5.5 and thegeneric-retry-afterconformance suite.Two follow-up commits fix a data-loss bug this introduced: routing a retryable status to the rate-limit path left
ShouldDeleteBatchinconsistent withHandleResponse, so with rate limiting on and backoff off a 503/529 carryingRetry-Afterwould rate-limit the pipeline and delete the batch, then stall waiting to retry events that no longer existed.ShouldDeleteBatchnow takes the sameretryAfterSecondsvalue asHandleResponseso the two agree by construction. The first attempt at that fix was too broad and thebackoffConfig.enabled: falsee2e case caught it.3xx as success
Analytics-CSharp-plan.mdstates "Spec item 1: 2xx and 3xx are success", butIsSuccessStatusCodeand theRetryStateMachinechecks were 2xx-only. Brought in line with the plan and with analytics-go, -python and -php.Testing
retry-settingssuites.Authorizationassertion is new (sdk-e2e-testsbranchtest/header-agreements) and is what caught the missing header here. It is opt-in per SDK via"AUTH_HEADER": "true", because analytics-kotlin and analytics-swift deliberately do not send it yet pending the v2 endpoint discussion. This SDK opts in.E2E was run locally because CI cannot check out the private
sdk-e2e-testsrepo — thetoken: ${{ secrets.E2E_TESTS_TOKEN }}line was dropped from that checkout during CI hardening, and the same omission is present in this repo'sci/harden-build-publishbranch and in go/ruby/php.Before release
Analytics-CSharp.csprojstill says<Version>2.6.0</Version>, which is already published (2025-12-04). It needs a bump.