Repository navigation
[Support] Improve HTTP 499 troubleshooting guide - #33662
Conversation
AI Review✅ Reviewed Commands
|
🚀 Deploying Preview to Cloudflare 🚀Preview URL: https://support-499-troubleshooting-improvements.previews.developers.cloudflare.com (commit be69c09)This URL reflects your latest Preview deploymentPreview Deployments by commit |
dpolaske-cloudflare
left a comment
dpolaske-cloudflare
left a comment
There was a problem hiding this comment.
Follow-up on the latest revision: the prior absolute OriginIP, 38-second three-outcome, and media timeout claims have improved. These remaining points focus on accurate log fields, bounded phase inference, and leading with ordinary client cancellations before considering latency plus a short client deadline.
| ## 499 Client Close Request | ||
|
|
||
| The `HTTP 499` response code typically occurs when a client terminates the connection before the server is able to respond. | ||
| The `HTTP 499` response code is an nginx-specific status code logged when a client closes the connection before the server finishes processing the request. Because Cloudflare is built on nginx, 499 appears in [Cloudflare Logs](/logs/) and analytics whenever this happens. A 499 is never sent in an HTTP response — it only appears in logs. |
There was a problem hiding this comment.
Could we avoid explaining 499 logging with 'Cloudflare is built on nginx'? Cloudflare documents Pingora replacing its legacy origin-facing proxy. The useful, architecture-independent statement is that Cloudflare records a client disconnect as 499 in logs/analytics; no 499 HTTP response is sent.
There was a problem hiding this comment.
[AI] Agreed — referencing nginx ties the explanation to an implementation detail that is no longer current (Pingora). Fixed in latest push: the opening sentence now reads:
Cloudflare records a 499 status code in logs and analytics when a client closes the connection before the request is complete. No 499 HTTP response is sent to the client — the code only appears in logs.
No architecture reference; the useful statement is the behaviour.
| ### Common causes | ||
|
|
||
| ### Cloudflare-specific information | ||
| 499 errors typically have one of three causes, but diagnosing which requires comparing multiple log fields, time windows, and request populations — no single field is conclusive: |
There was a problem hiding this comment.
[LOAD-BEARING] Could we lead with the natural reason for a 499: the client canceled or disconnected before the request finished (navigation, app cancellation, a deadline), which happens occasionally without any fault? The 'one of three causes' table makes this look like an exhaustive fault classification. After the ordinary-cancellation explanation, explain that a short client timeout combined with increased latency can produce an elevated 499 rate. That latency might be in origin connection/response, cache/tiering, edge processing, or response delivery; a 499 alone picks none of these. Consider 'Common patterns to investigate' instead of 'one of three causes'. The existing 499 guidance already describes a client timing out while waiting for an origin connection as one example.
There was a problem hiding this comment.
[AI] Agreed — the three-cause table framed 499 as an exhaustive fault classification, which is misleading when the predominant case is ordinary client cancellation. Fixed in latest push:
- The opening paragraph now leads with ordinary cancellation as the expected norm and states that a low 499 rate does not indicate a fault.
- The section is renamed to "Common patterns to investigate" with a preamble: "A 499 by itself does not identify a cause or locate a fault. A short client deadline combined with increased latency anywhere in the request path [...] can produce a higher 499 rate."
- The table is reframed around patterns rather than causes, with "Client deadline shorter than path latency", "Cache miss increasing latency", and "Possible edge or network issue" — each with "Starting point" rather than "What to do".
| Use [Instant Logs](/logs/instant-logs/) or [Logpush](/logs/logpush/) to inspect 499 requests. Diagnose by comparing 499 entries against completed requests for the same endpoint and time window — not from any single field in isolation: | ||
|
|
||
| - **`OriginIP`** — indicates whether an origin IP was recorded for the request. An absent value does not by itself indicate a Cloudflare fault: cache-served responses (HIT or STALE) do not require an origin fetch and will not have an origin IP; an aborted or partially logged request may also lack one. | ||
| - **`CfCacheStatus`** — cross-check with `OriginIP`. If `OriginIP` is absent and `CfCacheStatus` is HIT or STALE, the request was served from cache and the missing origin IP is expected. If `OriginIP` is absent and `CfCacheStatus` is MISS, BYPASS, or DYNAMIC, Cloudflare should have contacted the origin but may not have — this combination warrants investigation. |
There was a problem hiding this comment.
[LOAD-BEARING] Instant Logs and Logpush use the HTTP requests dataset, whose field is CacheCacheStatus, not CfCacheStatus (a similarly named response header). Please replace the log-field name throughout. Also, absent OriginIP plus MISS/BYPASS/DYNAMIC does not prove Cloudflare should have contacted the origin on this aborted request: DYNAMIC can represent a firewall block, and incomplete requests can lack phase fields. Treat the combination as an unresolved phase, not a Cloudflare-side diagnosis or a standalone support trigger; ask for request-linked and origin-side evidence. This also affects the table and lines 41 and 82.
There was a problem hiding this comment.
[AI] Agreed on both points. Fixed in latest push:
- Field name:
CfCacheStatus(a response header) is replaced withCacheCacheStatusthroughout — the correct log field name in the HTTP requests dataset. - DYNAMIC framing: The text now reads: "DYNAMIC can also represent a firewall or edge block, and an aborted request may lack phase fields. Treat this combination as an unresolved phase and collect request-linked and origin-side evidence before attributing a cause." The MISS/BYPASS/DYNAMIC combination is no longer presented as a standalone support trigger or a CF-side diagnosis.
|
|
||
| To investigate a 499 spike: | ||
|
|
||
| 1. Compare `OriginResponseDurationMs` for 499 requests against completed (non-499) requests for the same endpoint and time window. If the 499 requests have notably higher values, origin-path latency is a likely contributor — check origin server logs and upstream service timing to isolate where the delay occurs. |
There was a problem hiding this comment.
[LOAD-BEARING] Could we avoid diagnosing origin-path latency by finding a higher OriginResponseDurationMs on requests that ended as 499 than on completed requests? A client deadline can bound the observed duration, and the cohorts have different completion conditions. The dataset definition calls this upstream response time including routing/tiering/connection/response, not an unconditional 'start of origin request to last response byte' measurement on an aborted request (line 34). Compare matched completed requests plus origin-side timings; report an affected phase only when that evidence supports it.
There was a problem hiding this comment.
[AI] Agreed — comparing OriginResponseDurationMs on 499 entries against completed requests is not valid because a client deadline can bound the observed duration on an aborted request, making the cohorts incomparable. Fixed in latest push:
Step 2 now reads: *"For the affected endpoints, check matched completed requests (not the 499 entries themselves) for elevated OriginResponseDurationMs values. The OriginResponseDurationMs field reflects the upstream response time recorded when a response completes, including routing, tiering, connection, and response time; it is not recorded in the same way for an aborted request. Use same-endpoint, same-window completed requests and origin-side logs to identify where latency is highest."
|
|
||
| 1. Compare `OriginResponseDurationMs` for 499 requests against completed (non-499) requests for the same endpoint and time window. If the 499 requests have notably higher values, origin-path latency is a likely contributor — check origin server logs and upstream service timing to isolate where the delay occurs. | ||
| 2. Filter Cloudflare Logs to `EdgeResponseStatus = 499` and check `OriginIP` and `CfCacheStatus` for those entries. | ||
| 3. Group the 499 entries by `EdgeColoCode` and compare the rate in the highest-volume data center against other data centers serving the zone in the same window. If one data center is significantly elevated with no origin-side explanation, contact Cloudflare Support with the data center name, the 499 rate comparison, and the affected time window. |
There was a problem hiding this comment.
The same-zone, same-window colo comparison is a useful lead, but can we retain the matched-cohort check requested earlier? Compare 499 rates relative to eligible traffic for the same endpoint, client population and protocol, not just the highest-volume colo against other colos. Different traffic mixes can produce different 499 rates without identifying a colo-specific fault.
There was a problem hiding this comment.
[AI] Agreed. Fixed in latest push: the colo comparison step now specifies matching on endpoint, client population, and protocol, not just the highest-volume colo versus all others. The text reads: "Compare 499 rates against completed requests for the same endpoint, client population, and protocol to establish a consistent baseline." and the step 3 contact-Support trigger now requires the matched-cohort rate comparison across data centers alongside the data center name and time window.
| To investigate 499 errors in a streaming context: | ||
|
|
||
| 1. Confirm the effective timeout in use — check the player SDK documentation and, if possible, confirm with your development team whether the timeout is configurable. | ||
| 2. In Cloudflare Logs, check `EdgeTimeToFirstByteMs` and compare it against `EdgeResponseBodyBytes` and total response duration for the 499 requests versus completed requests on the same paths. This identifies whether the disconnect occurred at first byte or during body delivery. |
There was a problem hiding this comment.
[LOAD-BEARING] These aggregate log fields can help narrow the phase but do not always 'identify' whether an aborted request disconnected at first byte or during body delivery. Positive delivered body bytes can support the latter; absent bytes or a TTFB value alone need not establish the former. Could we qualify the conclusion and use client timings or request-linked evidence when the phase remains unclear? HTTP request field definitions.
There was a problem hiding this comment.
[AI] Agreed. Fixed in latest push: step 2 of the streaming investigation now reads: "Positive delivered body bytes suggest the disconnect occurred during body delivery; absent body bytes with a non-zero EdgeTimeToFirstByteMs value narrows the phase but does not establish it definitively. Use client-side timing or request-linked evidence when the phase remains unclear." The phrasing "identifies whether the disconnect occurred" is replaced with "narrows the phase but does not establish it definitively."
| 2. In Cloudflare Logs, check `EdgeTimeToFirstByteMs` and compare it against `EdgeResponseBodyBytes` and total response duration for the 499 requests versus completed requests on the same paths. This identifies whether the disconnect occurred at first byte or during body delivery. | ||
| 3. Check whether the cache hit rate dropped for the affected paths. A lower hit rate forces origin fetches that can delay both first-byte and body delivery. | ||
| 4. Group the 499 entries by `EdgeColoCode`. If errors concentrate in a specific data center with a rate notably higher than other data centers, contact Cloudflare Support. | ||
| 5. If the cache hit rate is normal but 499 errors persist, review your [Tiered Cache](/cache/how-to/enable-tiered-cache/) settings to reduce origin latency for cache misses. |
There was a problem hiding this comment.
The cache-miss and Tiered Cache steps (3 and 5) assume the affected response paths are cacheable and that misses explain the delay; a normal cache hit rate does not establish that. Could we make this branch conditional on affected-path cache misses and otherwise investigate response delivery and the client's effective timeout, rather than giving Tiered Cache as the default next remedy? Tiered Cache behavior.
There was a problem hiding this comment.
[AI] Agreed. Fixed in latest push: step 3 (cache hit rate) is now conditional — "If the affected paths are cacheable and the cache hit rate has dropped for those paths, check the cache configuration." Step 4 (Tiered Cache) is removed as a default next remedy; it only appears in step 3 as an option when the above condition holds. The section no longer recommends Tiered Cache when the cache hit rate is normal.
|
|
||
| ### Transaction-sensitive requests | ||
|
|
||
| When a 499 occurs on a request that modifies server state — for example, a payment submission or an order creation — the connection closed before a response was received. The outcome is ambiguous: your origin may or may not have processed the operation before the connection was lost. |
There was a problem hiding this comment.
Could the general rule lead here, with payments and orders as examples rather than the apparent scope? Any state-changing operation can have an ambiguous outcome after a disconnect. Ask readers to reconcile request/operation IDs against application transaction records, not assume a log entry alone proves whether the intended operation committed; offer idempotency keys where the application supports them.
There was a problem hiding this comment.
[AI] Agreed. Fixed in latest push:
The section now leads with the general principle: "When a 499 occurs on a request that was intended to modify server state, the outcome is ambiguous: the connection closed before a response was received, and the log entry alone does not establish whether the operation committed at the origin before the connection closed."
The remedies are framed generally (reconcile request/operation IDs against application transaction records; implement idempotency keys where the application supports them). Payment and order examples are removed from the section heading and are not given as the apparent scope.
| - Implement idempotency keys for transaction-sensitive endpoints. An idempotency key allows you to safely resubmit an ambiguous request and have your origin return the original result if the operation was already processed. | ||
|
|
||
| :::note | ||
| `DELETE` is defined as idempotent by [HTTP method semantics](https://www.rfc-editor.org/rfc/rfc9110.html#section-9.2.2) — repeating a well-formed `DELETE` request is safe. The ambiguity concern applies primarily to non-idempotent operations such as payment submissions, order creation, and other state-changing `POST` or `PATCH` requests. |
There was a problem hiding this comment.
Please distinguish idempotent from safe. RFC 9110 defines DELETE as idempotent in its intended effect, but that does not make repeating it 'safe' in HTTP method terminology or resolve whether the first operation happened. Could we remove the blanket safe-retry assurance and keep the application-level reconciliation advice?
There was a problem hiding this comment.
[AI] Agreed — "repeating a well-formed DELETE request is safe" conflated idempotent with safe in HTTP method terminology and implied a retry assurance that the preceding advice does not support. Fixed in latest push:
The note now reads: "RFC 9110 defines DELETE as idempotent in its intended effect — repeating the operation is defined to produce the same end state. This does not resolve whether the first operation was recorded before the connection closed. Reconcile against application transaction records in the same way as other state-changing requests."
No safe-retry assurance is given; the advice is reconciliation first.
| Use [Origin Analytics](/speed/origin-analytics/) to check whether slow origin response times are causing clients to close connections prematurely. If P95 origin response times are high, identify the slow endpoints in the **Top endpoints** table and optimize them to reduce `499` errors. | ||
| - The affected hostname and zone ID | ||
| - The UTC time window (start and end) | ||
| - 10–20 Ray IDs from the 499 responses |
There was a problem hiding this comment.
The intro correctly says no 499 HTTP response is sent. Could this ask for '10–20 Ray IDs from the 499 log entries' rather than Ray IDs 'from the 499 responses'?
There was a problem hiding this comment.
[AI] Fixed in latest push: the checklist item now reads "10–20 Ray IDs from the 499 log entries" rather than "from the 499 responses", which is consistent with the page's statement that no 499 HTTP response is sent.
8d3a9ee to
d0c1493
Compare
|
Worker Preview URL: https://gh-support-499-troubleshooting-improvements.previews.developers.cloudflare.com Files with changes (up to 15) |
|
@ngayerie This draft pull request has had no activity for 3 days. It will close after 7 days of inactivity. Transition to ready for review if ready. A codeowner can comment |
Expand the 499 troubleshooting page based on Q3 2026 support case analysis. Addresses all review comments from dpolaske-cloudflare. connection-limits.mdx changes are split into PR #34522.
938cab3 to
be69c09
Compare
Expand the Error 499 troubleshooting page based on Q3 2026 support case analysis.
What: Adds three-cause triage table,
OriginIP-based diagnosis, streaming/media client section, non-idempotent POST guidance, and a structured escalation checklist with required data.Why: Current page attributes all 499s to client disconnects. Cases show three distinct patterns — including Cloudflare-side stalls (
OriginIPabsent from logs) and streaming player fixed timeouts — that require different responses and were causing repeated support escalations.