Skip to content

http proxy: raise readiness budget for node cold-start + throttle poll log#414

Merged
sisuresh merged 4 commits into
stellar:mainfrom
Jonathan-Eid:jonathan/proxy-ready-budget
Jul 20, 2026
Merged

http proxy: raise readiness budget for node cold-start + throttle poll log#414
sisuresh merged 4 commits into
stellar:mainfrom
Jonathan-Eid:jonathan/proxy-ready-budget

Conversation

@Jonathan-Eid

@Jonathan-Eid Jonathan-Eid commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

What

  • Raise the HTTP proxy readiness budget in waitProxyReady from 60 attempts (~120s) to 300 (~10 min).

Why

  • proxy pods switched to karpenter, but now have to wait for karpenter cold start times

Testing

  • dotnet build -c Release clean; fantomas --check --recurse src clean.
  • Definitive live test on Core/stellar-supercluster-test with SUPERCLUSTER_PR=<this> + --require-node-labels=purpose:largetests --tolerate-node-taints=largetests, forcing a karpenter cold-start, confirming the proxy becomes ready within the new budget and the mission succeeds (see PR comment).

Known limitations / follow-up (infra, not this PR)

  • The underlying aws-cni failed to assign an IP address on largetests nodes is IP/ENI exhaustion when many pods land on a freshly-scaled node — a VPC-CNI / karpenter-nodeclass concern (prefix delegation / max-pods / instance ENI capacity). This PR tolerates the delay; the CNI capacity itself is worth a separate infra fix.

Issue

🤖 Generated with Claude Code

waitProxyReady allowed only 60 attempts x 2s (~120s). Now the proxy is pinned
to the mission node pool (largetests/catchup), first-ready can take minutes
while karpenter cold-provisions a node + the CNI assigns an IP, so the 120s
budget expires before ready -> 'not ready after 60 attempts' -> formation build
fails (seen across prod #1867/#1868). Raise to 300 attempts (~10 min); the
mission timeout still bounds a genuinely stuck proxy. Also throttle the
'N ready' log to ~once/min instead of every 2s (was ~300 lines/run).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 20, 2026 20:30

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Extends HTTP proxy readiness handling for cold-started mission nodes while reducing poll-log noise.

Changes:

  • Raises readiness budget to approximately 10 minutes.
  • Logs readiness roughly once per minute and on success.
  • Improves timeout diagnostics.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/FSLibrary/StellarSupercluster.fs Outdated
Jonathan-Eid and others added 3 commits July 20, 2026 16:40
Cold-start on a dedicated node pool (karpenter provision + CNI IP
assignment) can exceed 10 min under contention; raise the cap to
450 attempts (~15 min). Still finite so a genuinely stuck proxy
fails with a clear error well inside the mission timeout.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Start at 1 so the loop runs exactly proxyReadyMaxAttempts (300) polls
and the log reads attempt 1/300..300/300 instead of 0/300..300/300
(301 polls). Keep an initial log line via n = 1.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@sisuresh
sisuresh merged commit 79e714a into stellar:main Jul 20, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants