Skip to content

fix(client): concurrent first init_client calls share one attempt - #255

Open
XieX wants to merge 6 commits into
mainfrom
xie/aic-3600-shared-init-attempt
Open

XieX wants to merge 6 commits into
mainfrom
xie/aic-3600-shared-init-attempt

Conversation

@XieX

@XieX XieX commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #254. Retarget to main once #254 merges.

Problem

Since #254, init_client raises when the client it built has not initialized within start_wait=10, and caches nothing so a later call retries. While LaunchDarkly is unreachable, every lazy call (config().invoke(), graph(), inspect_config) builds its own client in asyncio.to_thread and blocks for 10 s. N concurrent callers each hold a default-executor thread for that long, which can starve other to_thread users.

Change

Concurrent first calls now share one in-flight attempt, as the TypeScript SDK's initClient does.

  • The attempt builds the client on a daemon thread it starts itself, not on a default-executor thread. Its result is a concurrent.futures.Future, so callers on any event loop or thread can join it. The skills watcher, for example, runs asyncio.run on its own thread.
  • The attempt thread adopts the client itself, under _state_lock. Each caller waits on a future of its own loop, which is woken through call_soon_threadsafe, so cancelling one caller leaves the attempt running for the rest. (asyncio.wrap_future would cancel the shared future.)
  • Every caller of a failed attempt gets the same error. The attempt is cleared before any of them sees it, so a later call retries. A failure is still not cached.
  • The attempt closes its client instead of adopting it when shutdown() ran, when a client was passed in, or when every caller waiting on it was cancelled. A lone cancelled call still leaves nothing behind. Before this change, an attempt still in flight when shutdown() ran adopted its client afterwards.
  • Callers waiting on an attempt that a passed-in client replaced get the passed client, as before.
  • The README no longer says concurrent calls each build their own client.

Tests

New tests in packages/client/tests/test_lifecycle.py use a client constructor that blocks until the test releases it:

  • 5 concurrent calls construct one client.
  • 5 concurrent calls during a failing init construct one client, all raise, and the client is closed once; a later call constructs a new one.
  • Cancelling one of two waiters doesn't cancel the attempt for the other.
  • A call from asyncio.run on another thread joins the attempt.
  • shutdown() abandons an attempt in flight.
  • A client passed in mid-attempt replaces the attempt.

The old two-calls-build-two-clients test was replaced, since it tested the old behavior.

make lint format-check typecheck api-report-check test passes: 3805 passed, 18 skipped.

🤖 Generated with Claude Code


Note

Overview
Fixes a regression where N concurrent lazy init_client calls each blocked a default executor thread for up to 10s while LaunchDarkly was down, which could starve other asyncio.to_thread work.

Concurrent first-time inits now share a single _InitAttempt, aligned with the TypeScript SDK. One daemon thread builds LDClient with start_wait=10; callers on any event loop or thread wait on a shared concurrent.futures.Future, with per-loop waiters so cancelling one caller does not tear down the attempt for the rest. Failed attempts are still not cached: all waiters get the same error, unadopted clients are closed, and the next call retries.

Lifecycle edge cases are tightened under _state_lock: shutdown() or a BYOC init_client(client=...) abandons an in-flight attempt (waiters get abandoned or the passed client); adoption no longer races telemetry setup. The README now states that concurrent lazy calls share one attempt instead of each building their own client.

Tests replace the old “two clients” race with gated-SDK coverage for shared attempts, shared failure, cross-thread/cross-loop join, shutdown/BYOC replacement, and cancellation behavior.

Reviewed by Cursor Bugbot for commit f7972fe. Bugbot is set up for automated code reviews on this repo. Configure here.

XieX and others added 3 commits October 30, 2026 13:00
…t initialize

The LaunchDarkly SDK's constructor returns normally when start_wait runs
out and when the service rejects the SDK key; it only logs. init_client
adopted that client anyway, so a bad key or an unreachable LaunchDarkly
served every AI Config disabled with no error at startup.

init_client now checks is_initialized() on the client it built. When it
is false the client is closed and RuntimeError is raised; nothing is
cached, so a later call retries. This matches the JavaScript and Go
SDKs, which already fail here. A client passed as client= is not
checked: the caller owns its readiness.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With no shared in-flight attempt, every concurrent lazy call during an
outage built its own LaunchDarkly client in asyncio.to_thread and held a
default-executor thread for the whole 10 s start wait, which could starve
other to_thread users.

Calls made while an attempt is in flight now wait for it. The attempt runs
on a thread of its own, and its result is a concurrent.futures.Future, so
callers on any event loop or thread can join it. The attempt thread adopts
the client itself under a lock. Each caller waits on a future of its own
loop, so cancelling one leaves the attempt running for the rest. Every
caller of a failed attempt gets the failure, and the attempt is dropped
before they see it, so a later call retries. The client is closed instead
of adopted when shutdown() or a passed client replaced the attempt, or
when every caller waiting on it was cancelled.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@XieX
XieX marked this pull request as ready for review October 30, 2026 18:43
@XieX
XieX requested a review from jeffdupont October 30, 2026 18:43
@XieX XieX changed the title fix(AIC-3600): concurrent first init_client calls share one attempt fix(client): concurrent first init_client calls share one attempt Oct 30, 2026

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit be42e21. Configure here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@jeffdupont jeffdupont left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The shared-attempt design is sound. I checked cancellation (per-loop futures, waiters only under the lock), retry after failure (attempt cleared before the future settles), and the all-cancelled path; they behave as described. Three timed-out wait_for calls during an outage build one client.

Requesting changes for the two spec mismatches in the inline comments (passed client when the built client is uninitialized, and shutdown followed by re-init). Both are small. Please add tests for those two orders and for "all waiters cancelled, a new caller joins before the attempt finishes".

Nit: start_wait=10 and the "10 seconds" in the message and docstring could be one constant.

Base automatically changed from xie/aic-3600-init-client-check to main October 30, 2026 19:51
XieX and others added 2 commits October 30, 2026 15:52
…nt's state

Review of the shared init attempt found two orders where a waiting call
got the wrong answer:

- A client passed while an attempt was in flight, during an outage: the
  attempt's own client was uninitialized, and that check ran first, so
  its callers raised "failed to initialize" although the passed client
  was now the singleton. The replaced check now comes first.
- shutdown(), then a new call's attempt completing before the old one:
  the old attempt returned _client, which was the new attempt's client.
  A passed client now records itself on the attempt it replaces, and
  only that is returned; an attempt shutdown() replaced always raises.

Adds tests for both orders and for a call made after every waiter was
cancelled joining the attempt, and puts the 10 s start wait in one
constant.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nit-attempt

# Conflicts:
#	packages/client/README.md
#	packages/client/src/launchdarkly_ai_server/lifecycle.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants