4.x: CQL throttler slot and request timeout leak when onThrottleReady() throws

[nikagra]

Remaining problem after #965

#965 fixed pre-acquired stream-ID ownership and also repaired the equivalent admin-request and continuous-request cleanup paths. This issue now tracks the remaining normal CQL request lifecycle failure.

ConcurrencyLimitingRequestThrottler.register() claims a concurrency slot and then invokes onThrottleReady() synchronously. If that callback throws, the exception escapes and the claimed slot is never returned.

The dequeue path has the same shape: signalSuccess() invokes the next delayed request's onThrottleReady(true) while completing the current request, so a throw can escape into an unrelated request's completion path.

Remaining affected path

CqlRequestHandler schedules its timeout and then registers with the throttler from its constructor. Admission can synchronously enter sendRequest(), where request-ID decoration, statement conversion, callback construction, or other pre-write setup can throw.

Current consequences:

• the throttler concurrency slot is permanently lost;
• the scheduled timeout is not cancelled and fires later against an unobserved result;
• the exception can escape session.execute() synchronously instead of completing the returned stage;
• on delayed admission, the exception can escape through another request's throttler completion.

The regression test added by #965 verifies only that the pre-acquired stream ID is released on request-decoration failure. It intentionally does not close these lifecycle gaps.

Already fixed by #965

• ThrottledAdminRequestHandler.onThrottleReady() now releases its reservation and signals terminal error exactly once.
• ContinuousRequestHandlerBase now unwinds active execution/callback bookkeeping and signals the throttler on terminal pre-write failure.
• Pre-acquired stream-ID cleanup remains covered by 4.x: Pre-acquired stream id can leak when request fails before InFlightHandler write #947/fix: release pre-acquired stream ids #965.

Expected behavior

A synchronous CQL pre-write failure after throttler admission must:

1. release the pre-acquired stream-ID reservation;
2. complete the request result exceptionally rather than escape from session.execute();
3. cancel scheduled timeout/speculative work through normal terminal cleanup;
4. return the throttler permit exactly once;
5. not propagate through an unrelated request when admission came from the delayed queue.

Implementation direction

Route synchronous failures from CQL admission/send setup through the handler's existing terminal-error path, which already owns timeout cancellation and throttler signaling. Add coverage for both immediate and delayed admission, including unchanged throttler capacity and completed result stage.

Related

• fix: release pre-acquired stream ids #965 — merged pre-submission reservation cleanup
• 4.x: Stream ID can leak when InFlightHandler fails after acquisition #1002 — separate post-acquisition concrete stream-ID leak in InFlightHandler

[nikagra]

Third affected path

Found while re-reviewing #965 at ef698f9.

ContinuousRequestHandlerBase.sendRequest() leaks activeExecutionsCount on the same class of failure. The write-setup finally unwinds the stream id and the in-flight callback, but not the execution count:

java-driver/core/src/main/java/com/datastax/dse/driver/internal/core/cql/continuous/ContinuousRequestHandlerBase.java

Lines 391 to 398 in ef698f9

	} finally {
	if (!writeSubmitted) {
	if (nodeResponseCallback != null) {
	inFlightCallbacks.remove(nodeResponseCallback);
	}
	channel.cancelPreAcquireId();
	}
	}

whereas the channel == null branch a few lines above does, and can abort the session as a result:

java-driver/core/src/main/java/com/datastax/dse/driver/internal/core/cql/continuous/ContinuousRequestHandlerBase.java

Lines 363 to 368 in ef698f9

	if (channel == null) {
	// We've reached the end of the query plan without finding any node to write to; abort the
	// continuous paging session.
	if (activeExecutionsCount.decrementAndGet() == 0) {
	abortGlobalRequestOrChosenCallback(AllNodesFailedException.fromErrors(errors));
	}

So a throw from getMessage(), createPayload() or the NodeResponseCallback constructor — after the channel is acquired, before the write is submitted — decrements nothing.

Different counter and a different handler from paths 1 and 2, but the same shape: a synchronous throw between channel acquisition and InFlightHandler leaves per-request bookkeeping inflated with no owner left to unwind it.

Distinct consequence

This one doesn't lose a slot, it hangs. If the throwing execution was the last live one, nothing calls abortGlobalRequestOrChosenCallback, so the caller's CompletionStage is never completed and only the global timeout ends the continuous paging session. That's independent of which throttler is configured — unlike the concurrency-slot half, which needs ConcurrencyLimitingRequestThrottler.

Relationship to #965

#965 is right not to touch this. Its finally deliberately scopes to the stream id, which is #947's subject, and the else branch it adds at L400 is genuinely fine — chosenCallback is already done there, so the result is set and the count no longer matters. Recording it here so it isn't lost when someone picks this issue up.