Skip to content

THRIFT-5830: Add per-call transport isolation for netstd clients - #4065

Merged
Jens-G merged 1 commit into
apache:masterfrom
birschick-bq:dev/birschick-bq/v4/THRIFT-5830
Nov 12, 2026
Merged

Jens-G merged 1 commit into
apache:masterfrom
birschick-bq:dev/birschick-bq/v4/THRIFT-5830

Conversation

@birschick-bq

@birschick-bq birschick-bq commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Asynchronous calls that overlap on one netstd client share a protocol and its transport, so their requests and responses can corrupt each other. This is the problem reported in THRIFT-5830 for THttpTransport ("Using THttpTransport using interleaved asynchronous calls may throw exception").

This PR adds opt-in support that gives every call its own transport view and protocol. Without the new generator option, generated output is byte-identical to today, and existing constructors and transports behave as before.

Usage

thrift -gen netstd:isolated my.thrift
var transport = new THttpTransport(new Uri("http://localhost:9090/thrift"), new TConfiguration());
using var client = new MyService.Client(transport, new TBinaryProtocol.Factory());

// these calls may overlap
await Task.WhenAll(client.add(1, 2), client.add(3, 4));

// explicit contexts allow out-of-order replies and a per-call timeout
using var first = client.CreateCallContext(TimeSpan.FromSeconds(5));
using var second = client.CreateCallContext();
await client.send_add(first, 1, 2);
await client.send_add(second, 3, 4);
var r2 = await client.recv_add(second);
var r1 = await client.recv_add(first);

A client generated with isolated is dual-mode. It keeps the legacy Client(TProtocol) and Client(TProtocol, TProtocol) constructors, which behave exactly as before, and gains Client(ICallIsolatedTransport, TProtocolFactory). The constructor you pick decides the behavior. All three constructors carry XML documentation that says when to use them.

What changed

Runtime (lib/netstd):

  • ICallIsolatedTransport hands out one transport view per call, owned by the caller. IAsyncCallIsolatedTransport adds asynchronous creation and IPerCallTimeoutTransport lets a transport declare a per-call timeout.
  • THttpTransport and TMemoryBufferTransport implement them. A view owns its request buffer and response, releases them on success, failure and cancellation, and reports NotOpen once disposed. Disposing or cancelling one view does not affect any other.
  • TIsolatedBufferedTransport and TIsolatedFramedTransport only wrap call-isolated transports. Passing a legacy TBufferedTransport or TFramedTransport to the new constructor does not compile.
  • TCallContext carries a call's protocol, sequence id, cancellation token and optional timeout. A reply with a different sequence id fails with BadSequenceId. The transport's timeout and a per-call timeout are linked with the caller's token and the shortest wins. A timeout surfaces as TTransportException TimedOut, distinct from cancellation by the caller. The timeout of the underlying HttpClient (ConnectTimeout, 30 seconds by default) also bounds every request of an HTTP call view; when it elapses without a cancellation by the caller, the view reports TimedOut too. The legacy THttpTransport is unchanged. Cancellation that arrives while a call view is being created disposes the view and fails the creation, even when the creator ignores the token. An HTTP call view that is disposed while its request is pending releases the late response and fails the flush with NotOpen, and it reads at most what the message size limit still allows, so a buffered layer that prefetches a full buffer does not reject a small valid reply.
  • TBaseClient gains the isolated constructor, CreateCallContext, CreateCallContextAsync and IsCallIsolated. SeqId now increments atomically.

Generator (compiler/cpp):

  • New netstd option isolated. High-level operations use a context per call when the client was built with the isolated constructor and the shared protocols otherwise. send_ and recv_ keep their signatures and gain overloads that take a TCallContext. The names that the option adds to the output are fully qualified, the System ones (global::System.Exception in the timeout catch clause) and the Thrift types (global::Thrift.TCallContext and so on), and the inherited members IsCallIsolated and CreateCallContextAsync are reached through base, so an IDL type or method with one of these names cannot capture them. The context variable gets a name that is free in each method, so the parameters of the shared-protocol and high-level operations keep the names of the IDL.

Docs and tests: a README section, a tutorial -isolated example with a smoke test case, a codegen harness -Isolated mode that also checks constructor documentation, compile fixtures (IsolatedCall.thrift, IsolatedExceptionName.thrift) for netstandard2.0, net8, net9 and net10, and unit and Catch tests. Makefile.am lists the new runtime file.

Compatibility

No breaking changes for users who do not opt in. Using isolated needs a thrift compiler built from this change. I compared default-mode output of this compiler with a compiler built from unmodified apache master for six IDL files (ThriftTest, tutorial, CassandraTest, Thrift6474, name_conflicts, fb303) with two option sets: 196 generated files, all byte-identical.

Testing

All of this was run on the final commit, rebased onto current apache master.

  • Thrift.Tests: 219 tests pass on net10 (upstream's tests and mine), covering the transports and layers, the call context and timeouts, and the generated client end to end (overlapping calls, out-of-order receive, one-way calls, exceptions, mismatched replies, cancellation, timeouts, multiplexed protocol, constructor shape, an IDL that defines an Exception type, the HttpClient timeout, cancellation during view creation, a late response, a small reply under a low size limit, IDL names that look like generated names).
  • Compiler tests: the Catch suite passes (440 assertions), including the new option.
  • The runtime builds for netstandard2.0, netstandard2.1, net8.0, net9.0 and net10.0, and the four compile fixtures and the tutorial build.
  • The codegen harness generates and compiles every IDL in the repository on net10, with and without -Isolated (249 tests each). CI now runs it with -Isolated as well.
  • The tutorial -isolated client was built and run against the tutorial server over HTTP with the binary protocol and two clients. I ran those cases by hand and have not run smoketest.sh itself.

Not covered: the netstandard2.0 branches (what .NET Framework runs) build but are not exercised at runtime, because the unit tests run on net10. The HTTP call view buffers the response with the default HttpClient completion option, which avoids the response-body timeout problem found on the earlier attempt, but I could not test .NET Framework 4.8.

Notes for reviewers

  • This supersedes my earlier attempts, THRIFT-5830: THttpTransport can now be used in interleaved async calls #3930 and THRIFT-5830: THttpTransport can now be used in interleaved async calls #3978 (closed) and THRIFT-5830: Add per-call transport support for netstd #4000. THRIFT-5830: Add per-call transport support for netstd #4000 selects per-call state implicitly with an AsyncLocal and wraps every generated method. This PR uses explicit call contexts instead, mainly so that the split send_/recv_ operations stay usable for interleaved replies and default generated output does not change. It keeps two ideas from THRIFT-5830: Add per-call transport support for netstd #4000: the per-call timeout and asynchronous transport creation. If you prefer the implicit design, I am happy to discuss it.
  • Open question: should a later PR drop the isolated option and emit the dual-mode client by default? I kept the option here so that this PR does not change every generated client.
  • The constructor shape has been confirmed to fit the use case of the hiveserver2 driver (adbc-drivers/hiveserver2), the downstream project whose generated code was hand-edited for this problem.
  • The generator change is part of THRIFT-5830 and not split into a separate ticket.
  • Review follow-ups: the two suggestions of the automated review of this PR are addressed. The generated timeout catch clause is now global::System.Exception, with a runtime test for an IDL that defines exception Exception (it compiled before and then reported a timeout as a plain cancellation; upstream fixed the same pattern in the generated processor in THRIFT-6474). The README now says that the HttpClient timeout is a third limit and that a longer PerCallTimeout needs a longer ConnectTimeout; beyond that suggestion, the HTTP call view now reports an elapsed HttpClient timeout as TimedOut, so it can be told apart from a cancellation by the caller.
  • A second automated review (Copilot) listed 12 findings. I checked each against the code and fixed those that were valid: the missing [TestMethod] on the overlapping-calls test (my mistake when I merged a conflict, so that test had not been running), IDL names that shadow the types and members of the generated client, parameter names that collide with generated locals (and the renaming of a callContext parameter, which could break callers that pass arguments by name), cancellation during view creation, a late response after dispose, and buffered reads under a low size limit. The README now says that a service and its base service both need isolated, and CI runs the harness with -Isolated. One finding is valid but left out on purpose: the HTTP transports buffer the whole response before a message size limit is checked. The legacy THttpTransport does the same, and streaming the response brings back the body-read timeout problem of the earlier attempts, so it deserves its own change and threat-model review.
  • Jens Geyer's commit on this branch, which makes TMemoryBufferTransport.CreateCallTransport throw ObjectDisposedException once the shared transport is disposed (as ICallIsolatedTransport documents and the other transports already did), is included unchanged in the squashed commit. He is credited with a Co-Authored-By trailer, together with the AI trailer from his commit. Thank you for the fix.
  • The branch was rebased onto current apache master; the one conflict, in THttpTransportTests.cs, was resolved by keeping the tests of both sides.
  • Not related to this change: in the tutorial, the compact protocol over HTTP already fails with a 400 without -isolated. I left it alone.

AI assistance

The runtime classes, generator changes, tests and documentation were largely written with AI assistance (Claude Sonnet 5.5) and then reviewed and tested by me. The commit carries the Co-Authored-By trailer.

Checklist

  • Did you create an Apache Jira ticket? THRIFT-5830
  • If a ticket exists: Does your pull request title follow the pattern "THRIFT-NNNN: describe my issue"?
  • Did you squash your changes to a single commit?
  • Did you do your best to avoid breaking changes? There are none for users who do not opt in, so no "Breaking-Change" label is needed.

Copilot AI balanced review requested due to automatic review settings October 29, 2026 17:53

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

@Jens-G Jens-G self-assigned this Oct 30, 2026
@mergeable mergeable Bot added c# Pull requests that update C# code Pull requests that update .NET code compiler build and general CI cmake, automake and build system changes labels Oct 30, 2026
@Jens-G

Jens-G commented Oct 30, 2026

Copy link
Copy Markdown
Member

Code review

This is a code-level review only. It does not answer the design questions under "Notes for reviewers": explicit contexts or the implicit design, and whether isolated should become the default.

Reviewed at b30689ad3. No blocking issues found. Checked for bugs and CLAUDE.md compliance.

The path listed under "Not covered" works for the HTTP call view. I ran a generated client on real .NET Framework 4.8 under Wine 10, which uses the netstandard2.0 build of the Thrift library, and on .NET 10. (Under Wine the managed code is Microsoft's and the socket layer is Wine's.) The peer was a local socket server that sends the headers and half of the body, then stalls. The two runtimes gave the same results:

Case Result (TTransportException type)
PerCallTimeout of 1 s TimedOut after about 1 s
Context timeout of 1 s, send_ then recv_ TimedOut after about 1 s
Caller token cancelled after 1 s Interrupted after about 1 s
One stalled call next to five normal ones all five correct, the stalled one TimedOut
20 overlapping calls, isolated constructor 20 of 20 correct
20 overlapping calls, legacy constructor 0 of 20 correct

Also at this head:

  • smoketest.sh passes all 8 cases, including -tr:http -isolated -mc:2.
  • Default output is byte-identical to the merge-base compiler for 248 repo IDLs with 12 option sets.
  • ThriftTest.thrift and IsolatedCall.thrift with isolated build with 9 option sets on each of netstandard2.0, net8.0, net9.0 and net10.0.

Two suggestions, below the bar for an issue but verified:

1. The new catch can bind to an IDL type. The generator emits catch (Exception timeoutCause_) without a namespace. If a type named Exception exists in the service's namespace, an enclosing namespace or the global namespace, the name means that type, even if the service does not use it. With exception Exception the client still builds without a warning, but the catch no longer sees the cancellation. A timeout then no longer surfaces as TimedOut and looks like a caller cancellation.

I added exception Exception to IsolatedCall.thrift and reran the cases above. The right-hand column comes from a compiler built with only that type changed in line 2510. Both runtimes gave the same results:

Case This PR catch (global::System.Exception timeoutCause_)
PerCallTimeout of 1 s Interrupted TimedOut
Context timeout of 1 s, send_ then recv_ Interrupted TimedOut
Context timeout of 1 s, reply in time, recv_ 1.5 s later raw OperationCanceledException TimedOut
Caller token cancelled after 1 s Interrupted Interrupted

With that change the compiler tests and all 140 Thrift.Tests still pass, and the generated output differs only in those catch lines. The generator already qualifies names this way, for example global::System.Threading.Tasks.Task. lib/java/src/test/resources/JavaTypes.thrift is the only repo IDL with such a type, and generated with isolated it shows the same difference. A compile fixture would not catch this, because the output builds; a test has to assert TimedOut at runtime.

out << indent() << "}" << '\n'
<< indent() << "catch (Exception timeoutCause_) when (" << CALL_CONTEXT_NAME << ".IsTimeoutFailure(timeoutCause_))" << '\n'
<< indent() << "{" << '\n';
indent_up();
out << indent() << "throw " << CALL_CONTEXT_NAME << ".CreateTimeoutException(timeoutCause_);" << '\n';

2. ConnectTimeout is a third limit the README does not name. The section says "Two timeouts can apply to a call", but HttpClient.Timeout, which is THttpTransport.ConnectTimeout and 30 s by default, still bounds each request of a call view. A longer PerCallTimeout does not extend it. When it fires, the call fails with Interrupted, the same type a caller cancellation gets, so "the two can be told apart" does not hold for this limit.

With ConnectTimeout = 1000 and a PerCallTimeout of 3 s against the stalling server, the call failed after about 1 s with Interrupted on both runtimes. The message names HttpClient.Timeout on .NET 10 and is only "A task was canceled." on .NET Framework. The XML doc on PerCallTimeout does mention ConnectTimeout. A sentence in the README section would tell readers that a PerCallTimeout above 30 s also needs a higher ConnectTimeout.

thrift/lib/netstd/README.md

Lines 158 to 162 in b30689a

Two timeouts can apply to a call. A transport can declare a default for all of its calls, for example
`THttpTransport.PerCallTimeout`, and one call can be given a tighter one of its own. Both are linked with the
cancellation token you pass, and the shortest bound wins. A per-call timeout cannot loosen the transport's.

🤖 Generated with Claude Code

Copilot AI balanced review requested due to automatic review settings October 30, 2026 22:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@Jens-G

Jens-G commented Oct 30, 2026

Copy link
Copy Markdown
Member

This branch has conflicts that must be resolved

@birschick-bq
birschick-bq force-pushed the dev/birschick-bq/v4/THRIFT-5830 branch from f111b53 to cfe61d7 Compare November 1, 2026 19:42
Copilot AI balanced review requested due to automatic review settings November 1, 2026 19:42

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot AI balanced review requested due to automatic review settings November 1, 2026 19:50

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Generated C# can fail to compile or break named-argument callers, while HTTP response handling has resource and size-limit flaws and the isolation test is not discovered.

9 open issues
2 previously missed issues

In code that hasn't changed since last review

Medium severity Run the codegen harness with Isolated in CI

lib/​netstd/​Tests/​codegen/​run-NetStd-Codegen-Tests.ps1:24

[P2] The new broad codegen mode is never exercised by CI: .github/workflows/build.yml:604-608 invokes this harness without -Isolated. The two isolated fixtures do not cover the repository-wide IDLs checked by this harness, so subsequent generator changes can break other isolated clients without failing CI. Run the harness with -Isolated in the existing codegen matrix as well.

Medium severity Limit buffered reads to the remaining message allowance

lib/​netstd/​Thrift/​Transport/​Layered/​TIsolatedLayeredTransport.cs:120

The buffered view rejects valid short replies when MaxMessageSize is smaller than its buffer: TBufferedTransport.ReadAsync requests 1024 bytes, and the HTTP view checks that request against its remaining allowance before reading. The same problem occurs near the limit of a larger reply. Bound prefetches by the remaining message allowance rather than always requesting the full buffer.

1 resolved since last review

Review effort: Balanced

Comment thread lib/netstd/README.md Resolved
@birschick-bq
birschick-bq force-pushed the dev/birschick-bq/v4/THRIFT-5830 branch from 71ebf1f to a0297a7 Compare November 1, 2026 20:01
@birschick-bq

Copy link
Copy Markdown
Contributor Author

🤖 Generated with Claude Code

Thanks for the review. Both suggestions are addressed in the updated commit.

1. Unqualified Exception in the generated timeout catch clause

I reproduced it with an IDL that defines exception Exception. The generated code still compiled, but the catch bound to the IDL type, so the call failed with a plain TaskCanceledException instead of TimedOut. The clause is now catch (global::System.Exception ...), the same way THRIFT-6474 qualifies the processor's catch-all. As suggested, a compile fixture alone would not catch this, so there is a runtime test: IsolatedExceptionName.thrift defines exception Exception, and the test asserts TimedOut (it failed before the fix). The System names that the isolated option adds are now qualified; the existing unqualified names, for example in the legacy operations, are untouched so that default output stays byte-identical.

2. HttpClient.Timeout is a third limit

Confirmed: with ConnectTimeout at 100 ms, an elapsed HttpClient timeout and a caller cancellation both surfaced as Interrupted. The README now says that the timeout of the HttpClient (ConnectTimeout, 30 seconds by default) also bounds each request, that a longer PerCallTimeout needs a longer ConnectTimeout, and how each case is reported.

I went one step further than the README sentence. The HTTP call view now reports an elapsed HttpClient timeout as TTransportException TimedOut when the caller did not cancel. Otherwise the requirement that a timeout can be told apart from a cancellation held only for the per-call timeout. A cancellation by the caller is still Interrupted, and the legacy THttpTransport is unchanged. There are tests for the HttpClient timeout, for caller cancellation, and for a PerCallTimeout that is longer than the HttpClient timeout. The message text of the exception differs between .NET and .NET Framework, which the README mentions. I could not test .NET Framework itself.

Also

  • Rebased onto current apache master. The one conflict, in THttpTransportTests.cs, was resolved by keeping the tests of both sides.
  • Re-ran everything on the final commit: 210 unit tests, the Catch compiler tests (440 assertions), the codegen harness in default and -Isolated mode (248 tests each), and all runtime targets and compile fixtures. Default-mode output is still byte-identical to a compiler built from unmodified master (six IDL files, two option sets, 196 files).

@birschick-bq
birschick-bq force-pushed the dev/birschick-bq/v4/THRIFT-5830 branch from a0297a7 to 5884924 Compare November 2, 2026 17:43
Copilot AI balanced review requested due to automatic review settings November 2, 2026 17:43
@birschick-bq

Copy link
Copy Markdown
Contributor Author

🟡 Changes recommended

Generated C# can fail to compile or break named-argument callers, while HTTP response handling has resource and size-limit flaws and the isolation test is not discovered.

9 open issues

2 previously missed issues
In code that hasn't changed since last review

Medium severity Run the codegen harness with Isolated in CI
lib/​netstd/​Tests/​codegen/​run-NetStd-Codegen-Tests.ps1:24

[P2] The new broad codegen mode is never exercised by CI: .github/workflows/build.yml:604-608 invokes this harness without -Isolated. The two isolated fixtures do not cover the repository-wide IDLs checked by this harness, so subsequent generator changes can break other isolated clients without failing CI. Run the harness with -Isolated in the existing codegen matrix as well.

Medium severity Limit buffered reads to the remaining message allowance
lib/​netstd/​Thrift/​Transport/​Layered/​TIsolatedLayeredTransport.cs:120

The buffered view rejects valid short replies when MaxMessageSize is smaller than its buffer: TBufferedTransport.ReadAsync requests 1024 bytes, and the HTTP view checks that request against its remaining allowance before reading. The same problem occurs near the limit of a larger reply. Bound prefetches by the remaining message allowance rather than always requesting the full buffer.

1 resolved since last review

Review effort: Balanced

Thanks for the review. I checked each of the 12 findings against the code and reproduced the ones that looked like runtime or compile problems before changing anything. The fixes are in the force-pushed commit, and each came with a test that failed first.

Fixed:

  • The overlapping-calls test had lost its [TestMethod] attribute when I resolved a merge conflict, so it was not running. It is restored and passes.
  • Generated code could be captured by IDL names. The Thrift types the option adds are now global::Thrift..., and IsCallIsolated and CreateCallContextAsync are reached through base. A new fixture (IsolatedNames.thrift) with reflection tests covers it.
  • The context variable now gets a name that is free in each method, and parameters are no longer renamed, so callers that pass arguments by name keep working.
  • Cancellation that arrives while a call view is being created now disposes the view and fails the creation, in the isolated layers and in TBaseClient, even if the factory ignores the token.
  • An HTTP call view disposed while its request is pending now releases the late response and fails the flush with NotOpen.
  • The HTTP call view reads at most what the message size limit still allows, so a buffered layer that prefetches a full buffer no longer rejects a small valid reply. Larger replies are still rejected.
  • The README now says that a service and its base service both need isolated, and CI runs the codegen harness with -Isolated in addition to the default run.

Already resolved: the disposed TMemoryBufferTransport finding is covered by Jens Geyer's commit, which is included unchanged.

Valid but not changed here: the HTTP transports buffer the whole response before a message size limit is checked. The legacy THttpTransport behaves the same way, and streaming the response brings back the body-read timeout problem of the earlier attempts, so I think it needs its own change and a look at the threat model. I called this out in the PR description.

Verification on the final commit: 219 unit tests, the Catch suite, the codegen harness in both modes (249 tests each), all five runtime targets, and default output still byte-identical to a compiler built from unmodified master.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread lib/netstd/Thrift/TBaseClient.cs Resolved
Comment thread lib/netstd/Thrift/TBaseClient.cs Resolved
Comment thread tutorial/netstd/Client/Program.cs Resolved
@birschick-bq
birschick-bq force-pushed the dev/birschick-bq/v4/THRIFT-5830 branch from 5884924 to 8c1ad98 Compare November 2, 2026 18:32
Copilot AI balanced review requested due to automatic review settings November 2, 2026 18:32
@birschick-bq

Copy link
Copy Markdown
Contributor Author

🟡 Changes recommended

Inherited RPC name collisions can break generated client compilation, while unresolved disposal and cancellation bugs risk resource leaks and incorrect call behavior.

7 open issues

9 resolved since last review

Review effort: Balanced

Thanks for the third round. I checked each of the seven findings against the code, and each one that was valid came with a test that failed before the fix.

Fixed:

  • The generated client now reaches IsCallIsolated and CreateCallContextAsync through a cast to TBaseClient, not through base, which is the parent client for a service that extends another. A fixture service that extends a parent with those method names covers it.
  • Synchronous context creation checks the token before and after the view is created.
  • A context that cannot be built disposes the protocol as well as the view.
  • The HTTP call view forwards the token in the convenience WriteAsync overload, and the memory call view checks the token on reads and writes.
  • The tutorial client rejects -isolated with -bf:framed, because its HTTP server reads unframed requests.

Not changed: the buffered and framed layers are the legacy transports. They reach the inner view only at flush or read, where the token now applies, so wrapping them would add a layer without a clear gain.

Verification on the amended commit: 228 unit tests, the Catch suite (440 assertions), the codegen harness in default and -Isolated mode (249 tests each), all five runtime targets, the compile fixtures and the tutorial projects. Default output is still byte-identical to a compiler built from unmodified master.

@mergeable mergeable Bot added the github_actions Pull requests that update GitHub Actions code label Nov 2, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines +56 to +57
$EXE_EXT = $IsWindows ? ".exe" : ""

Comment on lines +143 to +144
# a constructor is documented when the line above it is part of an XML documentation comment
$documented = $lines[$i - 1].Trim().StartsWith('///')
}
catch (OperationCanceledException ocx)
{
throw new TTransportException(TTransportException.ExceptionType.Interrupted, ocx.Message, ocx);
catch (HttpRequestException wx)
{
ReleaseResponse();
throw new TTransportException(TTransportException.ExceptionType.Unknown, "Couldn't connect to server: " + wx, wx);
if (Logger.IsEnabled(LogLevel.Information))
Logger.LogInformation("Multiplex {mplex}", mplex);

var isolated = args.Any(x => x.Equals("-isolated"));
@Jens-G

Jens-G commented Nov 3, 2026

Copy link
Copy Markdown
Member

Code review

This is a code-level review only. It does not answer the design questions under "Notes for reviewers": explicit contexts or the implicit design, and whether isolated should become the default.

Re-reviewed at 8c1ad98b9. No issues found. Checked for bugs and CLAUDE.md compliance.

Both suggestions from the last round are in, and the disposed check on TMemoryBufferTransport.CreateCallTransport() from f111b539a is carried over with its test.

Also at this head:

  • Thrift.Tests pass 228 of 228, and the compiler's Catch suite passes (509 assertions).
  • Default output, without isolated, is byte-identical to the merge-base compiler for the repo IDLs with 10 option sets: 2,620 runs, none differ. Eight IDLs fail with both compilers alike.
  • With isolated, 11 option combinations build on netstandard2.0, net8.0 and net10.0. The inputs are the three new fixtures, ThriftTest.thrift and an extra IDL whose method, parameter and type names collide with the new members (CreateCallContext, callContext, timeoutCause_, cancellationToken, Exception, TCallContext, and a derived service with an IsCallIsolated() method).

🤖 Generated with Claude Code

@Jens-G

Jens-G commented Nov 7, 2026

Copy link
Copy Markdown
Member

I'll wait with this until #3963 is complete and merged. Have it on my list.

Copilot AI balanced review requested due to automatic review settings November 7, 2026 12:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@Jens-G

Jens-G commented Nov 7, 2026

Copy link
Copy Markdown
Member

I added a maintainer commit on top of yours, 90cab3c. Merged with today's master, lib/netstd/Thrift.slnx no longer builds. Since THRIFT-4833 (#4103, merged on November 6), the Thrift.Compile projects treat CS0108 and CS0109 as errors, and the generated IsCallIsolated(int, CancellationToken) of Names.Client and Parent.Client hides TBaseClient.IsCallIsolated. That gives 8 CS0108 errors, two each in Thrift.Compile.net8, net9, net10 and netstd2. The last CI run of this PR was before #4103.

The commit:

  • marks a generated client method new where it hides a member that this PR adds to TBaseClient: any method named IsCallIsolated, and CreateCallContext or CreateCallContextAsync without arguments, as the cancellation token is then its only parameter, the same as for the TBaseClient methods. No other method gets new, as that would be CS0109. CreateCallContextAsync(1: i32 a) in the fixture is such an overload.
  • adds a service NoArgs to IsolatedNames.thrift with methods of these three names without arguments, and a test that checks that the token is their only parameter.

Tested on master (30a9a92) merged with this branch, with SDK 10.0.403:

  • Without the generator change, the extended fixture gives 5 CS0108 errors per Thrift.Compile project, 20 in all. With it, Thrift.slnx builds in Release and Debug with no errors and no warnings, and so do ThriftTest.slnx and Tutorial.slnx.
  • Thrift.Tests passes 298/298, Thrift.IntegrationTests 57/57, and the tutorial smoke test 8 of 8, -tr:http -isolated included.
  • The codegen legs for net8, net9 and net10 pass 254/254 each, with and without -Isolated.
  • Without the isolated option, the fixture's generated code, compiled with CS0108, CS0109 and CS0114 as errors, gives 5 CS0108 errors before the change and builds cleanly after it.
  • With async_postfix, CreateCallContext() becomes CreateCallContextAsync(CancellationToken) and is marked new, while IsCallIsolatedAsync is not.

Please pull before your next push, so that the commit stays on the branch. A force-push from your current head would drop it.

Service methods can also hide members that TBaseClient had before this PR: ClientId, InputProtocol, OutputProtocol and SeqId by name, and the virtual OpenTransportAsync when they take no arguments (CS0114). They are not part of this commit.

Once #3963 (net11) is merged, this PR also needs its net11 counterparts: three Exec lines in Thrift.Compile.net11.csproj for IsolatedCall.thrift, IsolatedExceptionName.thrift and IsolatedNames.thrift, and net11,isolated in tutorial/netstd/Interfaces/Interfaces.csproj.

🤖 Generated with Claude Code

@Jens-G

Jens-G commented Nov 7, 2026

Copy link
Copy Markdown
Member

The collisions with the older TBaseClient members are now THRIFT-6528. For InputProtocol, OutputProtocol and SeqId they are compile errors, not only hiding.

🤖 Generated with Claude Code

@Jens-G

Jens-G commented Nov 7, 2026

Copy link
Copy Markdown
Member

lib-haxe-codegen is red here because of a bug in the Haxe generator, not in this PR. The generator names the argument and result structs of a method after the method alone, so the NoArgs service that my commit 90cab3c added to IsolatedNames.thrift overwrote the structs of Names and Parent. That is THRIFT-6529, fixed in #4133.

Nothing to change on your side. Once the fix is on master, the job passes with the next run of this PR, after your next push or a close and reopen: I ran it on master merged with this branch and the fix. Re-running the current run does not help, as it builds the old merge commit.

🤖 Generated with Claude Code

@Jens-G Jens-G closed this Nov 7, 2026
@Jens-G Jens-G reopened this Nov 7, 2026
@Jens-G Jens-G mentioned this pull request Nov 11, 2026
5 tasks done
Copilot AI balanced review requested due to automatic review settings November 11, 2026 23:18
@Jens-G

Jens-G commented Nov 11, 2026

Copy link
Copy Markdown
Member

I merged master into this branch, f91bd65. Master now has #3963 (net11) and #4168 (THRIFT-6383), which drops net8 and net9. The merge resolves three conflicts:

  • Thrift.Compile.net8 and Thrift.Compile.net9 are gone, and their isolated lines with them.
  • Thrift.Compile.net11.csproj gets the three Exec lines for IsolatedCall.thrift, IsolatedExceptionName.thrift and IsolatedNames.thrift, with net11,isolated. Thrift.Tests builds against Thrift.Compile.net11 now, and your new tests use the types generated from these files.
  • tutorial/netstd/Interfaces/Interfaces.csproj keeps master's net11 and adds isolated.

Every other file has the same changes against master as this PR had against its base.

Tested at f91bd65 with SDK 11.0.100:

  • The compiler's Catch suite passes 45 test cases (546 assertions), and ctest 25 of 25.
  • --gen netstd:net8 and netstd:net9 stop with "unknown option". net10, net11 and isolated, alone and combined, generate code.
  • Thrift.slnx, ThriftTest.slnx and Tutorial.slnx build in Release and Debug with no errors and no warnings.
  • Thrift.Tests passes 298/298 and Thrift.IntegrationTests 57/57 on net11.0.
  • The codegen legs for net10 and net11 pass 254/254 each, with and without -Isolated, with every warning an error.
  • The steps of the lib-netstd job pass, the tutorial smoke test 8 of 8 included (-tr:http -isolated among them), and the netstd-netstd cross test passes 5 of 5.

Please pull before your next push, so that the merge stays on the branch.

The description still names net8 and net9 for the fixtures and the runtime targets. After the merge, the fixtures are for netstandard2.0, net10 and net11, and the runtime targets netstandard2.0, netstandard2.1, net10.0 and net11.0.

🤖 Generated with Claude Code

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@Jens-G
Jens-G force-pushed the dev/birschick-bq/v4/THRIFT-5830 branch from f91bd65 to a57991c Compare November 11, 2026 23:49
@Jens-G

Jens-G commented Nov 12, 2026

Copy link
Copy Markdown
Member

Update: the branch has been rebased onto master since, instead of merged, at a57991c. Its content is the same as f91bd65, so the tests above still apply, and the CI run for f91bd65 passed. If you have a local copy of the branch, reset it to the new head (git fetch and git reset --hard to dev/birschick-bq/v4/THRIFT-5830 of your fork) rather than pulling, as a pull would bring the old commits back.

🤖 Generated with Claude Code

Client: netstd,cpp

Asynchronous calls that overlap on one client share a protocol and its
transport, so their requests and responses can corrupt each other, for
example over THttpTransport. Add opt-in support that gives every call its
own transport view and protocol. Behavior and generated output without the
new option are unchanged.

Runtime (lib/netstd):
- ICallIsolatedTransport hands out one transport view per call; the caller
  owns and disposes it. IAsyncCallIsolatedTransport adds asynchronous
  creation, and IPerCallTimeoutTransport lets a transport declare a
  per-call timeout (unbounded by default).
- THttpTransport and TMemoryBufferTransport implement them. A view owns its
  request buffer and response, releases them on success, failure and
  cancellation, and reports NotOpen once disposed. Disposing or cancelling
  one view does not affect any other. Once the shared transport is
  disposed, THttpTransport, the isolated layers and, as of the change by
  Jens Geyer included here, TMemoryBufferTransport refuse to create call
  views and throw ObjectDisposedException, as ICallIsolatedTransport
  documents. Cancellation that arrives while a call view is being created
  disposes the view and fails the creation, even when the creator ignores
  the token. An HTTP call view that is disposed while its request is
  pending releases the late response and fails the flush with NotOpen. It
  reads at most what the message size limit still allows, so a buffered
  layer that prefetches a full buffer does not reject a small valid reply.
  The call views of HTTP and memory honor the token of every write overload
  and the memory view that of a read, so a cancelled or timed-out call does
  not keep reading or modifying its buffer. Creating a context
  synchronously checks the token before and after the view is created, as
  the asynchronous path does, and a context that cannot be built disposes
  the protocol as well as the view.
- TIsolatedBufferedTransport and TIsolatedFramedTransport only wrap
  call-isolated transports. Passing a legacy TBufferedTransport or
  TFramedTransport to the new constructor does not compile.
- TCallContext carries a call's protocol, its sequence id, its cancellation
  token and an optional timeout. A reply with a different sequence id is
  rejected with BadSequenceId. The transport's timeout and a timeout given
  for one call are linked with the caller's token, the shortest wins, and a
  timeout is reported as TTransportException TimedOut, distinct from
  cancellation by the caller. The timeout of the underlying HttpClient
  (ConnectTimeout) also bounds every request of an HTTP call view; when it
  elapses without a cancellation by the caller, the view reports TimedOut
  too. The legacy THttpTransport is unchanged.
- TBaseClient gains a constructor for a call-isolated transport and a
  protocol factory, CreateCallContext and CreateCallContextAsync, and
  IsCallIsolated. The existing constructors keep working; SeqId now
  increments atomically.

Generator (compiler/cpp):
- New netstd option "isolated". A client generated with it keeps the legacy
  constructors and gains the call-isolated constructor, each with XML
  documentation that says when to use it. High-level operations use a
  per-call context when the client was built with the new constructor and
  the shared protocols otherwise. send_ and recv_ keep their signatures and
  gain overloads that take a TCallContext, so replies can be received out
  of order.
- The names that the option adds to the output are fully qualified, the
  System ones (global::System.Exception in the timeout catch clause) and
  the Thrift types (global::Thrift.TCallContext and so on), and the
  inherited members IsCallIsolated and CreateCallContextAsync are reached
  through a cast to the client base class, not through base, which is the
  client of the parent for a service that extends another service. An IDL
  type or method with one of these names cannot capture them. The
  context variable gets a name that is free in each
  method, so the parameters of the shared-protocol and high-level
  operations keep the names of the IDL.
- A client method that hides a member of TBaseClient is declared new:
  any method named IsCallIsolated, and one named CreateCallContext or
  CreateCallContextAsync without arguments, as the cancellation token is
  then its only parameter, the same as for the TBaseClient methods. The
  Thrift.Compile projects treat CS0108 and CS0109 as errors.
- Apart from that new modifier, output without the option is
  byte-identical to before. Using "isolated" needs a thrift compiler
  built from this change.

Tests and documentation:
- Unit tests for transports, layers, call context, timeouts and generated
  clients (overlapping calls, out-of-order receive, one-way calls,
  exceptions, mismatched replies, cancellation, timeouts, multiplexed
  protocol, constructor shape, an IDL that defines an Exception type, the
  HttpClient timeout, a disposed shared memory transport, cancellation
  during view creation, a late response, a small reply under a low size
  limit, cancelled writes and reads of call views, synchronous creation
  with a cancelled token, an ancestor service with methods named like
  client members, methods named like the TBaseClient members). Catch
  tests for the option. Compile fixtures (IsolatedCall.thrift,
  IsolatedExceptionName.thrift, IsolatedNames.thrift) for netstandard2.0,
  net10 and net11.
- The codegen harness gains -Isolated, checks the constructor
  documentation, and now detects Windows correctly. CI runs it with
  -Isolated next to the existing run.
- lib/netstd/README.md documents usage, timeouts, ownership, multiplexing
  and that a service and its base service both need the option; tutorial/netstd gets an -isolated example and a smoke test
  case; Makefile.am lists TCallContext.cs.

Every await that the change adds to the library and to the generated
code uses ConfigureAwait(false), as lib/netstd requires since
THRIFT-6538.

No breaking changes for users who do not opt in. Supersedes the earlier
approaches in PR apache#3930, apache#3978 and apache#4000.

AI assistance: the runtime classes, generator changes, tests and
documentation were largely written with AI assistance and then reviewed
and tested by the author. The change to TMemoryBufferTransport (refuse
call views once it is disposed) and its test, and the new modifier for
client methods that hide TBaseClient members, were written by Jens
Geyer, with the AI assistance noted in his trailer below, and are
included unchanged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Jens Geyer <jensg@apache.org>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
@Jens-G
Jens-G force-pushed the dev/birschick-bq/v4/THRIFT-5830 branch from a57991c to 4d9a42d Compare November 12, 2026 21:48
Copilot AI balanced review requested due to automatic review settings November 12, 2026 21:48

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@Jens-G
Jens-G merged commit 84de810 into apache:master Nov 12, 2026
93 of 94 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build and general CI cmake, automake and build system changes c# Pull requests that update C# code Pull requests that update .NET code compiler github_actions Pull requests that update GitHub Actions code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants