[BUG] Large host-memory regression during IExecutionContext creation since TensorRT-RTX 1.4 on RTX 5060

[Lahahaha]

Describe the bug
Starting with TensorRT-RTX 1.4, creating an IExecutionContext causes an unexpectedly large host-memory allocation on an NVIDIA GeForce RTX 5060.

Each execution context retains approximately:

• 1.4–1.5 GiB of process private memory
• 0.95–1.0 GiB of working-set memory

With 10 static batch-size-1 image segmentation models loaded simultaneously, process private memory grows to approximately 15 GiB.

The same models and application use only approximately 2.3 GiB of additional private memory with TensorRT-RTX 1.3.

Detailed measurements show that the large allocation occurs specifically in ICudaEngine::createExecutionContext(). Runtime creation, engine deserialization, CUDA stream creation, and the first inference do not account for the increase.

The memory is mostly released when the execution contexts are destroyed. Therefore, this does not appear to be an application-side leak, but rather unexpectedly large memory retained for the lifetime of each execution context.

Steps to reproduce

1. Build several static batch-size-1 ONNX image segmentation models into TensorRT-RTX engines.
   • Most model inputs are 1x3x512x512.
   • One model input is 1x3x640x640.
   • None of the model inputs contain dynamic dimensions.
2. For every model:
   1. Create an IRuntime.
   2. Deserialize the engine using IRuntime::deserializeCudaEngine().
   3. Create an execution context using ICudaEngine::createExecutionContext().
   4. Keep the engine and execution context alive.
   5. Run one inference.
3. Repeat this for 10 models in one process.
4. Record process private memory and working-set memory after every API stage.
5. Repeat with TensorRT-RTX 1.3 and 1.4 or later on the same machine.

Simplified API sequence:

std::vector<ModelInstance> models;

for (const auto& engineData : engineFiles)
{
    ModelInstance model;

    model.runtime.reset(nvinfer1::createInferRuntime(logger));
    model.engine.reset(
        model.runtime->deserializeCudaEngine(
            engineData.data(), engineData.size()));

    // The large host-memory increase occurs here.
    model.context.reset(model.engine->createExecutionContext());

    runOneInference(*model.context);
    models.emplace_back(std::move(model));
}

// Keep all models alive while measuring process memory.

Version comparison on the same RTX 5060, using the same 10 models:

TensorRT-RTX version	GPU increase	Private-memory increase	Working-set increase
1.2.0.54	0.699 GiB	2.240 GiB	1.315 GiB
1.3.0.35	0.701 GiB	2.330 GiB	1.425 GiB
1.4.0.76	0.975 GiB	14.911 GiB	9.797 GiB
1.5.0.114	~0.82 GiB released on destruction	~13.67 GiB released	~9.12 GiB released
1.6.1.120	0.857 GiB	14.636 GiB	9.826 GiB

The clear regression starts between TensorRT-RTX 1.3 and 1.4.

Example measurements for one model with TensorRT-RTX 1.6.1:

stage=runtime-created
private=0.321 GiB working=0.237 GiB gpu=1.093 GiB

stage=engine-deserialized
private=0.398 GiB working=0.250 GiB gpu=1.156 GiB

stage=context-created
private=1.831 GiB working=1.278 GiB gpu=1.201 GiB

stage=first-inference-after
private=1.827 GiB working=1.269 GiB gpu=1.207 GiB

This shows an approximately 1.43 GiB private-memory increase during execution-context creation for a single model.

After all 10 contexts are destroyed with TensorRT-RTX 1.6.1:

GPU released       : 0.826 GiB
Private released   : 13.776 GiB
Working released   : 9.206 GiB

I also compared two AOT engine targets with TensorRT-RTX 1.4:

• Portable/cross-device AOT engine:
  • Private: 14.911 GiB
  • Working set: 9.797 GiB
• Current-GPU-only AOT engine using ComputeCapability::kCURRENT:
  • Private: 14.906 GiB
  • Working set: 9.783 GiB

Therefore, restricting the AOT engine to the current GPU does not reduce the memory usage.

Expected behavior

Creating an execution context should not retain approximately 1.4–1.5 GiB of host private memory per static batch-size-1 model.

Host-memory usage in TensorRT-RTX 1.4 and later should be comparable to TensorRT-RTX 1.3, or there should be a documented configuration/API that allows applications to avoid this per-context allocation.

Please confirm:

1. Is this amount of per-context host memory expected in TensorRT-RTX 1.4 and later?
2. Is it related to JIT-compiled kernels, the Windows batch-size-1 convolution backend introduced in 1.4, or another optimization?
3. Is there an API, builder option, runtime cache, or memory-pool configuration that can reduce or share this memory across execution contexts?
4. If this is a regression, is there a planned fix or recommended workaround other than using TensorRT-RTX 1.3?

Environment

• TensorRT-RTX versions tested:
  • 1.2.0.54
  • 1.3.0.35
  • 1.4.0.76
  • 1.5.0.114
  • 1.6.1.120
• GPU: NVIDIA GeForce RTX 5060, 8 GiB
• Compute capability: 12.0
• NVIDIA driver: 595.97
• Driver-reported CUDA version: 13.2
• CUDA Toolkit installed: 12.8
• Operating system: Microsoft Windows 11 Pro, 64-bit
• Windows version: 10.0.26200, build 26200
• Driver model: WDDM
• CPU: AMD Ryzen 5 8600G with Radeon 760M Graphics
• Models: 10 static batch-size-1 image segmentation models
• Input shapes: 1x3x512x512 and 1x3x640x640

Screenshots

Not applicable. Detailed per-stage process-memory logs can be attached.

Additional context

Inference results are correct. The issue is the amount of host memory retained while multiple execution contexts are alive.

The increase is highly repeatable and approximately linear with the number of execution contexts. Destroying each context releases most of its corresponding allocation.

TensorRT-RTX 1.3 does not exhibit this behavior on the same GPU, application, and models.