TensorRT 11.0.0.114 builder finds no valid tactic for BF16 ConvTranspose2d (works fine in FP16)

[robbysun]

TensorRT 11.0.0.114 builder finds no valid tactic for BF16 ConvTranspose2d (works fine in FP16)

Bug description

IBuilder::buildEngineWithConfig fails to build a serialized engine for a network containing
a single IDeconvolutionLayer (ConvTranspose2d) when the layer's I/O tensors are BF16.
The identical layer (same shapes, same strongly-typed network construction) builds
successfully when the dtype is FP16 instead. This reproduces both through the pure
TensorRT Python API (no PyTorch / torch_tensorrt involved) and through torch_tensorrt.dynamo.compile.

Encountered while exporting a real model (a ViT + FPN image encoder) to TensorRT via
torch_tensorrt; a 2x-upsampling nn.ConvTranspose2d(1024, 512, kernel_size=2, stride=2)
layer in the FPN is the only thing blocking the whole graph from compiling in BF16.

Minimal reproducer (pure TensorRT Python API, no torch/torch_tensorrt)

import numpy as np
import tensorrt as trt

TRT_LOGGER = trt.Logger(trt.Logger.VERBOSE)

def build(dtype, label):
    builder = trt.Builder(TRT_LOGGER)
    flags = 1 << int(trt.NetworkDefinitionCreationFlag.STRONGLY_TYPED)
    network = builder.create_network(flags)
    config = builder.create_builder_config()

    IN_C, OUT_C, K, S = 1024, 512, 2, 2
    H = W = 72

    inp = network.add_input(name="x", dtype=dtype, shape=(1, IN_C, H, W))

    rng = np.random.default_rng(0)
    w = rng.standard_normal((IN_C, OUT_C, K, K)).astype(np.float32)
    if dtype == trt.DataType.BF16:
        import ml_dtypes
        w_cast = w.astype(ml_dtypes.bfloat16)
    else:
        w_cast = w.astype(np.float16)
    weights = trt.Weights(dtype, w_cast.ctypes.data, w_cast.size)

    deconv = network.add_deconvolution_nd(
        input=inp, num_output_maps=OUT_C, kernel_shape=(K, K), kernel=weights,
    )
    deconv.stride_nd = (S, S)
    network.mark_output(deconv.get_output(0))

    print(f"\n=== Building {label} engine ===")
    serialized = builder.build_serialized_network(network, config)
    if serialized is None:
        print(f"{label}: BUILD FAILED (builder returned None)")
        return False
    print(f"{label}: BUILD SUCCEEDED ({serialized.nbytes} bytes)")
    return True

print("TensorRT version:", trt.__version__)
build(trt.DataType.BF16, "BF16")
build(trt.DataType.HALF, "FP16")

Output:

TensorRT version: 11.0.0.114

=== Building BF16 engine ===
BF16: BUILD FAILED (builder returned None)

=== Building FP16 engine ===
FP16: BUILD SUCCEEDED (4222860 bytes)

Relevant verbose log lines for the BF16 attempt:

[TRT] [V] =============== Computing costs for (Unnamed Layer* 0) [Deconvolution]
[TRT] [V] *************** Autotuning format combination: Half(5308416,5184,72,1) -> Half(10616832,20736,144,1) ***************
[TRT] [V] Skipping CaskDeconvolution: No valid tactics for (Unnamed Layer* 0) [Deconvolution]
[TRT] [V] Skipping CaskDeconvolutionV2: No valid tactics for (Unnamed Layer* 0) [Deconvolution]
[TRT] [V] (Unnamed Layer* 0) [Deconvolution]: 189 available tactics, 0 unparsable, 94 pruned, 95 remaining after tactic pruning.

(189 tactics are enumerated and timed, but none is accepted — the builder ultimately returns None
with no explicit error printed at default log severity; only visible in VERBOSE.)

Second reproducer: via torch_tensorrt (same root cause, more informative error)

Compiling the same shape through torch_tensorrt.dynamo.compile surfaces an explicit error
instead of a silent None:

import torch
import torch.nn as nn
import torch_tensorrt

class Deconv(nn.Module):
    def __init__(self):
        super().__init__()
        self.dconv = nn.ConvTranspose2d(1024, 512, kernel_size=2, stride=2)
    def forward(self, x):
        return self.dconv(x)

device = torch.device("cuda")
m = Deconv().eval().to(device).to(torch.bfloat16)
x = torch.randn(1, 1024, 72, 72, device=device, dtype=torch.bfloat16)

with torch.no_grad():
    ep = torch.export.export(m, (x,))

trt_ep = torch_tensorrt.dynamo.compile(
    ep, inputs=[x],
    enabled_precisions={torch.bfloat16},
    use_explicit_typing=False,
    device=torch_tensorrt.Device(gpu_id=0),
    min_block_size=1,  # force conversion of this small subgraph
)

Error:

ERROR:torch_tensorrt [TensorRT Conversion Context]:Error Code: 9: Skipping tactic 0x0000000000000000 due to exception [type.cpp:186: infer_type] Could not infer output types for operation:    35: deconv: output0_before_bias- | x-(bf16[1,1024,72,72][]so[3,2,1,0]p[0,0,0,0], mem_prop=0, align=2), [DECONVOLUTION]-[aten_ops.convolution.default]-[dconv/convolution] filterWeightsBFloat16-{-0.00448608, -0.00393677, -0.00421143, 0.00521851, -0.0133057, 0.00176239, 0.0220947, -0.0159912, ...}(bf16[1024,512,2,2][2048,4,2,1]so[3,2,1,0], mem_prop=0, align=2)<entry>, __mye36[DECONVOLUTION]-[aten_ops.convolution.default]-[dconv/convolution]_alpha-1F:(f32[][]so[], mem_prop=0, align=4)<entry>, __mye37[DECONVOLUTION]-[aten_ops.convolution.default]-[dconv/convolution]_beta-0F:(f32[][]so[], mem_prop=0, align=4)<entry>, stream = 0 // [DECONVOLUTION]-[aten_ops.convolution.default]-[dconv/convolution]
         | n_groups: 1  lpad: {0, 0}  rpad: {0, 0}  pad_mode: 0 strides: {2, 2}  dilations: {1, 1} , No matching rules found for input operand types In compileGraph at /_src/optimize
ERROR:torch_tensorrt [TensorRT Conversion Context]:IBuilder::buildEngineWithConfig: Error Code 10: Internal Error (Could not find any implementation for node {ForeignNode[[DECONVOLUTION]-[aten_ops.convolution.default]-[dconv/convolution]]}. In computeCosts at /_src/optimizer/common/tactic/optimizer.cpp:4284)

Same layer compiled with enabled_precisions={torch.float16} (fp16 weights/input instead of bf16,
otherwise identical) succeeds with rel_err ≈ 0.0015 vs. the PyTorch eager reference (normal fp16
rounding).

Environment

tensorrt	11.0.0.114
torch-tensorrt	2.13.0
torch	2.13.0+cu130
CUDA	13.0
GPU	NVIDIA GeForce RTX 4080 (sm_89)
Python	3.12
OS	Linux

Additional notes

• Only BF16 is affected; FP16 and (implicitly, since it's the historically well-supported
  path) FP32 work for the identical layer/shape.
• Reproduces regardless of output_padding (default 0 here) — this is not the same issue as
  the older, already-fixed pytorch/TensorRT#3352
  (output_padding != 0 crash).
• Shape used in the repro (in_channels=1024, out_channels=512, kernel_size=2, stride=2,
  72x72 -> 144x144 spatial) is the real shape from a production FPN upsampling layer, not
  arbitrarily chosen — this isn't a degenerate edge case.
• I have not yet tried other kernel/stride/channel combinations to determine how narrow the
  affected shape space is; happy to run more combinations if useful for triage.