[None][fix] Pass num_fused_shared_experts in FP8 MoE fallback-tactic path

[brnguyen2]

Description

PR #15297 added a numFusedSharedExpert argument to
FP8BlockScaleMoERunner::getValidConfigs and updated the autotuner
get_valid_tactics call site, but not the memoized fallback-tactic path used
on autotuner cache misses. Any FP8 block-scale MoE forward that misses the
profiling cache now fails with:

RuntimeError: get_valid_configs() is missing value for argument '_6'.

Observed as a hard failure of
perf/test_perf_sanity.py::test_e2e[aggr_upload-host_perf_deepseek_v3_lite-v3lite_fp8_bs8_128_256]
on two independent pre-merge pipelines whose bases include #15297; the
failure is intermittent because it depends on hitting the autotuner
cache-miss path.

Fix: pass num_fused_shared_experts and include it in the memo key, matching
the get_valid_tactics call site on the same class (the valid-config set
depends on it, so it belongs in the key).

Test Coverage

Covered by the existing DeepSeek V3 Lite FP8 perf-sanity case, which
exercises the fallback path.

PR Checklist

• PR title follows the [None][fix] convention

Dev Engineer Review

• Updated the FP8 MoE fallback-tactic path in tensorrt_llm/_torch/custom_ops/trtllm_gen_custom_ops.py.
• Added num_fused_shared_experts to the getValidConfigs call and memoization key.
• This keeps fallback validation consistent with the autotuner path and prevents cache collisions.
• No public declarations or configuration files changed.
• The change addresses the missing _6 argument runtime error.

QA Engineer Review

No test changes.

[brnguyen2]

/bot run

[tensorrt-cicd]

PR_Github #64333 [ run ] triggered by Bot. Commit: fcbed76 Link to invocation

[coderabbitai]

Walkthrough

The FP8 MoE fallback-tactic cache and validation key now includes num_fused_shared_experts, separating configurations with different fused shared-expert counts.

Changes

FP8 MoE tactic caching

Layer / File(s)	Summary
Include fused shared-expert count tensorrt_llm/_torch/custom_ops/trtllm_gen_custom_ops.py	FP8BlockScaleMoERunner.get_fallback_tactic includes num_fused_shared_experts in its cache and tactic-validation key.

Estimated code review effort: 1 (Trivial) | ~2 minutes

Possibly related PRs

• NVIDIA/TensorRT-LLM#15297: Extends fused shared-expert support in FP8 MoE fallback-tactic cache keys.
• NVIDIA/TensorRT-LLM#17328: Updates FP8 MoE tactic cache and fallback validation keys with num_fused_shared_experts.

Suggested reviewers: bowenfu

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title follows the required format and clearly describes the FP8 MoE fallback-tactic fix.
Description check	✅ Passed	The description explains the issue, root cause, fix, failure mode, and relevant test coverage; the checklist is partially completed but sufficient.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tensorrt_llm/_torch/custom_ops/trtllm_gen_custom_ops.py`:
- Around line 925-926: Update the unique_id() autotuner cache key to include
num_fused_shared_experts, matching the fallback key and get_valid_configs()
parameter validation. Add a regression test that verifies distinct fused
shared-expert counts produce distinct cache identities and do not reuse the same
autotuner entry.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: a41c22c7-f985-4817-9376-3ac92d9f9085

📥 Commits

Reviewing files that changed from the base of the PR and between 4aea8b1 and fcbed76.

📒 Files selected for processing (1)

• tensorrt_llm/_torch/custom_ops/trtllm_gen_custom_ops.py

[coderabbitai]

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Include num_fused_shared_experts in unique_id() too.

The fallback key now separates fused shared-expert counts, but unique_id() at Lines [905-907] still omits this field. The code defines unique_id() as the autotuner cache key, while get_valid_configs() validates this parameter at Lines [968-975]. Different counts can therefore reuse the same autotuner entry and tactic.

Add num_fused_shared_experts to unique_id() and add a regression test covering two counts.

Proposed fix

     def unique_id(self):
-        return (self.top_k, self.intermediate_size, self.local_num_experts,
-                self.act_type)
+        return (self.top_k, self.num_fused_shared_experts,
+                self.intermediate_size, self.local_num_experts, self.act_type)

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

	key = (self.top_k, self.num_fused_shared_experts, hidden_size,
	self.intermediate_size, self.local_num_experts, num_tokens)
	def unique_id(self):
	return (self.top_k, self.num_fused_shared_experts,
	self.intermediate_size, self.local_num_experts, self.act_type)

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tensorrt_llm/_torch/custom_ops/trtllm_gen_custom_ops.py` around lines 925 -
926, Update the unique_id() autotuner cache key to include
num_fused_shared_experts, matching the fallback key and get_valid_configs()
parameter validation. Add a regression test that verifies distinct fused
shared-expert counts produce distinct cache identities and do not reuse the same
autotuner entry.

[leslie-fang25]

Duplicated with #17328?

[tensorrt-cicd]

PR_Github #64333 [ run ] completed with state SUCCESS. Commit: fcbed76
/LLM/main/L0_MergeRequest_PR pipeline #52228 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[brnguyen2]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #64373 [ run ] triggered by Bot. Commit: fcbed76 Link to invocation

[tensorrt-cicd]

PR_Github #64373 [ run ] completed with state SUCCESS. Commit: fcbed76
/LLM/main/L0_MergeRequest_PR pipeline #52264 completed with status: 'SUCCESS'
Pipeline passed with automatic retried tests. Check the rerun report for details.

CI Report

Link to invocation