Skip to content

bugfix:DBO ubatch NPUGraph capture passes per-stage metadata to the standard full-graph updater - #178

Open
yujuancao07 wants to merge 1 commit into
vllm-project:mainfrom
yujuancao07:bugfix/dbo
Open

bugfix:DBO ubatch NPUGraph capture passes per-stage metadata to the standard full-graph updater#178
yujuancao07 wants to merge 1 commit into
vllm-project:mainfrom
yujuancao07:bugfix/dbo

Conversation

@yujuancao07

Copy link
Copy Markdown
Contributor

Purpose

When AFD runs with DBO, ubatch-size > 1, and full decode NPUGraph capture, the Attention side can fail during graph warm-up with:

AttributeError: 'list' object has no attribute 'keys'

Issue

Signed-off-by: yujuancao07 <yujuancao07@gmail.com>
@jiangkuaixue123

Copy link
Copy Markdown
Collaborator

Thanks for the investigation. The root cause is not the shape of attn_metadata, but that MLA on Ascend has not yet been adapted to DBO + FULL graph.

In the previous AFD v0.13-based implementation, MLA had a dedicated mechanism to update the full-graph parameters. That mechanism should be ported to the current implementation. Skipping _update_full_graph_params_if_needed() for ubatches only avoids the immediate exception; it does not provide the MLA full-graph update semantics that are required.

This is why DeepSeek-V2-Lite cannot currently run with DBO + FULL graph. DeepSeek-V3.2 can run with the same configuration because it uses DSA rather than MLA, so it does not hit this MLA-specific gap.

Therefore, we will not accept the current workaround as the fix. Please port the previous MLA-specific full-graph update mechanism and validate DeepSeek-V2-Lite with DBO + FULL graph.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: DBO ubatch NPUGraph capture passes per-stage metadata to the standard full-graph updater

2 participants