Skip to content

Fix preserving output_router_logits in multimodal template forward hooks - #10190

Merged
tastelikefeet merged 2 commits into
modelscope:mainfrom
hkxxxxx:fix/preserve-output-router-logits
Sep 22, 2026
Merged

tastelikefeet merged 2 commits into
modelscope:mainfrom
hkxxxxx:fix/preserve-output-router-logits

Conversation

@hkxxxxx

@hkxxxxx hkxxxxx commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Summary

Preserve output_router_logits when Template.pre_forward_hook restores model-forward kwargs after multimodal _post_encode.

Fixes #10189.

Background

Native Swift DPO/SFT can enable the MoE router auxiliary loss with:

output_router_logits = True

The Qwen3.5 multimodal template reuses a Qwen2VL post-encoding path that returns a new kwargs dictionary. The generic template hook restores only fields in a manually maintained whitelist. Because output_router_logits was missing from that whitelist, the flag was silently removed before the Hugging Face model forward.

With router_aux_loss_coef > 0, Qwen3.5 MoE then returns aux_loss=None; native DPO fails when it evaluates:

self.aux_loss_coef * model_output['aux_loss']

Changes

  • Add output_router_logits to the forward-kwargs restoration whitelist in Template.pre_forward_hook.
  • Add a regression test covering a post-encode path that rebuilds kwargs while preserving output_router_logits=True.

Validation

  • Reproduced the failure with Qwen3.5/Qwen3.6 MoE native DPO, LoRA, DeepSpeed, and router_aux_loss_coef=1e-3.
  • Confirmed the first Python failure is aux_loss=None, not NCCL initialization.
  • Confirmed the one-line fix produces a numeric aux_loss and allows the training step to continue.
  • conda run -n llm python -m pytest tests/general/test_template_forward_hook.py -q1 passed.
  • python -m py_compile swift/template/base.py tests/general/test_template_forward_hook.py.
  • git diff --check.

Compatibility and risk

This change only preserves a caller-provided forward kwarg. It does not change the default model behavior when the kwarg is absent, and it does not alter the auxiliary-loss coefficient or loss formula.

The field is already used by the native DPO/SFT trainers for supported MoE models. The regression test targets the parameter-loss mechanism directly and does not require model weights or private data.

Checklist

  • Bug Fix
  • Add/update regression test.
  • Run the relevant focused test.
  • Confirm no private paths, logs, credentials, or local environment files are included.
  • Run a full Qwen3.5 MoE smoke test when the target hardware/environment is available.

@tastelikefeet

Copy link
Copy Markdown
Collaborator

Conflicting

@tastelikefeet
tastelikefeet merged commit 8626b85 into modelscope:main Sep 22, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Qwen3.5 MoE multimodal DPO drops output_router_logits and crashes with aux_loss=None

2 participants