Conversation
Contributor
|
[For maintainers] Suggested jobs to run (before merge) run-slow: hunyuan_vl |
Contributor
CI recapDashboard: View test results in Grafana |
1 task
zucchini-nlp
approved these changes
Sep 21, 2026
zucchini-nlp
left a comment
zucchini-nlp
left a comment
Member
There was a problem hiding this comment.
Interesting, can you also fix the prev hunyuan which i think uses the same rope type?
Comment on lines
+574
to
577
| # Explicit decorator replaces Llama's inherited `@dynamic_rope_update`. HunYuan VL keeps | ||
| # `rope_type="dynamic"` for the NTK-alpha init path, but those frequencies are static. | ||
| @torch.no_grad() | ||
| def forward(self, x, position_ids): |
Member
There was a problem hiding this comment.
oh wow, i didn't notice that model has special scaling. That means we fix it in Hunyuan-v2-dense as well no?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
HunYuanVLTextConfig maps rope_type "xdrope" onto "dynamic" so that HunYuanDenseV1RotaryEmbedding.init takes the NTK-alpha branch, where base = rope_theta * alpha ** (d / (d - 2)) is folded into inv_freq once at init. Those frequencies are static; "dynamic" is only used here to select the formula.
The generated forward, however, inherited @dynamic_rope_update from LlamaRotaryEmbedding. Once a sequence passes max_position_embeddings the decorator recomputes inv_freq through ROPE_INIT_FUNCTIONS["dynamic"], which reads rope_theta and factor and never looks at alpha. For tencent/HunyuanOCR (alpha=1000.0, factor=1.0 - the latter only present to satisfy rope validation) this moves the high-frequency end of inv_freq by ~250x, so long inputs silently get a corrupted frequency table. The check that guards the update also compares a CUDA tensor against a Python int on every forward, which forces a device-to-host sync and makes the module impossible to capture in a CUDA graph.
Declare the decorator stack explicitly on the override so the modular converter no longer copies Llama's, and add a regression test asserting inv_freq survives positions beyond max_position_embeddings.