feat: support kimi k25 eagle3. - #2147
Open
wxh571001500 wants to merge 23 commits into
Open
Conversation
wxh571001500
commented
Aug 6, 2026
Contributor
- 补充 VLM 场景下的 Eagle3 推测解码框架,包括 Eagle3 Engine、Master、Worker,以及 target 模型和 draft 模型之间的调度、输入传递和结果校验。
- 新增 Kimi K2.5 Eagle3 draft 模型,实现模型注册、权重加载和 NPU Eagle3 Decoder,并适配 Kimi K2.5 的 MLA、MoE 和张量并行配置。
- 从 Kimi K2.5 target 模型的指定中间层提取三路 hidden states,拼接后作为 Eagle3 draft 模型的输入。
- 适配 VLM 多模态流程,处理图片 embedding、文本 embedding 和 Eagle3 speculative embedding 之间的缓存、槽位分配与状态传递。
- 适配 Kimi K2.5 target/draft 的 KV Cache,包括独立的 draft KV Cache Shape、prefill/decode 状态维护及 speculative token 对应的缓存更新。
- 适配 Kimi K2.5 特有的验证输入布局、DP 场景输入构造,以及 target 和 draft 之间的同步逻辑。
- target 模型同时支持 ACL Graph 和 eager 执行;Eagle3 draft 模型保持 eager 执行。
- 恢复 Kimi K2.5 Eagle3 基线中的概率验收逻辑:使用 target logits 和 draft probabilities 完成 rejection sampling,保证 graph/eager 模式下的推理精度一致。
- 处理了 rebase 到最新 main 后的运行时接口变化,包括 Worker、MTP、DFlash、Graph Warmup 和异构 KV 转发等接口兼容问题,并将相关特殊处理限制在 Kimi K2.5 Eagle3 范围内。
wxh571001500
requested review from
DongheJin,
DragonFive,
JimHsiung,
Kang-Meng,
liujinguang0125,
liutongxuan,
ustcfy,
xiao-yu-chen,
yingxudeng,
yinjiawei01 and
zhang-minchao
as code owners
August 6, 2026 10:03
wxh571001500
force-pushed
the
kimi-k25/feat/eagle3
branch
from
August 6, 2026 10:18
e6cc921 to
67c2f1e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.