perf(KV-FP8): serve fp8 KV prefill through the bf16 dense dequant - #3360

Open

iiLaurens wants to merge 3 commits into

mudler:mainfrom

iiLaurens:row/KV-FP8

Commits on Sep 29, 2026

spec(KV-FP8): the fp8 KV prefill dense dequant
dev
committed
fca6a1c
perf(KV-FP8): serve fp8 KV prefill through the bf16 dense dequant
dev
committed
a40ba50

Commits on Sep 30, 2026

fix(KV-FP8): release the fp8 dense scratch on every unwind path
dev
committed
b2f195e