Think managed kv

[ruota]

Allow managed KV override for CUDA

Summary

Add the documented DS4_CUDA_FORCE_MANAGED_KV=1 opt-in for CUDA. It forces
only the long-lived KV allocation into CUDA managed memory, while model and
optional DSpark support weights remain resident in VRAM.

This makes it possible to run long-context experiments where the weights fit
in VRAM but the KV allocation would otherwise prevent session creation.

The change also includes <float.h> so the CUDA backend has the standard
FLT_MAX definition it already uses.

Scope

• Opt-in only: ordinary CUDA sessions continue to use device-only KV.
• Only the exact value DS4_CUDA_FORCE_MANAGED_KV=1 enables the override.
• Does not change Metal, ROCm, SSD streaming, tensor parallelism, or model
  loading behavior.
• Intended for long-context CUDA experiments such as Think Max with DSpark.

Testing

Builds completed on an NVIDIA RTX PRO 6000 Blackwell (sm_120):

• make cpu
• make cuda CUDA_ARCH=sm_120
• make cuda-regression CUDA_ARCH=sm_120
• git diff --check

Runtime validation used DeepSeek V4 Flash IQ2 imatrix (80.76 GiB) plus the
DSpark support GGUF (5.58 GiB) at ctx=393216:

DS4_CUDA_Q8_F16_CACHE_MB=0 \
DS4_CUDA_FORCE_MANAGED_KV=1 \
./ds4 --cuda -m ./ds4flash.gguf \
  --mtp ./gguf/DeepSeek-V4-Flash-DSpark-support.gguf \
  --dspark --think-max --ctx 393216 --prefill-chunk 256 \
  --tokens 32 --temp 0 -p "..."

The run loaded the main model and DSpark support resident in VRAM, placed the
5.08 GiB KV allocation in managed memory, and completed at 415.66 prefill t/s
and 59.46 generation t/s in the CLI benchmark. A warm server request completed
64 tokens in 1.2667 seconds (about 50.5 end-to-end tokens/s).

The DSpark acceptance fixture was also run with the existing local GGUFs and a
256-token prefill chunk so the resident main and support models fit on the 96 GB
GPU. Managed-KV and device-KV runs produced matching target/DSpark output and
the same proposal statistics. Four cases accepted every proposed draft; the
existing c_add proposal-quality guard failed in both modes because confidence
0.9 produced no draft for that case. This is therefore not caused by the
managed-KV override.

The stock dspark-verify-depth fixture uses a fixed 4096-token prefill buffer
and cannot create its sessions with both resident GGUFs on this 96 GB GPU.

Notes

This is an explicit capacity override. Managed memory can reduce throughput if
the KV pages are not resident, so it is not enabled automatically.