podman run --rm --device /dev/kfd --device /dev/dri --group-add video --group-add render --security-opt seccomp=unconfined -p 8080:8080 -v "$PWD/models:/opt/lucebox-hub/server/models" ghcr.io/luce-org/lucebox-hub:rocm
[INFO] Wrote /opt/lucebox-hub/HOST_INFO (source=unknown)
[INFO] Auto-detected target: DeepSeek-V4-Flash-0731-ROCMFPX-MIX-STRIX.gguf
[INFO] Resolved draft dir /opt/lucebox-hub/server/models/draft → /opt/lucebox-hub/server/models/draft/DeepSeek-V4-Flash-0731-DSpark-draft-Q4RMFP4-denseF16.gguf
[INFO] lucebox-hub container starting (target=DeepSeek-V4-Flash-0731-ROCMFPX-MIX-STRIX.gguf, max_ctx=16384, budget=22, lazy=0)
[server] warning: --draft ignored: architecture 'deepseek4' has no speculative decode support
[server] warning: --ddtree ignored: architecture 'deepseek4' has no DDTree speculative decode support
[server] loading tokenizer from /opt/lucebox-hub/server/models/DeepSeek-V4-Flash-0731-ROCMFPX-MIX-STRIX.gguf
[tokenizer] added_tokens: 1283 special tokens
[tokenizer] loaded vocab=129280 merges=127741 bos=0 eos=1 eot=-1 pre=qwen2 sp=no
[server] creating backend...
[backend_factory] detected arch=deepseek4
[deepseek4] gfx1201 hybrid prefill: defaulting hot expert sub-batch to 4
ggml_cuda_init: found 2 ROCm devices (Total VRAM: 130928 MiB):
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32, VRAM: 32624 MiB
Device 1: AMD Radeon Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32, VRAM: 98304 MiB
[deepseek4] heterogeneous target detected; using hybrid expert load path
[deepseek4] model: layers=43 embd=4096 heads=64 head_dim=512 lora_q=1024 lora_o=1024 out_groups=8
[deepseek4] moe: experts=256 used=6 shared=1 ff=2048 hash_layers=3
[deepseek4] attention: swa=128 rot=64 indexer_heads=64 top_k=512 hc=4
[deepseek4] qtype-106 experts not resident on this shard (skip_expert_tensors) — fused decode disabled here
[deepseek4] qtype-105 down-experts not resident on this shard (skip_expert_tensors) — fused decode disabled here
[deepseek4] loaded 1328 tensors, 3593.2 MB GPU buffer, 0.0 MB dense TP split
[deepseek4] hybrid placement: gpu_total=31.86 GiB gpu_free=28.14 GiB core=3.72 GiB kv=0.13 GiB warm=0.25 GiB safety=0.50 GiB expert_budget=27.26 GiB hot/layer=79
[deepseek4] placement expert_memory: total=87.62 GiB uniform_round=350.50 MiB hot=3397 27.04 GiB cold=7611 60.58 GiB
[deepseek4] qtype-106 (mixed ROCmFP2) gate experts cannot decode from hybrid/cold placement; falling back to monolithic full load
[deepseek4] model: layers=43 embd=4096 heads=64 head_dim=512 lora_q=1024 lora_o=1024 out_groups=8
[deepseek4] moe: experts=256 used=6 shared=1 ff=2048 hash_layers=3
[deepseek4] attention: swa=128 rot=64 indexer_heads=64 top_k=512 hc=4
ggml_backend_cuda_buffer_type_alloc_buffer: allocating 93321.23 MiB on device 0: cudaMalloc failed: out of memory
[deepseek4] monolithic fallback failed (model does not fit resident): /opt/lucebox-hub/server/models/DeepSeek-V4-Flash-0731-ROCMFPX-MIX-STRIX.gguf
[deepseek4] hybrid mode failed: /opt/lucebox-hub/server/models/DeepSeek-V4-Flash-0731-ROCMFPX-MIX-STRIX.gguf
[backend_factory] DeepSeek4Backend init failed
[server] backend creation failed
It looks like dflash_server found the wrong device.
podman run --rm --device /dev/kfd --device /dev/dri --group-add video --group-add render --security-opt seccomp=unconfined -p 8080:8080 -v "$PWD/models:/opt/lucebox-hub/server/models" ghcr.io/luce-org/lucebox-hub:rocmIt looks like dflash_server found the wrong device.