Problem Description
On Linux, every request to /v1/embeddings fails with the same internal XRT error, deterministically, while chat completions on the same server instance keep working fine.
Request:
curl http://127.0.0.1:52625/v1/embeddings \
-d '{"model":"embed-gemma:300m","input":"hola"}'
Response (100% of requests):
{"error":"qds_device::wait() unexpected command state"}
Server log shows the request arriving and being accepted, then the error:
[LOG] Target: /v1/embeddings
"model": "embed-gemma:300m"
Embedding input[0]: hola
→ {"error":"qds_device::wait() unexpected command state"}
Notes:
Related issues (but not the same symptom): #732 and #661 report embeddings that complete but are wrong/collapsed on Linux; #647 reports non-determinism; #655 reports an IOCTL regression on the same chip family (Krackan Point). This report adds the case where the embed dispatch itself errors out on every request.
Environment
| Item |
Value |
| Hardware |
AMD Ryzen AI 5 PRO 340 (Krackan Point), PCI 1022:17f0, Lenovo subsystem |
| OS |
Omarchy (Arch Linux), Wayland |
| Kernel |
7.2.5-4-omarchy |
| NPU FW |
1.1.2.64 |
| Device |
/dev/accel/accel0 with 8 columns |
| amdxdna |
in-tree, reported as 0.10 by flm validate |
| XRT |
xrt 2.21.75-14 + xrt-plugin-amdxdna 2.21.75-2 (Arch extra) |
| FLM |
fastflowlm 1.0.6-1 (Arch extra) |
| Memlock |
unlimited (via ulimit wrapper) |
flm validate output:
[Linux] Kernel: 7.2.5-4-omarchy
[Linux] NPU: /dev/accel/accel0 with 8 columns
[Linux] NPU FW Version: 1.1.2.64
[Linux] amdxdna version: 0.10
[Linux] Memlock Limit: infinity
Steps to reproduce
flm pull llama3.2:1b
flm pull embed-gemma:300m
flm serve llama3.2:1b --embed 1
curl http://127.0.0.1:52625/v1/embeddings \
-H 'Content-Type: application/json' \
-d '{"model":"embed-gemma:300m","input":"hola"}'
Expected
A vector array response:
{"data":[{"embedding":[...],...}]}
Actual
{"error":"qds_device::wait() unexpected command state"}
CPU governor/pmode (--pmode balanced) does not change the outcome; also reproduced with pmode default. Model files pass flm check after a fresh verified download.
Problem Description
On Linux, every request to
/v1/embeddingsfails with the same internal XRT error, deterministically, while chat completions on the same server instance keep working fine.Request:
curl http://127.0.0.1:52625/v1/embeddings \ -d '{"model":"embed-gemma:300m","input":"hola"}'Response (100% of requests):
{"error":"qds_device::wait() unexpected command state"}Server log shows the request arriving and being accepted, then the error:
Notes:
flm serve llama3.2:1b --embed 1loads both models successfully (Embedding mode enabled: reserving additional 300MB); failure only occurs at the actual embed dispatch.--embed 1, chat on the same box is fine (96 tok/s prefill / 42 tok/s decode with llama3.2:1b).run_llmfailed in ryzenai 1.4.0 (linux) amd/RyzenAI-SW#177.Related issues (but not the same symptom): #732 and #661 report embeddings that complete but are wrong/collapsed on Linux; #647 reports non-determinism; #655 reports an IOCTL regression on the same chip family (Krackan Point). This report adds the case where the embed dispatch itself errors out on every request.
Environment
1022:17f0, Lenovo subsystem7.2.5-4-omarchy/dev/accel/accel0with 8 columns0.10byflm validatexrt 2.21.75-14+xrt-plugin-amdxdna 2.21.75-2(Archextra)fastflowlm 1.0.6-1(Archextra)flm validateoutput:Steps to reproduce
Expected
A vector array response:
{"data":[{"embedding":[...],...}]}Actual
{"error":"qds_device::wait() unexpected command state"}CPU governor/pmode (
--pmode balanced) does not change the outcome; also reproduced withpmodedefault. Model files passflm checkafter a fresh verified download.