Skip to content

[Bug][Linux] /v1/embeddings always fails with qds_device::wait() unexpected command state (Krackan Point, FLM 1.0.6) #752

Description

@ripiox

Problem Description

On Linux, every request to /v1/embeddings fails with the same internal XRT error, deterministically, while chat completions on the same server instance keep working fine.

Request:

curl http://127.0.0.1:52625/v1/embeddings \
  -d '{"model":"embed-gemma:300m","input":"hola"}'

Response (100% of requests):

{"error":"qds_device::wait() unexpected command state"}

Server log shows the request arriving and being accepted, then the error:

[LOG]  Target: /v1/embeddings
    "model": "embed-gemma:300m"
Embedding input[0]: hola
→ {"error":"qds_device::wait() unexpected command state"}

Notes:

Related issues (but not the same symptom): #732 and #661 report embeddings that complete but are wrong/collapsed on Linux; #647 reports non-determinism; #655 reports an IOCTL regression on the same chip family (Krackan Point). This report adds the case where the embed dispatch itself errors out on every request.

Environment

Item Value
Hardware AMD Ryzen AI 5 PRO 340 (Krackan Point), PCI 1022:17f0, Lenovo subsystem
OS Omarchy (Arch Linux), Wayland
Kernel 7.2.5-4-omarchy
NPU FW 1.1.2.64
Device /dev/accel/accel0 with 8 columns
amdxdna in-tree, reported as 0.10 by flm validate
XRT xrt 2.21.75-14 + xrt-plugin-amdxdna 2.21.75-2 (Arch extra)
FLM fastflowlm 1.0.6-1 (Arch extra)
Memlock unlimited (via ulimit wrapper)

flm validate output:

[Linux]  Kernel: 7.2.5-4-omarchy
[Linux]  NPU: /dev/accel/accel0 with 8 columns
[Linux]  NPU FW Version: 1.1.2.64
[Linux]  amdxdna version: 0.10
[Linux]  Memlock Limit: infinity

Steps to reproduce

flm pull llama3.2:1b
flm pull embed-gemma:300m
flm serve llama3.2:1b --embed 1

curl http://127.0.0.1:52625/v1/embeddings \
  -H 'Content-Type: application/json' \
  -d '{"model":"embed-gemma:300m","input":"hola"}'

Expected

A vector array response:

{"data":[{"embedding":[...],...}]}

Actual

{"error":"qds_device::wait() unexpected command state"}

CPU governor/pmode (--pmode balanced) does not change the outcome; also reproduced with pmode default. Model files pass flm check after a fresh verified download.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions