Upstream: ggml-org#29006
--json-schema-file makes chat requests fail during sampler initialization, while the same schema works on /completion or when supplied per request.
Reproduced on spark-a934.local, Linux aarch64, CPU Release, upstream 972d2313bc0bf0a45f634f77d95c9fb03aeab12c, Qwen2.5-0.5B-Instruct Q4_K_M (representative model; not the reporter's Windows/Vulkan setup).
schema.json:
{"type":"object","properties":{"action":{"enum":["ALLOW","DENY","REVIEW"]},"reason":{"type":"string"}},"required":["action","reason"]}
llama-server -m qwen2.5-0.5b-instruct-q4_k_m.gguf -ngl 0 -c 2048 -t 4 -np 1 --host 127.0.0.1 --port 18918 --jinja --json-schema-file schema.json
curl -i localhost:18918/v1/chat/completions -H 'Content-Type: application/json' -d '{"messages":[{"role":"user","content":"Return exactly one JSON object with action ALLOW, DENY, or REVIEW and a short reason."}],"temperature":0,"seed":23,"max_tokens":80}'
Observed: HTTP 400 Failed to initialize samplers: std::exception. The server logs the failing generation prompt as <|im_start|>assistant\n.
Controls: /completion returns 200; removing the CLI schema and supplying json_schema in the chat request returns 200. Passing the exact generated grammar through --grammar-file also returns 200 on both endpoints.
Cause: CLI schema is marked OUTPUT_FORMAT, so common_sampler_init() prefills the assistant header into a grammar that starts with {. A user-supplied GBNF grammar bypasses that prefill. Introducing commit not bisected.
Upstream: ggml-org#29006
--json-schema-filemakes chat requests fail during sampler initialization, while the same schema works on/completionor when supplied per request.Reproduced on spark-a934.local, Linux aarch64, CPU Release, upstream
972d2313bc0bf0a45f634f77d95c9fb03aeab12c, Qwen2.5-0.5B-Instruct Q4_K_M (representative model; not the reporter's Windows/Vulkan setup).schema.json:{"type":"object","properties":{"action":{"enum":["ALLOW","DENY","REVIEW"]},"reason":{"type":"string"}},"required":["action","reason"]}Observed: HTTP 400
Failed to initialize samplers: std::exception. The server logs the failing generation prompt as<|im_start|>assistant\n.Controls:
/completionreturns 200; removing the CLI schema and supplyingjson_schemain the chat request returns 200. Passing the exact generated grammar through--grammar-filealso returns 200 on both endpoints.Cause: CLI schema is marked
OUTPUT_FORMAT, socommon_sampler_init()prefills the assistant header into a grammar that starts with{. A user-supplied GBNF grammar bypasses that prefill. Introducing commit not bisected.