Skip to content

server router: CLI API key fails for chat when an API-key file is also used #46

Description

@am17an

Upstream: ggml-org#28820

A router accepts a CLI API key for model listing but rejects it for chat when --api-key-file is also supplied.

Reproduced on spark-a934.local: Linux aarch64, upstream 790cf51aabd61763486050dec7451d9147cb7c61, CPU-only Release build, Qwen2.5-0.5B-Instruct Q4_K_M.

presets.ini:

[*]
model = /home/aman/models/llama-cpp-triage/qwen2.5-0.5b-instruct-q4_k_m.gguf
ngl = 0
ctx-size = 512
parallel = 1
threads = 2
fit = off
load-mode = none
[triage-a]
load-on-startup = true
[triage-b]
load-on-startup = false

keys.txt contains triage-file-one and triage-file-two, one per line.

llama-server --host 127.0.0.1 --port 18913 --offline --no-ui --models-preset presets.ini --models-max 1 --api-key triage-cli --api-key-file keys.txt
curl -i localhost:18913/v1/models -H 'Authorization: Bearer triage-cli'
curl -i localhost:18913/v1/chat/completions -H 'Authorization: Bearer triage-cli' -H 'Content-Type: application/json' -d '{"model":"triage-a","messages":[{"role":"user","content":"Say hi."}],"max_tokens":2,"temperature":0}'

Observed: listing returns 200, chat returns 401 Invalid API Key. Both file keys return 200 for both endpoints. A direct, non-router server accepts all three keys for chat.

Cause: unset_reserved_args() strips LLAMA_API_KEY but preserves LLAMA_ARG_API_KEY_FILE; the spawned child only receives the file source. This filtering was added in 98c1c7a7bf433f1f64dee23f25e7237caab868c9 (ggml-org#18169); runtime introduction has not been bisected.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingserver

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions