Skip to content

fix(llm): preserve unloaded llama-swap models during /v1/models discovery - #23

Merged
jentfoo merged 3 commits into
jentfoo:mainfrom
aloneangel45:fix/llamacpp-discover-llamaswap-models
Nov 9, 2026
Merged

jentfoo merged 3 commits into
jentfoo:mainfrom
aloneangel45:fix/llamacpp-discover-llamaswap-models

Conversation

@aloneangel45

@aloneangel45 aloneangel45 commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #19

llama-swap reports "status": {"value": "unloaded"} for models until a request triggers an on-demand swap. Updates parseOpenAIModels to keep models owned by llama-swap (or carrying meta.llamaswap) and populate ModelConfig.Name from meta.llamaswap.name when present.

@jentfoo jentfoo left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the contribution!

Looking at the api spec I think there may be a issue with where the Name is defined. Can you confirm if this was tested?

Comment thread pkg/llm/llamacpp.go Resolved

@jentfoo jentfoo left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! I plan to tag this with #18 and a couple other features in the pipe later this week.

@jentfoo
jentfoo merged commit e33c14f into jentfoo:main Nov 9, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fails to discover models from llama-swap

2 participants