[Feature Request] Improve model management: discovery, multi-provider selection, visibility controls, and fast failure

[str0203]

Summary

When using Pi through Codeg's ACP adapter with an OpenAI-compatible provider, the current setup requires users to type model IDs manually. A small typo can be saved as a configuration value that is difficult to edit or delete. The invalid model is then retried for a long time, making a simple request look like a connection timeout.

This request proposes a safer model-management workflow and clearer handling of deterministic model errors. It is intended to improve the Codeg UI and ACP integration; it does not require changing Pi or the provider's API.

Background and Reproduction

I configured Pi in Codeg to use an OpenAI-compatible relay provider.

• Correct model ID: gpt-5.6-sol
• Mistyped model ID: 5.6-sol

Steps to reproduce:

1. Configure an OpenAI-compatible provider for the Pi Agent.
2. Add a model by manually entering an invalid or mistyped model ID.
3. Start a simple request in Pi.
4. Observe that the request remains busy for about one minute while the invalid model is retried.
5. Correct or remove the invalid model and send the same request again.

The provider reports that the model is unsupported. In the relay logs, the invalid-model request showed a first-token delay of roughly one minute. After the model configuration was corrected, requests returned normally. Switching the protocol to openai-responses was also fast, so the main issue appears to be the invalid saved model and the retry/error-handling path rather than the protocol itself.

Current Problems

1. Manual model entry is error-prone. Users must know and type the exact provider model ID.
2. Saved model values are not easy to manage. There is no obvious, reliable edit/delete workflow for correcting a typo.
3. There is no automatic model discovery. OpenAI-compatible providers commonly expose a model list, but Codeg does not appear to use it during provider setup.
4. The current configuration is too limited for practical use. An Agent should be able to use multiple providers and multiple models without duplicating configuration.
5. Model visibility cannot be managed. Users cannot easily control which models appear in an Agent's model picker.
6. Deterministic configuration errors can be retried like transient failures. An invalid model ID should fail quickly instead of producing a long, opaque processing state.

Proposed Requirements

1. Automatically discover models

For OpenAI-compatible providers, Codeg should support model discovery through the standard endpoints where available:

• GET /models
• GET /v1/models

Suggested behavior:

• Add a Refresh models action in the provider or Agent settings.
• Populate the model picker from the provider response instead of requiring manual typing.
• Store the provider's real model ID separately from the display name.
• Show useful metadata when available, such as display name, context length, capabilities, and availability.
• Show the last successful synchronization time.
• Display a clear, actionable error when discovery fails.
• Keep manual model entry as an explicit fallback for providers that do not implement model discovery.
• Handle base URLs consistently, including providers whose configured URL already contains /v1.
• Never expose API keys in logs or user-facing error messages.

2. Allow multiple providers and models per Agent

The Agent configuration should support a reusable provider-and-model list instead of a single model field:

• Associate multiple providers with one Agent.
• Allow each provider to expose multiple models.
• Select a default provider and default model.
• Switch between configured provider/model pairs from the session model picker.
• Optionally define a fallback priority for providers or models when a request fails.
• Reuse provider credentials and endpoint settings instead of asking users to duplicate them for every model.
• Migrate existing single-provider configurations without breaking current setups.

3. Add model visibility and selection controls

Users should be able to control the model list shown for each Agent:

• Show all discovered models or only an allowlist of selected models.
• Hide individual models without deleting the provider configuration.
• Search and filter models by provider, name, capability, or availability.
• Pin or favorite frequently used models.
• Mark models that disappeared from the latest synchronization as unavailable or stale.
• Apply visibility settings independently per Agent and, where appropriate, per provider.

4. Validate, edit, delete, and fail quickly

To prevent the specific failure mode described above:

• Validate a manually entered model ID against the discovered model list when discovery is available.
• If validation is not possible, show a clear warning that the model could not be verified before saving.
• Every saved model must have an obvious Edit and Delete action.
• If the user deletes the current default model, require a replacement or clear the default safely; never leave an invalid default selection behind.
• Treat known model-configuration errors such as model_not_found, HTTP 400, and HTTP 404 as non-retryable by default.
• Retry only errors that are plausibly transient, such as rate limits (429), server errors (5xx), or temporary network failures.
• Cap retry count and backoff duration.
• Surface the provider's actionable error message instead of leaving the UI in an indefinite or opaque processing state.

Suggested Delivery Order

To reduce implementation risk, the work could be delivered in stages:

1. Add model discovery, refresh, validation warnings, and reliable edit/delete controls.
2. Fix error classification so invalid model IDs fail quickly and do not trigger long silent retries.
3. Add multiple models and multiple providers per Agent, with migration for existing configurations.
4. Add per-Agent/per-provider visibility, search, filtering, and favorites.

Acceptance Criteria

• OpenAI-compatible providers can populate models automatically from a supported models endpoint.
• Discovery failures show a clear error while preserving a manual-entry fallback.
• A single Agent can use multiple providers.
• A provider can expose multiple models.
• A default provider and default model can be selected.
• Users can switch among configured provider/model pairs.
• Saved models can always be edited and deleted.
• Deleting a default model cannot leave an invalid default configuration.
• Models can be shown, hidden, searched, and filtered per Agent/provider.
• Unknown manually entered model IDs produce a clear warning whenever they cannot be verified.
• Deterministic model-not-found errors fail quickly and do not cause long silent retries.
• Request IDs, display names, and provider identity are stored as separate fields.
• Existing configurations migrate without breaking.
• API keys are never included in logs or user-facing errors.

Environment

• Platform: macOS
• Agent: Pi through Codeg's ACP adapter
• Provider: OpenAI-compatible relay
• Invalid model ID: 5.6-sol
• Valid model ID: gpt-5.6-sol

Related: #352