Skip to content

fix(router): NVIDIA inference provider URL path duplication causes 404 #18

Closed

Description

@rhuss

Bug Description

The nvidia inference provider type produces 404 errors from the NVIDIA Build API because of URL path duplication. The provider's default base_url is https://integrate.api.nvidia.com/v1, and the router appends the request path /v1/chat/completions, resulting in https://integrate.api.nvidia.com/v1/v1/chat/completions (double /v1).

The openai provider type works correctly with the same URL pattern, so the path deduplication logic works for OpenAI but not for NVIDIA.

Root Cause

The bug is in crates/openshell-router/src/backend.rs where the upstream URL is constructed by combining the route's endpoint (base URL) with the request path. The path deduplication that strips the /v1 prefix works for the OpenAI provider but does not apply to the NVIDIA provider.

Reproduction Steps

  1. Create an NVIDIA provider:
    openshell provider create --name inference --type nvidia --credential "NVIDIA_API_KEY=nvapi-..."
  2. Create a sandbox and send a chat completion request through inference.local
  3. Observe a 404 page not found response from the NVIDIA API

Verification that the API itself works

Direct curl to the NVIDIA endpoint succeeds:

curl -s https://integrate.api.nvidia.com/v1/chat/completions \
  -H "Authorization: Bearer nvapi-..." \
  -H "Content-Type: application/json" \
  -d '{"model":"meta/llama-3.3-70b-instruct","messages":[{"role":"user","content":"hello"}]}'

This returns a valid response, confirming the issue is in URL construction, not the API itself.

Expected Behavior

The router should deduplicate the /v1 prefix, producing:
https://integrate.api.nvidia.com/v1/chat/completions

Actual Behavior

The router produces:
https://integrate.api.nvidia.com/v1/v1/chat/completions

This results in a 404 page not found from the NVIDIA API.

Environment

  • OpenShell gateway v0.0.98-dev.23 (commit ea690499)
  • Deployed in SAW (Secure Agent Workspace) on OpenShift 4.22.8

Agent Diagnostic

  • Confirmed direct API access works with curl (valid 200 response from meta/llama-3.3-70b-instruct)
  • Traced the routing path: supervisor proxy -> gateway GetInferenceBundle -> router -> NVIDIA API
  • Identified the URL construction in crates/openshell-router/src/backend.rs as the source of the duplication
  • The OpenAI provider handles the same pattern correctly, indicating the fix should extend the existing deduplication logic to cover the NVIDIA provider type

Activity

  1. rhuss commented on Aug 9, 2026

    @rhuss
    OwnerAuthor

    Closing: the root cause claim was wrong.

    The /v1 deduplication in build_backend_url() at crates/openshell-router/src/backend.rs:822-847 handles NVIDIA and OpenAI identically. Both providers use the same code path ((None, false) arm), share OPENAI_PROTOCOLS, and the existing test at line 996 already covers this URL pattern.

    The 404 I experienced during the SAW deployment was real, but the cause was likely:

    • A stale inference bundle (the provider was recreated multiple times during debugging)
    • The sandbox using a cached route from a prior provider configuration
    • Or a transient API key scope issue on the NVIDIA Build API side

    The router code is correct. No fix needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions