Chat Server

• Setup
  • Configure
• Client
  • Endpoints
  • VitePress Plugin
• Development
• Support
• Contributing

Proxy Chat Server designed to work with the VitePress Chat plugin.
Secure your API key while live-streaming responses to the client.

Works with Claude, Gemini, OpenAI, or any OpenAI Compatible Provider.

💯 100% Free to use with Zen OpenCode or Gemini Free Tier!

To get started Setup and Configure the server. No API Key required!

• Client: https://github.com/cssnr/vitepress-chat
• Server: https://github.com/cssnr/chat-server

Features

• Works with Claude, OpenAI, Gemini and OpenAI Compatible Providers
• Includes Chat, Completion, and Object Endpoints
• Supports Multiple Clients Simultaneously
• Live Streams the Results to the Client
• Automatic Input Token Caching
• Automatic Retry on API Failures
• Deploy with Docker or Node
• Plus all the Client Features

Built with the AI SDK.

Setup

💡 The server works out-of-the-box with NO environment variables.

With Docker.

docker run -p 3000:3000 cssnr/chat-server:latest

With Docker Compose.

services:
  chat:
    image: cssnr/chat-server:latest
    environment:
      MODEL: 'gemini-2.5-flash'
      GOOGLE_GENERATIVE_AI_API_KEY: 'xxx'
    ports:
      - '3000:3000'

With Node.

npm i
npm start

Note: you will need to export your environment variables or set them in the settings.env file.

For a Docker Swarm + Traefik + Basic Auth example see the docker-compose-swarm.yaml.

For a Portainer Deploy workflow see the .github/workflows/deploy.yaml.

Configure

💡 All variables are optional. The default big-pickle model works with NO API Key.

Environment Variables (can be placed in a settings.env file).

Variable	Default	Description
MODEL	big-pickle	Model to Use
BASE_URL	https://opencode.ai/zen/v1	OpenAI Compatible Provider Base URL
MAX_TOKENS	-	Max Output Tokens
PROVIDER_OPTIONS	-	Provider Options JSON String
PROVIDER_USER_AGENT	see below	OpenAI Compatible Provider User-Agent
INSTRUCTIONS_CHAT	-	System Instructions for Chat
INSTRUCTIONS_COMPLETION	-	System Instructions for Completion
INSTRUCTIONS_OBJECT	-	System Instructions for Object
DISABLE_CLIENT_INSTRUCTIONS¹	-	Ignore Client System Instructions
AI_SDK_LOG_WARNINGS¹	-	Enable SDK Warnings Logging
CORS_ORIGINS	-	Allowed CORS Origins (supports *)
PORT	3000	Server Port
DEBUG	-	Set to app for Debug Logging

¹ Boolean Variables. True values include: ['1', 't', 'true', 'y', 'yes', 'on']

You must also set the API key for the MODEL you select.

Variable	Description
ANTHROPIC_API_KEY	Claude Models
GOOGLE_GENERATIVE_AI_API_KEY	Gemini Models
OPENAI_API_KEY	OpenAI Models
PROVIDER_API_KEY	OpenAI Compatible Provider

The PROVIDER_API_KEY is optional for free-tier models like big-pickle.

INSTRUCTIONS

There are mechanisms to override the instructions per-call for all clients on all endpoints.
These are used as fallback when those instructions are not sent for configurations where this is desired.

The INSTRUCTIONS variable (legacy) also points to the INSTRUCTIONS_CHAT variable (recommended).

To disable the clients ability to send custom instructions set DISABLE_CLIENT_INSTRUCTIONS=true

PROVIDER_OPTIONS

Provider Options: https://vercel.com/docs/ai-gateway/models-and-providers/provider-options

For example, to disable "Reasoning" on big-pickle model.

{ "zen": { "thinking": { "type": "disabled" } } }

You may need to wrap the variable in single quotes.

PROVIDER_OPTIONS='{"openai":{"serviceTier":"flex","reasoningEffort":"low"}}'

You are responsible for providing valid options for the chosen model. The SDK supports providing provider options for multiple providers simultaneously. The value is only checked for valid JSON at startup and will fail at runtime if it contains invalid options.

PROVIDER_USER_AGENT

If BASE_URL, MODEL, and PROVIDER_API_KEY are not set (Default Zen), a custom header is set:

User-Agent: opencode/version

This header increases the rate limit for request for the free OpenCode Zen endpoint.

To disable this without changing one of the above variables, you can set an empty value:

PROVIDER_USER_AGENT=

Otherwise, you can set a custom User-Agent (prefix) to anything you choose:

PROVIDER_USER_AGENT='my-app/1.0'

NOTE: The AI SDK appends a suffix to the UA: <userAgent> ai-sdk/provider-utils/x runtime/node

Client

Endpoints

Endpoint	Method	Description
/chat	POST	Use with useChat and VitePress Chat
/completion	POST	Use with useCompletion
/object	POST	Use with useObject

Note: The / endpoint (legacy) also points to the /chat endpoint (recommended).

chat

Reference: https://ai-sdk.dev/docs/reference/ai-sdk-ui/use-chat

import { useChat } from '@ai-sdk/vue'
import { DefaultChatTransport } from 'ai'

const { messages, sendMessage, status, stop } = useChat({
  transport: new DefaultChatTransport({
    api: 'https://chat-server.cssnr.com/chat',
    headers: { Authorization: 'Basic Abc123=' },
    body: { instructions: 'You are a helpful assistant.' },
  }),
})

To send System Instructions from the client, add them to the body.

completion

Reference: https://ai-sdk.dev/docs/reference/ai-sdk-ui/use-completion

import { useCompletion } from '@ai-sdk/vue'

const { completion, complete, isLoading, stop } = useCompletion({
  api: 'https://chat-server.cssnr.com/completion',
  headers: { Authorization: 'Basic Abc123=' },
  body: { instructions: 'You are a helpful assistant.' },
})

await complete('Explain how to set up cssnr/chat-server')

To send System Instructions from the client, add them to the body.

object

Reference: https://ai-sdk.dev/docs/reference/ai-sdk-ui/use-object

import { useObject } from '@ai-sdk/vue'
import { z } from 'zod'
import { zodToJsonSchema } from 'zod-to-json-schema'

const schema = z.object({ name: z.string(), age: z.number() })

const { object, submit } = useObject({
  api: 'https://chat-server.cssnr.com/object',
  headers: { Authorization: 'Basic Abc123=' },
  schema,
})

submit({
  instructions: 'You are a helpful assistant.',
  prompt: 'Extract the name and age from: John is 30 years old.',
  output: zodToJsonSchema(schema),
})

To send System Instructions and Output Schema from the client, add them to the body.

Note: Both instructions and output are custom body parameters parsed by the server.

VitePress Chat Plugin

The client is currently available as a VitePress Plugin.

• https://github.com/cssnr/vitepress-chat

Development

To enable debug logs set: DEBUG=app

This works with no configuration using the big-pickle model.
You can set your environment variables in the settings.env file.
If using big-pickle for testing it is much faster to disable reasoning.

PROVIDER_OPTIONS='{"zen":{"thinking":{"type":"disabled"}}}'

In all cases you can set the PORT environment variable.

With Node run.

npm run dev

Point your client to: http://localhost:3000/

With Docker compose (you may need to touch settings.env).

docker compose -f docker-compose-dev.yaml up --watch --build --remove-orphans

Point your client to: http://localhost/

Building

To build and test the docker image run.

bash build.sh
docker compose up

Support

If you run into any issues or need help getting started, please do one of the following:

• Report an Issue: https://github.com/cssnr/chat-server/issues
• Q&A Discussion: https://github.com/cssnr/chat-server/discussions/categories/q-a
• Request a Feature: https://github.com/cssnr/chat-server/issues/new?template=1-feature.yaml
• Chat with us on Discord: https://discord.gg/wXy6m2X8wY

Contributing

Please consider making a donation to support the development of this project and additional open source projects.

For a full list of current projects visit: https://cssnr.github.io/