semantic-router

Go library for learned LLM model routing. Hosts inject embeddings; this package stores traces, fits a profile, and selects a candidate member at request time.

It is closer to RouteLLM than to intent-matching routers such as semanticrouter-go or vLLM Semantic Router:

• semanticrouter-go: predefined utterances → named route
• vLLM Semantic Router: Envoy classifier → backend model
• this library: local traces + pairwise preferences → quality-constrained member

CSGLite is the first host. Evaluation workers, Gateway calls, and Provider Pool failover stay in the host.

Install

go get github.com/opencsgs/semantic-router

Match a query

router, err := semanticrouter.NewRouter(profile, semanticrouter.EncodeFunc(
    func(ctx context.Context, text string) ([]float64, error) {
        return embed(ctx, text)
    },
))
if err != nil {
    return err
}
decision, err := router.Match(ctx, []semanticrouter.Message{
    {Role: "user", Content: query},
})

The encoder is the only model I/O in the hot path. Profiles are immutable JSON artifacts (V1 cluster or V2 pairwise BT/forest).

Fit a profile

Hosts persist traces with Store, run their own candidate/judge calls, then:

profile, err := semanticrouter.BuildRouterProfileV2(semanticrouter.RouterProfileV2Input{
    PoolID: poolID,
    // snapshots, listwise rounds, preferences, frozen costs...
})

Small samples use similarity-weighted Bradley-Terry. Larger samples may train a deterministic pairwise forest, then calibrate a 95% quality-retention threshold.

Layout

File	Role
router.go	Encoder, Router.Match, V1 nearest cluster
curator.go	redaction, routing text, bounded benchmark
store.go	SQLite profiles, jobs, preferences
optimizer.go	V1 k-means profile
pairwise.go / pairwise_forest.go	V2 learner
calibration.go	quality/cost thresholds
profile_v2.go	V2 artifact + Route

License

Apache-2.0