Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,3 +29,7 @@ fixture passes; semantic attack detection has not been independently evaluated.
See `docs/deployment.md` for backup, restore, retention and SSO setup. SDKs remain
source distributions. The release workflows prepare container images, a CLI
archive, checksums and a draft GitHub release; publication is a separate action.

### Standalone CLI

CLI 0.2.0 now classifies with bundled local rules immediately after installation. Docker and a server are optional. Explicit `--semantic` calls TypeSafe directly; configured gateways keep the server client workflow.
85 changes: 51 additions & 34 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,12 +24,12 @@

## What Pyro does

Pyro is a self-hosted policy API and dashboard for teams adding LLM features or
Pyro is a standalone CLI, self-hosted policy API and dashboard for teams adding LLM features or
tool-using agents. Call it before forwarding an untrusted prompt, retrieved
passage, or tool payload. It returns `allow`, `review`, or `block`; your application
must hold review decisions and reject blocked ones before executing work.

Local rules run on your server. Semantic detectors currently use **TypeSafe
Local rules run in the CLI process or on your server. Semantic detectors currently use **TypeSafe
System One** and send inputs to that provider. Your application's LLM can be from
any vendor, but the semantic classifier implementation is currently TypeSafe.
Pyro does not automatically intercept a model or tool call and cannot guarantee
Expand All @@ -54,62 +54,79 @@ See [deployment and data retention](docs/deployment.md), [release/support notes]

## Quick start

Pyro is early-beta software. The CLI connects to a server; installing it does not start one. Local rules need no provider account. Semantic screening uses TypeSafe's hosted API and sends the input there.
### Install and classify — no Docker required

### 1. Start a server, or use an existing instance

Save [compose.yaml](https://delvisor.com/pyro/compose.yaml) and [.env.example](https://delvisor.com/pyro/pyro.env.example) in an empty folder. No source checkout is needed. With Docker Compose installed:
With Node.js 22.13+ and pnpm:

```sh
cp .env.example .env
# Fill in the four required credentials using the template's generation commands.
docker compose up --build -d
pnpm add --global @delvisor/pyro
pyro classify 'Summarize this document.'
pyro classify -- '-----BEGIN PRIVATE KEY-----'
```

Open [the dashboard](http://localhost:3000) and sign in using `ADMIN_PASSWORD` from `.env`. The gateway listens on port 8080 and the management API on 8081. Keep those interfaces private; see [deployment guidance](./SECURITY.md).

### 2. Install the published CLI

Use Node.js 22.13+ and pnpm, or install the same package with your preferred npm-compatible package manager:
CLI 0.2+ bundles the classification engine and local-secrets policy. Expect
`allow` then `block`, with `execution: standalone`. No server, database, account,
policy download or provider key is needed. The synthetic header tests a specific
local rule; it is not a semantic detection benchmark.

```sh
pnpm add --global @delvisor/pyro
pyro config set gateway-url http://localhost:8080
pyro config set control-url http://localhost:8081
pyro auth login
pyro classify --file prompt.txt
pyro classify 'A document to inspect' --profile-file ./my-policy.yaml
pyro doctor --local
```

Use your own server URLs if connecting to an existing instance. CLI 0.2.0 adds `pyro doctor` for connection diagnostics and `pyro doctor --semantic` to check provider configuration. These checks send no prompts; configured credentials do not prove that a provider is reachable or accurate.
Standalone results go to stdout; no input history or background services are
created. A configured gateway URL or `PYRO_API_KEY` selects the existing server
mode. `--local` overrides those settings; `--remote` explicitly uses the server.
See the [CLI guide](packages/cli/README.md) for files, stdin and structured inputs.

### 3. Get a decision without a provider key
### Optional semantic screening

Download [local-secrets.yaml](https://delvisor.com/pyro/profiles/local-secrets.yaml), then run from that folder:
Set `TYPESAFE_API_KEY` in your environment, then run:

```sh
pyro profiles import --file ./local-secrets.yaml
pyro playground 'Summarize this document.' --profile local-secrets
pyro playground 'Example: -----BEGIN PRIVATE KEY-----' --profile local-secrets
pyro classify 'Text to inspect' --semantic
```

Expect `allow` for the first request and `block` for the second. Both use local rules and appear in Activity. The fake header is test data, not a real secret. This verifies integration, not general prompt-injection detection. Imports reject duplicate IDs; skip the import if already installed.
This calls TypeSafe directly using the bundled balanced-assistant policy.
`--semantic` explicitly authorizes sending inputs to that provider and its usage
charges. It needs no Docker or Pyro server. Missing keys fail before the request;
provider failures return an indeterminate verdict and the policy's failure action.
A fail-closed block is not evidence that an attack was detected.

### 4. Enable semantic screening explicitly
### Optional shared dashboard and team workflows

Get a key from [TypeSafe](https://console.typesafe.ai/) and add it in **Settings → Classifier provider**. Hosted screening sends inputs to TypeSafe; review its data terms and usage charges. Download and import [balanced-assistant.yaml](https://delvisor.com/pyro/profiles/balanced-assistant.yaml), then select that profile. Local patterns can flag quoted or educational text; evaluate representative benign and attack examples before enforcement.
For durable jobs, activity history, policy rollouts, evaluations and team access,
use an existing server or save [compose.yaml](https://delvisor.com/pyro/compose.yaml)
and [.env.example](https://delvisor.com/pyro/pyro.env.example) in an empty folder.
No source clone is needed. With Docker Compose installed:

Without a configured provider, a semantic request returns an `indeterminate` verdict and follows the profile's failure policy. A fail-closed `block` is not evidence of an attack. Keep fail-closed behavior for workloads that require it; use the explicit local-only preset to try Pyro without a key.
```sh
cp .env.example .env
# Fill in the four required credentials using the generation commands in the file.
docker compose up --build --wait --wait-timeout 180
pyro config set gateway-url http://localhost:8080
pyro config set control-url http://localhost:8081
pyro auth login
```

### 5. Connect an application
Open [the dashboard](http://localhost:3000) and sign in with `ADMIN_PASSWORD`.
Keep the interfaces private; see [deployment guidance](docs/deployment.md).
Download [local-secrets.yaml](https://delvisor.com/pyro/profiles/local-secrets.yaml),
then import it for server use and create an application key:

```sh
pyro apps create --name 'Support' --default-profile-id local-secrets
# Substitute the ID returned above.
pyro profiles import --file ./local-secrets.yaml
pyro apps create --name Support --default-profile-id local-secrets
pyro keys create --name 'Support backend' --app-id APP_ID
# Set PYRO_API_KEY to the one-time key shown in the response.
pyro classify 'Summarize this document.' --profile local-secrets
# Set PYRO_API_KEY to the one-time key shown above; use the returned APP_ID.
pyro classify 'Summarize this document.' --remote --profile local-secrets
```

Keep the key on your backend. Your application enforces `allow`, `review`, and `block`; Pyro does not automatically intercept model calls. Begin in staging, or record decisions without changing your existing controls. A successful CLI classification exits 0 for any decision; scripts must inspect `action` and `verdict`.
Server semantic profiles use the provider configured in dashboard Settings.
Standalone semantic checks use your local `TYPESAFE_API_KEY`. Your application
must enforce the returned action: continue only on allow, hold review, and reject
block. Successful classification exits 0 for any action; scripts must inspect it.

## Protection profiles

Expand Down
1 change: 1 addition & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ services:
GATEWAY_API_KEY: ${GATEWAY_API_KEY:?Set GATEWAY_API_KEY in .env}
CONTROL_PLANE_SECRET: ${CONTROL_PLANE_SECRET:?Set CONTROL_PLANE_SECRET in .env}
ADMIN_PASSWORD: ${ADMIN_PASSWORD:?Set ADMIN_PASSWORD in .env}
EVENT_RETENTION_DAYS: ${EVENT_RETENTION_DAYS:-30}
OIDC_ISSUER: ${OIDC_ISSUER:-}
OIDC_CLIENT_ID: ${OIDC_CLIENT_ID:-}
OIDC_CLIENT_SECRET: ${OIDC_CLIENT_SECRET:-}
Expand Down
5 changes: 5 additions & 0 deletions docs/releases.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,11 @@ publication, then update website links and the compatibility notes together.

## Support policy

CLI 0.2+ supports standalone classification with no Docker or server. CLI 0.1
was a server client. Publish the tested 0.2 archive before deploying the new
standalone-first website instructions; verify a fresh install runs both local
allow/block examples without a server.

During beta, fixes target the newest published beta. main is development code,
not a release channel. Pin the CLI version, server image digest, PostgreSQL major
version and policy revision in deployments. Keep the previous release archive
Expand Down
69 changes: 69 additions & 0 deletions docs/validation.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,3 +29,72 @@ representative semantic evaluation with explicit provider-cost authorization,
load testing at intended traffic, and verification of published artifacts after
release. Release workflows are prepared; this record does not assert that an npm
version, container image, GitHub release or website deployment was published.

## Standalone CLI follow-up

The packed CLI installs outside the repository with installation scripts disabled,
then returns local allow/block decisions without any server or provider key.
All 20 CLI tests pass, including files/stdin, saved-server overrides, explicit
semantic consent and a fixture intercepting direct TypeSafe calls. The full
workspace check also passes. No live TypeSafe request was made.

## Earlier validation history

# Validation for the dashboard, profiles, outgoing webhooks and SDKs

Webhook form follow-up (2026-09-23):

- Replaced comma-separated resource IDs with named, searchable application/profile checklists built from shadcn/ui Radix primitives. Added field help, explicit All scopes, inline risk/action validation and loading/retry feedback.
- Dashboard tests: 12 passed. New coverage checks ID serialization, explicit wildcard selection, preservation of unavailable saved references and disabled webhook state, unchanged destination handling when editing, and risk/action validation. Dashboard production build and whitespace checks passed.
- Verified live application/profile choices, search with and without matches, multiple selection, keyboard toggling, Escape dismissal, focus return and minimum-risk help on localhost. Clearing the final specific selection disables Save. Verified the migrated shared checkbox on Applications without saving changes.
- Checked both themes and a 390px viewport. Corrected nested dialog scrolling so only the form body scrolls; header/footer and checklist contents stay within the viewport. Restored light mode and desktop size, and cancelled test drafts without creating a webhook.
- Restarted Vite on `127.0.0.1:3000` to clear stale imports after adding dependencies. No new runtime errors were observed after restart.

Webhook presentation follow-up (2026-09-23):

- Renamed product copy and navigation references to Webhooks across the dashboard, website and guides; retained technical descriptions of outgoing delivery.
- Removed the local receiver help panel. Delivery history now uses the card width, separates timestamps, labels HTTP responses, shows status badges and only includes an Action column when a delivery can be retried.
- Dashboard and website production builds passed, along with `git diff --check`. Verified Refresh, Add webhook, light/dark presentation and a 390px viewport against existing local delivery history. The table scrolls inside its card without page overflow; no browser errors or warnings were observed.
- Restored light mode and the desktop viewport. Restarted the standalone website on `127.0.0.1:3100` and verified its updated home and documentation copy. All services remain local.

Settings, navigation and library follow-up (2026-09-23):

- Dashboard tests: 8 passed, covering preference migration/validation, library filtering and response defaults, stable detector identity, and historical API response handling. Production dashboard build and whitespace checks passed.
- Verified all six dashboard preference controls, persistence after reload, reset to defaults, System theme selection, compact table cell padding, full Activity timestamps and the neutral-black dark palette (`#050505` canvas). The existing classifier settings remain available separately.
- Verified the restored dropdown opening animation and 180ms sliding highlight, arrow-key selection and focus return inside a profile dialog. Reduced motion suppresses transitions.
- Verified profile library text search, local-only filtering, empty results, YAML inspection and customization into an editable draft. Cancelled the draft without modifying saved policies.
- Verified Playground is under Observe, there is no Test group, Settings has a separate sidebar entry and sidebar hover fills are removed.
- Settings and the library fit a 390px viewport without page overflow. Restored the desktop viewport and default personal preferences after testing. Everything remains on localhost.

Dashboard redesign verification (2026-09-23):

- Dashboard unit tests: 4 passed, including detector editor identity and payload serialization. The editor-only row key stays stable while its API ID changes and is omitted from saves.
- Production dashboard build and `git diff --check`: passed.
- Verified continuous character-by-character typing in both a new detector and an existing detector on `localhost:3000`; full values appeared and focus stayed in the ID input. Cancelled both drafts without modifying saved policies.
- Verified shared dropdown keyboard selection and focus restoration inside the profile dialog. Confirmed Activity filters and request trace dialogs still work.
- Verified neutral light and dark themes, self-hosted Open Sans, themed charts/dialogs, and dark preference persistence after reload. Returned the dashboard to light mode.
- Navigated all nine dashboard pages successfully. A fresh final reload produced no browser console errors or warnings.
- At a 390px viewport, checked collapsible navigation, profile dialogs, Overview, Usage, Applications, Activity and outgoing webhooks. Corrected Applications overflow; those pages fit the viewport, with tables scrolling within their containers. Restored the desktop viewport afterward.

The shared UI conventions and repeatable browser checks are documented in `apps/dashboard/README.md`.

Latest local verification (2026-09-23):

- Reproduced the Protection Profiles white screen on the actual `localhost:3000` dashboard: older backend records omitted `localRules`. Schema defaults now normalize those records before rendering/editing. A page error boundary keeps navigation available if any page fails.
- Verified Activity against the existing database: the list, filters and an existing request trace render. Historical traces without detector arrays and responses without label catalogs have regression coverage; failed requests are shown in the page.
- Verified existing profile cards, the existing profile editor, all four curated presets, outgoing webhook configuration and delivered history in the browser after updating the running services.
- `npm run typecheck`: passed.
- `npm test` with `TEST_DATABASE_URL` pointing to an isolated PostgreSQL 17 instance: 42 tests passed, none skipped. This includes three dashboard compatibility regressions and a worker test ensuring unsupported persisted destinations cannot enqueue or send.
- `docker compose build gateway control-plane`: passed, including all package/application production builds. Both containers were recreated from the new images and report healthy. The existing Vite dashboard remains on port 3000; PostgreSQL data was retained.
- `npm run test:webhook` against the actual local Docker gateway/control plane: passed. Verified HMAC on `integration.test` and a real local-rule `decision.created` event; a deliberate HTTP 503 caused a retry and HTTP 204 completed delivery. Temporary destination/profile removed; audit records retained. All traffic stayed local, with no model request.
- The standalone website passed its Next.js production build and was restarted on `127.0.0.1:3100` with outgoing-webhook-only copy.
- `git diff --check`: passed.

Earlier verification for the unchanged SDK/profile work:

- PostgreSQL checks cover atomic event/outbox writes, duplicate suppression, concurrent claims, expired leases, stale-worker acknowledgements, manual retry and rollback on an invalid delivery.
- Rust SDK: four tests and Clippy with warnings denied passed. Existing Python SDK test passed.
- TypeScript SDK/contracts packed and installed in a separate temporary consumer; imports and a request succeeded.
- Website desktop/mobile layouts, interactive examples, documentation navigation and YAML downloads checked.

The optional observability overlay and non-webhook adapter have been removed. Neither SDK has been published. The website now lives in the sibling `website` directory alongside Delvisor's homepage, with Pyro at `/pyro` and its quickstart at `/pyro/docs`.
Loading
Loading