Skip to content

Commit fb03373

Browse files
TheGreatAxiosTheGreatAxios
committed
docs: user manual, design docs, skills and screenshot
1 parent b371919 commit fb03373

9 files changed

Lines changed: 1302 additions & 0 deletions

File tree

‎docs/ARCHITECTURE.md‎

Lines changed: 213 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,213 @@
1+
# Builder: architecture
2+
3+
How Builder is structured and why. What it does for a person is in
4+
[PRODUCT.md](https://github.com/corbitsdev/corbits-builder/blob/main/docs/PRODUCT.md);
5+
the concrete technology and layout are in
6+
[IMPLEMENTATION.md](https://github.com/corbitsdev/corbits-builder/blob/main/docs/IMPLEMENTATION.md).
7+
8+
## Overview
9+
10+
Builder is a control plane (the hub) on top of an agent runtime (Interchange).
11+
The runtime owns agents, workflows, mail between them, approvals, credentials,
12+
sandboxed workers and run lifecycles. The hub adds what is specific to
13+
Builder: projects, versioned documents, the tools agents use to file and hand
14+
off work, the app repository and live preview, preferences, and an API for the
15+
interface.
16+
17+
```
18+
person -> interface -> client API -> hub -> runtime (workflows, mail, approvals)
19+
| |
20+
| +-> sidecars running the agents
21+
|
22+
+-> documents, app repository, live app
23+
```
24+
25+
## Components
26+
27+
### The hub
28+
29+
The hub is the server and the system of record. It:
30+
31+
- signs people in and holds tenancy: an install has workspaces, and each
32+
project is a tenant of its own under its workspace, holding the project's
33+
principals, documents, approvals and runs;
34+
- creates projects and starts each project's coordinator;
35+
- stores documents and their versions, and records which versions an approval
36+
covered;
37+
- provides the tools agents call to read and file documents, hand off work and
38+
ask the person things;
39+
- hosts each project's app repository and runs its live preview;
40+
- answers the interface's API and streams run progress to it.
41+
42+
It embeds the runtime's control plane in the same process and composes it
43+
directly. There is no separate embedding layer.
44+
45+
### Agents and roles
46+
47+
Each role is a workflow package: an agent definition (prompt, tools,
48+
capabilities, model tier) and a workflow definition (trigger, steps, lifetime).
49+
50+
- **Coordinator**: one per project, long-lived. Talks to the person, writes the
51+
problem brief, hands off work and asks for decisions.
52+
- **Writer**: one bounded write of one document of a named kind.
53+
- **Reviewer**: one bounded review of one document; its findings are advice
54+
mailed to the coordinator.
55+
- **Designer**: long-lived, one per design concept. Builds a clickable HTML
56+
and CSS mockup in its workspace.
57+
- **Coder**: long-lived, one per project. Builds the whole app in its
58+
workspace and takes later changes there.
59+
60+
All roles are triggered by mail. The writer and reviewer take exactly one
61+
message and end. The coordinator, designers and coder keep their warm agent
62+
and workspace and take each later message as another turn on the same state.
63+
64+
Only the coordinator talks to the person. Specialists put questions in their
65+
mail to it, and it decides whether to ask the person.
66+
67+
### Documents and versions
68+
69+
A document has a kind, a title and a sequence of versions. Each version has
70+
content and a content hash, and may carry open questions for the person and,
71+
for the approaches, structured options for side-by-side comparison. The kinds
72+
are the problem brief, constraints, approaches, product requirements, build
73+
plan, design options, the design and the app.
74+
75+
- Text documents are written whole by the coordinator or a writer.
76+
- A design option is filed from a designer's workspace once the design check
77+
passes; the design is filed when the person chooses an option.
78+
- The app is filed from the coder's workspace each time the app check passes.
79+
80+
The hub assigns each version's id, number and hash. Writing a version is never
81+
an approval. A writer gets the documents it builds on in full inside its
82+
assignment and has no read tool, so it works only from what it was given.
83+
84+
### Approvals
85+
86+
Approvals are the runtime's own human approvals. A tool that needs a person's
87+
go-ahead (`request_approval`, `design_app`, `instruct_coder`, adding a package)
88+
is declared as asking, so the runtime parks the call until a person resolves
89+
it. After it resolves, the hub records the exact document versions and hashes
90+
it covered, and only those whose hash still matches storage.
91+
92+
- Versions and hashes come from the hub, never from model input, so an
93+
approval cannot be pointed at a version the person did not see.
94+
- Choosing a design option and importing a project as approved are the
95+
person's own acts; the hub records that person as the approver.
96+
- Hand-off tools check approvals before they start the next writer.
97+
98+
### Questions
99+
100+
`ask_questions` puts a structured card (up to five questions with options and a
101+
recommended answer) in front of the person and returns at once. The answers
102+
reach the coordinator as the person's next message. A document's open
103+
questions carry over into such a card. Questions are never modelled as
104+
approvals.
105+
106+
### The app repository and live preview
107+
108+
Each project has one git repository for its app. A coder run gets push access
109+
to its main branch, scoped to that run and revoked when the run ends. A new
110+
coder workspace starts from main when main holds a version, else from the last
111+
filed app, else from the app starter.
112+
113+
The live preview runs the project's app in its own sandboxed process per
114+
project. While a coder works, it follows the coder's workspace; afterwards it
115+
runs the latest filed app. It installs packages when they change, restarts on
116+
file changes, restarts a crash once and then waits for the next change, and
117+
stops when no one has had the project open for a short while. Each project's
118+
app is served from its own origin, and a request reaches it only when the hub
119+
would show that person the project; the hub's cookies never reach the app.
120+
121+
### Sandboxes
122+
123+
The designer and coder run on sidecars that offer a POSIX runtime. Their shell,
124+
file and browser tools run inside a sandbox with a file policy (the workspace
125+
is writable, filed inputs and the design are read-only) and a network policy
126+
(the package registry only, or the public internet when the person allows it).
127+
Browser tools reach only the app running in the same sandbox.
128+
129+
### Preferences
130+
131+
Preferences are a registry of named choices, each with options and a default,
132+
and the roles it guides. A person's choices live in their settings; a project
133+
freezes its creator's choices and model profile when it starts. The coordinator
134+
reads the resolved preferences as guidance; the hub reads them where they
135+
change behaviour (approval gating, package installs, network access, design
136+
concept count).
137+
138+
### The client API and interface
139+
140+
The client API is a typed library over the hub's HTTP API. The interface uses
141+
only that library and the shared contract types; it never imports server code
142+
or the database. Every action in the interface is an authenticated API call
143+
that other clients can make too.
144+
145+
## Control flow and data flow
146+
147+
1. The person creates a project. The hub creates its tenant, installs the role
148+
packages on the project's model bindings and deploys the coordinator.
149+
2. The person's messages are mail to the coordinator. Its replies are mail
150+
back, shown as the conversation.
151+
3. The coordinator files the brief itself, then hands work off with hub tools:
152+
`deploy_writer` and `review_artifact` for documents, `design_app` for design
153+
concepts, `deploy_coder` once for the app and `instruct_coder` for later
154+
changes.
155+
4. The hub builds each assignment from its own templates: the task, the
156+
documents it builds on in full and pinned by version and hash, and the kind
157+
to write. The tool checks prerequisites first: earlier documents exist and
158+
are approved, and no coder is already running.
159+
5. The specialist works, files its result as a document version and mails a
160+
short summary back on the assignment's thread.
161+
6. The coordinator reads the filed document, judges it and either has it
162+
revised, asks the person, or asks for approval.
163+
7. The interface reads documents, approvals, run state and the live app
164+
through the API, and progress through a run event stream.
165+
166+
Work is handed off by mail and results are filed as documents. Mail carries
167+
the conversation; documents carry the work.
168+
169+
## Design decisions
170+
171+
- **No fixed stages.** The coordinator decides the order of work from the
172+
document list and the conversation. The order documents build on each other
173+
is enforced as prerequisites in the hand-off tools, not as a stage machine.
174+
- **Agent behaviour lives in agent definitions and tools.** Prompts describe
175+
the task and the domain. Turn order, reply termination and loop detection are
176+
the runtime's job. Hub code covers human decisions and tenant state.
177+
- **Use the runtime's machinery.** Loops, retries, waits, timeouts, mail,
178+
approvals and failure records come from the runtime rather than being
179+
rebuilt in the hub.
180+
- **Native state over model output.** Progress, the person's next step,
181+
in-flight specialists and decisions are read from run status, run events and
182+
approval records, never from parsing what a model wrote.
183+
- **Deterministic code over model turns.** Anything code can do, code does:
184+
assignment messages, design and app checks, starting the app, filing
185+
snapshots, notices when a specialist ends.
186+
- **Exact versions everywhere.** Assignments pin documents by version and
187+
hash; approvals bind to the hash the hub computed.
188+
- **Model-agnostic.** Roles are bound to tiers, and a profile maps tiers to a
189+
provider's models, so any connected provider can run any role.
190+
- **Configuration holds wording.** User-visible names and text live in
191+
configuration, so a rename is a configuration change.
192+
193+
## Failure handling
194+
195+
- **Refusals are results.** A tool that cannot act returns the reason and what
196+
to do instead (a missing or unapproved document, a coder already running),
197+
and the underlying error code passes through unchanged.
198+
- **Run outcomes come from the run.** A failed run's cause is read from its
199+
own terminal events.
200+
- **A specialist that ends without answering** is noticed by a reconciler,
201+
which mails the coordinator once with the reason so it can start the work
202+
again.
203+
- **A coordinator that ended** is replaced with a fresh run when the project is
204+
next opened; the earlier runs and filed documents are carried to it as
205+
context with the person's next message.
206+
- **Coordinator upgrades** wait for a safe point (parked on input, nothing
207+
pending, no specialist at work) before replacing the run.
208+
- **Runaway model output** that repeats one passage is cut off, and long
209+
histories are compacted to fit the model's context.
210+
- **Unavailable is a state.** A project with no usable model provider says
211+
why and waits; nothing falls back silently to another provider.
212+
- **Capacity.** Work is placed on a sidecar with room; a placement refused for
213+
room is retried by the runtime.

‎docs/DEV.md‎

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
# Local development
2+
3+
Setup is in
4+
[CONTRIBUTING.md](https://github.com/corbitsdev/corbits-builder/blob/main/CONTRIBUTING.md#setup);
5+
running with Docker Compose is in
6+
[SELF_HOST.md](https://github.com/corbitsdev/corbits-builder/blob/main/docs/SELF_HOST.md).
7+
8+
## Architecture in brief
9+
10+
| Path | What it is |
11+
| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
12+
| `vendor/interchange` | [Interchange](https://github.com/faremeter/interchange) at the commit in `vendor/VENDORED_REVISION`, plus local patches recorded in `vendor/interchange.manifest.json`. It owns the agent loop, mail, spawning, approvals, credentials, sidecars and run lifecycle. |
13+
| `apps/hub` | The server: a Postgres control plane, tenancy and sign-in, documents (versioned artifacts with content hashes), approvals and the glue that starts each role. It runs Interchange in-process. |
14+
| `packages/client` | A typed client for the hub's API (`@corbits-builder/client`). It never touches the database or Interchange internals. |
15+
| `apps/web` | The UI: Vite and React over that client, with its own small router in `src/routes.ts`. |
16+
| `workflows/*` | One package per role: its agent, prompt, tools and workflow definition. |
17+
| `packages/schemas` | Types and schemas shared by the hub, the client and the workflows. |
18+
| `scripts/` | Dev, loop and verification scripts. |
19+
| `deploy/` | The Dockerfile and Compose files ([SELF_HOST.md](https://github.com/corbitsdev/corbits-builder/blob/main/docs/SELF_HOST.md)). |
20+
21+
The roles:
22+
23+
- **Coordinator** (`workflows/coordinator`): one long-running workflow per
24+
project. It talks with the person, writes the problem brief and starts the
25+
other roles.
26+
- **Writer** (`workflows/writer`): writes one document.
27+
- **Reviewer** (`workflows/reviewer`): reviews one document.
28+
- **Designer** (`workflows/designer`): builds the clickable design.
29+
- **Coder** (`workflows/coder`): builds the app in a sandbox from its
30+
starter.
31+
32+
Each role runs on the tier its profile assigns
33+
([apps/hub/src/config/models.ts](https://github.com/corbitsdev/corbits-builder/blob/main/apps/hub/src/config/models.ts)), so any
34+
connected provider can run any role. The project's conventions are in
35+
[AGENTS.md](https://github.com/corbitsdev/corbits-builder/blob/main/AGENTS.md), and the Interchange doc for each area in
36+
[IMPLEMENTATION.md](https://github.com/corbitsdev/corbits-builder/blob/main/docs/IMPLEMENTATION.md#interchange-docs-map).
37+
38+
## Model profile choice
39+
40+
There is no default profile. When a project's coordinator starts, it runs on:
41+
42+
1. The profile the project was created with, when this server can run it.
43+
2. Else the person's chosen profile under **Settings → Profiles**, when this
44+
server can run it.
45+
3. Else, with exactly one provider connected, that provider's profile.
46+
4. Else the coordinator does not start, and the UI asks the person to connect
47+
a provider or pick a profile. The project starts once one resolves.
48+
49+
The resolved profile is fixed for the life of the project.
50+
51+
Agents run in sidecar processes the hub starts on this machine; their state
52+
lives under `HUB_DATA_DIR` (`data/hub`). Docker sidecars are optional
53+
([SELF_HOST.md](https://github.com/corbitsdev/corbits-builder/blob/main/docs/SELF_HOST.md#sidecars-in-docker)).
54+
55+
## Tests
56+
57+
```sh
58+
bun run check # lint, format check, typecheck, unit and integration tests
59+
bun run test # unit and integration tests only
60+
bun run test:e2e # the API end to end against an isolated Postgres
61+
```
62+
63+
`bun run test:e2e` reads the `E2E_DB_*` values from `.env.hub`. Start its
64+
throwaway database under its own project name, so it never replaces the
65+
development database:
66+
67+
```sh
68+
docker compose -p builder-e2e --env-file .env.hub --project-directory . -f deploy/compose.test.yml up -d --wait
69+
```
70+
71+
Tests are load-bearing only: invariants and end-to-end behaviour through the
72+
real API ([CONTRIBUTING.md](https://github.com/corbitsdev/corbits-builder/blob/main/CONTRIBUTING.md#rules-worth-knowing)).
73+
74+
## Other scripts
75+
76+
| Script | What it does |
77+
| ------------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
78+
| `bun run start` | Starts the hub alone, without the migrations or the UI dev server. |
79+
| `bun run dev:hub` | Starts the hub alone for the UI dev server on `UI_PORT` (from the shell; `bun run dev` also reads it from `.env.web`). |
80+
| `bun run dev:web` | Starts the UI dev server alone. |
81+
| `bun run format` | Formats the tree with bun fmt. |
82+
| `bun run lint` | oxlint, warnings are errors. |
83+
| `bun run typecheck` | Typechecks the hub, the client API and the UI. |
84+
| `bun run db:migrate` | Applies the migrations. |
85+
| `bun run db:generate` | Generates a migration from `apps/hub/src/db/schema.ts`. |
86+
| `bun run docker:up` | Builds and starts Builder in Docker ([SELF_HOST.md](https://github.com/corbitsdev/corbits-builder/blob/main/docs/SELF_HOST.md)). |
87+
| `bun run docker:sidecar` | Builds the sidecar Docker image. |
88+
| `bun run auth:bootstrap` | Migrates and creates the `BOOTSTRAP_*` machine account and workspace. |
89+
| `bun run auth:emulate` | Starts a loopback Google sign-in emulator (`GOOGLE_EMULATE_URL` in `.env.hub.example`). |
90+
| `bun run vendor:verify` | Checks `vendor/interchange` matches its pinned upstream commit. |
91+
| `bun run skills:react-ui` | Regenerates the designer's `@corbits/react-ui` API reference. |
92+
93+
## Where things live
94+
95+
- Product names and user-visible wording: `apps/hub/src/config/` and
96+
`apps/web/src/config.ts`. The default name is **Builder**.
97+
- Providers, models, roles and profiles: `apps/hub/src/config/models.ts`;
98+
each model's limits: `apps/hub/src/inference/model-limits.ts`.
99+
- Database schema and migrations: `apps/hub/src/db/`.
100+
- Hub routes: `apps/hub/src/projects/`, `artifacts/`, `builds/`, `settings/`.
101+
- Starting and placing sidecars: `apps/hub/src/provisioning/`,
102+
`apps/hub/src/sidecar/`.
103+
- Each role's prompt and tools: `workflows/<role>/src/`.
104+
- Interchange docs: `vendor/interchange/docs/`.

0 commit comments

Comments
 (0)