diff --git a/CHANGELOG.md b/CHANGELOG.md index 747aaf9..377f28e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,8 +4,35 @@ All notable changes to Toolgraph are documented here. ## Unreleased +### Added + +- `toolgraph example init` now writes a `README.md` into the generated + directory. `docs/` is excluded from the sdist, so a user who installed from + PyPI previously landed in a directory of three unexplained files with only + the CLI's one-line hint to go on. The file lists the commands in order, + states the expected outcome, and links the full guide. +- `examples/README.md` maps each `servers-*.yaml` / `governance-*.yaml` input + to the script or workflow that consumes it, and to what it needs to run. +- A `Documentation` URL in the project metadata, so the beginner guide is + reachable from the PyPI sidebar. + ### Fixed +- The packaged quickstart governance manifest now carries `provenance` on + every authored edge. The beginner guide claimed every authored edge had an + evidence pointer and then told the reader to run `unbacked-edges`, which + listed all three of them. The audit now comes back empty, and both the + manifest and the guide say what that does and does not prove: the demo tools + return strings and touch no file, so the READS/WRITES effects are synthetic + and their evidence cites the fixture that defines them. An empty audit means + every edge has a pointer, not that a claim is true. +- Beginner guide corrections (both languages): the shared-backend section + needs a repository clone, since `docker-compose.yml` and `.env.example` ship + with neither the package nor the generated quickstart; `policy compile` + prints metadata describing the bundle, not the bundle; and the gateway step + is labelled optional, matching the README's statement that memtomem-stm is + not a required dependency. + - The `sse` crawl transport is now verified end to end. `ServerSpec` has always accepted `transport: sse`, but no test, script or CI job ever opened an SSE session, so the branch survived the MCP 2.x migration unexercised. The test diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index cd006cb..b4178c0 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -93,4 +93,7 @@ an issue. `toolgraph example init` builds a self-contained offline quickstart. A bug that reproduces from it is the most useful kind of report, because it needs no -access to your servers or your governance manifest. +access to your servers or your governance manifest. The generated directory +carries its own `README.md`, and the +[beginner guide](docs/beginner-guide.md) ([Korean](docs/ko-beginner-guide.md)) +walks the same path in full if you have not run Toolgraph before. diff --git a/docs/beginner-guide.md b/docs/beginner-guide.md index 5a9d8f5..9294b9f 100644 --- a/docs/beginner-guide.md +++ b/docs/beginner-guide.md @@ -50,7 +50,17 @@ toolgraph ingest-manifest \ ``` The example grants `vibe-coder` both tools but places the draft resource under -a DENY policy. Every authored edge includes a local evidence pointer. +a DENY policy. Every authored edge carries `provenance` with an evidence +pointer, so `toolgraph unbacked-edges` returns an empty list. Delete the +`provenance` block under one of the `data_access` entries, rerun the +`ingest-manifest` command above, and `unbacked-edges` names that edge. Grants +are audited separately, under `unbacked-edges --include-grants`. + +An empty result means every edge has a pointer, not that the claims behind +them are true. The demo tools return strings and touch no file, so the +READS/WRITES entries are synthetic; their evidence says so. Verifying a +pointer is the reviewer's work, and requiring one is what makes that work +possible. ## 4. Inspect the decision @@ -73,17 +83,22 @@ toolgraph policy compile \ --output .toolgraph/policy-bundle.json ``` -Success prints JSON containing the output path, exact byte digest, graph -instance/generation, and eligible/rejected counts. The artifact is canonical +Success prints JSON describing the bundle -- output path, exact byte digest, +graph instance/generation, and eligible/rejected counts -- not the bundle +itself. The artifact is canonical UTF-8 JSON, written privately with atomic replacement. Start with `review`: the reference gateway keeps rejected tools visible and records would-block calls. After inspecting the decisions, compile a matching `strict` bundle to hide and block rejected tools. -## 6. Enforce it with memtomem-stm +That is the end of the standalone quickstart. Everything below is optional. -Toolgraph and the gateway remain separate packages. Follow the +## 6. Optional: enforce it with memtomem-stm + +Toolgraph never blocks a call itself, so a gateway is how a DENY becomes real. +The reference consumer is a separate package and is not required to use +Toolgraph. Follow the [memtomem-stm Toolgraph policy gateway guide](https://github.com/memtomem/memtomem-stm/blob/main/docs/guides/toolgraph-policy-gateway.md) to point STM at this bundle, run `mms gateway status` and `explain`, and connect it to Codex or Claude Code. @@ -101,6 +116,12 @@ toolgraph drift toolgraph blast-radius draft-publish-deny ``` +Two of these return a row on purpose. `unsafe-tools` reports `publish_note` +reaching `draft-publish-deny`, classified as a violation, and `blast-radius` +reports the path that policy covers. Naming that reach is what they are for, so +on this fixture a non-empty result is the correct one. The other three come +back empty. + Good first experiments are removing the `read_note` grant, changing the draft policy binding, and recompiling. Always rerun `ingest-manifest` before compilation. Invalid manifests are rejected without replacing the previous @@ -109,9 +130,13 @@ graph state; failed compilation leaves the previous bundle intact. ## Shared or fleet operation Ladybug is the default single-process local backend. Use Neo4j when multiple -processes or operators need a shared graph: +processes or operators need a shared graph. This part needs the repository: +`docker-compose.yml` and `.env.example` ship with the clone, not with the +package or the generated quickstart. ```bash +git clone https://github.com/memtomem/toolgraph +cd toolgraph docker compose up -d --wait cp .env.example .env toolgraph init --backend neo4j diff --git a/docs/ko-beginner-guide.md b/docs/ko-beginner-guide.md index 28c25cf..3cd8049 100644 --- a/docs/ko-beginner-guide.md +++ b/docs/ko-beginner-guide.md @@ -48,7 +48,15 @@ toolgraph ingest-manifest \ ``` 예제는 `vibe-coder`에게 두 도구의 호출 권한을 주지만 draft 리소스에는 -DENY 정책을 적용합니다. 작성한 모든 edge에는 근거 포인터가 있습니다. +DENY 정책을 적용합니다. 작성한 모든 edge에 `provenance` 근거 포인터가 있어서 +`toolgraph unbacked-edges` 결과가 비어 있습니다. `data_access` 항목 하나에서 +`provenance` 블록을 지우고 다시 적재한 뒤 실행하면 그 edge가 목록에 나타납니다. +grant는 `unbacked-edges --include-grants`로 따로 감사합니다. + +결과가 비어 있다는 것은 모든 edge에 포인터가 있다는 뜻이지 그 주장이 사실이라는 +뜻이 아닙니다. demo 도구는 문자열만 반환하고 실제 파일을 건드리지 않으므로 +READS/WRITES 항목은 예제용이며 evidence에 그렇게 적혀 있습니다. 포인터를 따라가 +확인하는 것은 검토자의 몫이고, 포인터를 요구하는 이유가 바로 그것입니다. ## 4. 판정 확인 @@ -72,17 +80,21 @@ toolgraph policy compile \ --output .toolgraph/policy-bundle.json ``` -성공하면 출력 경로, 정확한 byte digest, 그래프 instance/generation, -eligible/rejected 개수가 JSON으로 표시됩니다. 번들은 canonical UTF-8 +성공하면 번들 자체가 아니라 번들을 설명하는 JSON(출력 경로, 정확한 byte +digest, 그래프 instance/generation, eligible/rejected 개수)이 표시됩니다. 번들은 canonical UTF-8 JSON이며 private 권한과 atomic replacement로 저장됩니다. 첫 적용은 `review`가 안전합니다. reference gateway는 rejected 도구를 계속 보이게 두고 would-block 호출을 기록합니다. 판정을 확인한 뒤 같은 정책을 `strict` profile로 다시 컴파일하면 rejected 도구가 숨겨지고 차단됩니다. -## 6. memtomem-stm에서 적용 +여기까지가 단독으로 완결되는 quickstart입니다. 아래는 모두 선택 사항입니다. + +## 6. (선택) memtomem-stm에서 적용 -Toolgraph와 gateway는 서로 독립된 패키지입니다. +Toolgraph는 호출을 직접 막지 않으므로 DENY를 실제 차단으로 만들려면 +gateway가 필요합니다. reference gateway는 별도 패키지이며 Toolgraph를 +쓰는 데 필수는 아닙니다. [memtomem-stm Toolgraph 정책 gateway 가이드](https://github.com/memtomem/memtomem-stm/blob/main/docs/guides/toolgraph-policy-gateway.md)를 따라 이 번들을 STM에 연결하고 `mms gateway status`, `explain`으로 확인한 뒤 Codex 또는 Claude Code에 등록합니다. @@ -100,6 +112,11 @@ toolgraph drift toolgraph blast-radius draft-publish-deny ``` +이 중 둘은 결과가 나오는 것이 정상입니다. `unsafe-tools`는 `publish_note`가 +`draft-publish-deny`에 닿는 것을 violation으로 보고하고, `blast-radius`는 그 +정책이 덮는 경로를 보고합니다. 그 도달 범위를 짚어 주는 것이 두 명령의 목적이므로 +이 예제에서는 결과가 비어 있지 않은 쪽이 맞습니다. 나머지 셋은 비어 있습니다. + `read_note` grant를 제거하거나 draft 정책 binding을 바꾼 뒤 다시 적재·컴파일해 보세요. 잘못된 manifest는 이전 그래프 상태를 유지한 채 거부되고, 컴파일 실패도 마지막 정상 번들을 덮어쓰지 않습니다. @@ -107,9 +124,13 @@ toolgraph blast-radius draft-publish-deny ## 팀·공유 환경 Ladybug는 로컬 단일 프로세스 기본값입니다. 여러 프로세스나 운영자가 -그래프를 공유해야 할 때 Neo4j를 사용합니다. +그래프를 공유해야 할 때 Neo4j를 사용합니다. 이 부분은 저장소가 필요합니다. +`docker-compose.yml`과 `.env.example`은 패키지나 생성된 quickstart가 아니라 +clone에만 들어 있습니다. ```bash +git clone https://github.com/memtomem/toolgraph +cd toolgraph docker compose up -d --wait cp .env.example .env toolgraph init --backend neo4j diff --git a/examples/README.md b/examples/README.md new file mode 100644 index 0000000..85ae59e --- /dev/null +++ b/examples/README.md @@ -0,0 +1,33 @@ +# Example inputs + +Crawl targets (`servers*.yaml`) and operator-authored governance +(`governance*.yaml`) for the repository demos. These ship with the clone only, +not with the PyPI package. + +If you are here to try Toolgraph for the first time, do not start with these. +Run `toolgraph example init`, which writes an offline, self-contained +quickstart with its own README, and follow the +[beginner guide](../docs/beginner-guide.md) +([Korean](../docs/ko-beginner-guide.md)). + +All paths below are relative to the repository root; the demos expect to be run +from there. + +| Input | Used by | Needs | +| :--- | :--- | :--- | +| `servers.yaml` + `governance.yaml` | `scripts/demo.sh`, and the `Makefile` defaults (`SERVERS` / `GOVERNANCE`) | `npx` and network on first run | +| `servers-public.yaml` + `governance-public.yaml` | `scripts/demo-public.sh` | `npx`, `uvx`, network; downloads several real public servers | +| `servers-gate.yaml` + `governance-gate.yaml` | The proxy/surfacing wedge in the main [README](../README.md) | The two companion servers installed separately | +| `servers-fleet.yaml` + `governance-fleet.yaml` | Fleet-scale authoring exhibit, 7 servers | `npx`, `uvx`, network, the companion servers, and `mkdir -p /tmp/toolgraph-fleet-sandbox` for the filesystem server | +| `servers-policy-gateway.yaml` + `governance-policy-gateway.yaml` | Manual policy-gateway walkthrough against `policy_gateway_server.py` | Nothing external; runs offline | +| `governance-syncmill-smoke.yaml` | `scripts/ecosystem_smoke.py` and `scripts/gate_e_operational.py` | Those harnesses supply their own crawl input | +| `policy_gateway_server.py` | The pair above, and `scripts/policy_bundle_gateway_smoke.py`, which writes its own `servers.yaml` at run time | Nothing external | +| `policy_propose_from_observed.py` | Run directly; reads a governance file plus a window of observed gateway usage you supply | No fixed pair. `--self-test` runs it standalone | + +Not every file has a partner: the two harness entries above bring their own +crawl input, and the proposal script takes whatever governance and observation +files you point it at. + +`governance-*.yaml` files are operator assertions, not crawled facts. The +richer ones carry per-edge `provenance` with an evidence pointer; +`toolgraph unbacked-edges` is what finds the ones that do not. diff --git a/pyproject.toml b/pyproject.toml index 4823823..5a3fca9 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -46,6 +46,7 @@ ladybug = [ [project.urls] Homepage = "https://github.com/memtomem/toolgraph" +Documentation = "https://github.com/memtomem/toolgraph/blob/main/docs/beginner-guide.md" Repository = "https://github.com/memtomem/toolgraph" Issues = "https://github.com/memtomem/toolgraph/issues" @@ -85,18 +86,26 @@ packages = ["toolgraph"] # the project page, the license and changelog, and the JSON contracts. # `contracts/fixtures/` is test data and is excluded with the tests. [tool.hatch.build.targets.sdist] +# Every pattern here is anchored with a leading `/`, because Hatchling matches +# an unanchored pattern at any depth. A bare `README.md` also pulled in +# `examples/README.md`, which `scripts/verify-artifacts.sh` then rejected as +# outside the allowlist -- its own allowlist is a root-anchored regex. The +# directory entries had the same latent reach in both directions: `toolgraph/` +# would have swept in a nested directory of that name, and `docs/` would have +# excluded a future `toolgraph/docs/`. Anchor them and the two allowlists +# cannot drift apart again. include = [ - "toolgraph/", - "contracts/", - "README.md", - "CHANGELOG.md", - "LICENSE", - "SECURITY.md", - "pyproject.toml", + "/toolgraph/", + "/contracts/", + "/README.md", + "/CHANGELOG.md", + "/LICENSE", + "/SECURITY.md", + "/pyproject.toml", ] exclude = [ - "contracts/fixtures/", - "docs/", + "/contracts/fixtures/", + "/docs/", ] # Note: hatchling always ships the VCS ignore file (.gitignore) in an sdist; # an exclude entry for it has no effect. diff --git a/tests/test_release_hardening.py b/tests/test_release_hardening.py index 0b45904..2bfc174 100644 --- a/tests/test_release_hardening.py +++ b/tests/test_release_hardening.py @@ -176,10 +176,34 @@ def test_example_init_creates_self_contained_assets_and_refuses_collision(tmp_pa created = runner.invoke(app, ["example", "init", str(destination)]) assert created.exit_code == 0, created.output assert sorted(path.name for path in destination.iterdir()) == [ + "README.md", "governance.yaml", "policy_gateway_server.py", "servers.yaml", ] + # The generated directory is the only guidance a PyPI user has on disk: + # `docs/` is excluded from the sdist, so the beginner guide is a URL away. + readme = (destination / "README.md").read_text() + assert "toolgraph init" in readme + assert "docs/beginner-guide.md" in readme + # The guide claims `unbacked-edges` comes back empty for this fixture; that + # only holds while every authored edge carries a non-blank evidence + # pointer. Counting the string would pass on an empty value, so parse it. + governance = yaml.safe_load((destination / "governance.yaml").read_text()) + authored = [ + *governance["grants"], + *governance["data_access"], + *( + binding + for bindings in governance["governed_by"].values() + for binding in bindings + ), + ] + assert len(authored) == 5 + for edge in authored: + evidence = edge["provenance"]["evidence"] + assert isinstance(evidence, str) and evidence.strip(), edge + servers = (destination / "servers.yaml").read_text() assert sys.executable in servers assert "__TOOLGRAPH_PYTHON__" not in servers diff --git a/toolgraph/quickstart/README.md b/toolgraph/quickstart/README.md new file mode 100644 index 0000000..127ce33 --- /dev/null +++ b/toolgraph/quickstart/README.md @@ -0,0 +1,55 @@ +# Toolgraph quickstart + +You are inside a generated directory. It is self-contained and offline: the +three files here are a demo MCP server, a crawl target list, and an +operator-authored governance manifest. + +| File | What it is | +| :--- | :--- | +| `policy_gateway_server.py` | A local MCP server exposing `read_note` (read-only) and `publish_note` (destructive). | +| `servers.yaml` | Points the crawler at that server, named `policy-gateway`. | +| `governance.yaml` | Who may call what, what each tool is modelled as touching, and a DENY policy on the draft resource. | + +## Run it + +```bash +toolgraph init +toolgraph crawl --servers servers.yaml +toolgraph ingest-manifest --governance governance.yaml --strict-drift +toolgraph eligible-tools vibe-coder \ + policy-gateway::read_note policy-gateway::publish_note --profile review +toolgraph selection-explain vibe-coder policy-gateway::publish_note +toolgraph policy compile --agent vibe-coder --profile review \ + --output .toolgraph/policy-bundle.json +``` + +`read_note` comes back eligible. `publish_note` comes back rejected, because +the manifest models it as writing to a resource under a DENY policy, and +`selection-explain` prints that path. Neither demo tool touches a file; both +return a string. The data-flow claims live in `governance.yaml` and exist to +make the policy path reachable, which is what Toolgraph reasons over. The last command writes the bundle and prints its path, byte digest, and +eligible/rejected counts. + +Toolgraph decided nothing at runtime here. It analyzed and compiled; a gateway +that consumes the bundle is what would actually block a call. + +## Then + +Try `toolgraph unbacked-edges`, which comes back empty because every authored +edge here carries an evidence pointer. Then try `unsafe-tools vibe-coder` and +`blast-radius draft-publish-deny`, which do return a row: both are meant to +name `publish_note` reaching the DENY policy. That is the demo working, not a +failure. + +Edit `governance.yaml`, rerun `ingest-manifest`, and recompile to see a +decision change. + +These files are a copy, not a link, so upgrading Toolgraph does not update +them. After an upgrade, generate a fresh directory alongside this one +(`toolgraph example init toolgraph-quickstart-new`) and move your edits across: +`example init` refuses to write into a directory that already exists. The +CHANGELOG says when a release changed these files. + +Full walkthrough, including handing the bundle to a gateway: + +(Korean: `docs/ko-beginner-guide.md`). diff --git a/toolgraph/quickstart/governance.yaml b/toolgraph/quickstart/governance.yaml index 6e23c04..16b27fd 100644 --- a/toolgraph/quickstart/governance.yaml +++ b/toolgraph/quickstart/governance.yaml @@ -1,3 +1,18 @@ +# The operator-authored half of the quickstart graph. Crawling discovers what +# tools exist; this file asserts who may call them and what they touch. +# +# Every authored edge carries `provenance`. `source: operator_asserted` means a +# human claimed it (not a crawled fact), and `evidence` points at where the +# claim can be checked. `toolgraph unbacked-edges` lists edges missing that +# pointer, so a real deployment should keep this section filled in. +# +# Read the pointers here for what they are. The demo tools return strings and +# touch no file at all, so the READS/WRITES entries are synthetic effects that +# exist to make the DENY path reachable; their evidence cites the fixture that +# defines them. An empty `unbacked-edges` result means every edge has a +# pointer, never that a claim is true. Checking the claim is the reviewer's +# job, which is the whole reason the pointer is required. + agents: - vibe-coder @@ -11,18 +26,38 @@ grants: - agent: vibe-coder tool: "policy-gateway::read_note" granted_by: quickstart + provenance: + source: operator_asserted + confidence: high + evidence: governance.yaml:read-grant - agent: vibe-coder tool: "policy-gateway::publish_note" granted_by: quickstart + provenance: + source: operator_asserted + confidence: high + evidence: governance.yaml:publish-grant data_access: - tool: "policy-gateway::read_note" resource: "file:///demo/notes" mode: READS + provenance: + source: operator_asserted + confidence: high + evidence: "policy_gateway_server.py:read_note (synthetic demo effect)" - tool: "policy-gateway::publish_note" resource: "file:///demo/drafts" mode: WRITES + provenance: + source: operator_asserted + confidence: high + evidence: "policy_gateway_server.py:publish_note (synthetic demo effect)" governed_by: "file:///demo/drafts": - - draft-publish-deny + - policy: draft-publish-deny + provenance: + source: operator_asserted + confidence: high + evidence: governance.yaml:draft-policy