Skip to content

Commit 232bfb8

Browse files
fix(cleanup): never delete resources matched by .vapi-ignore
.vapi-ignore marks platform resources a repository must not manage, and pull never writes them to state. cleanup treats every platform resource missing from state as an orphan and didn't read .vapi-ignore, so a destructive cleanup deleted exactly the resources a team had excluded, such as another team's assistants in a shared org. Push already orphan-protects them. cleanup now loads the org's ignore patterns and keeps every orphan that matches, checking the IDs pull would give the resource (its name slug, with and without the UUID suffix). Kept resources are listed as retained in both dry and destructive runs. tests/cleanup-ignore.test.ts runs the real cleanup command against a stub API: before this change a destructive run deleted an ignored assistant and tool along with the true orphan; now only the orphan. Docs: AGENTS.md, the workflows guide and .vapi-ignore.example describe the fixed behaviour; improvements.md #36 is resolved. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 66e1f33 commit 232bfb8

6 files changed

Lines changed: 264 additions & 28 deletions

File tree

‎AGENTS.md‎

Lines changed: 3 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -229,13 +229,9 @@ The engine resolves IDs and credential names to each org's UUIDs on push.
229229

230230
`resources/<org>/.vapi-ignore` lists platform resources this repo must not
231231
manage, as gitignore-style patterns (see `resources/.vapi-ignore.example`).
232-
Matched resources are skipped on pull and push, and push never deletes them.
233-
A resource that references an ignored one is a validation error.
234-
235-
**`npm run cleanup` does not read `.vapi-ignore`.** Ignored resources are
236-
never in the state file, so cleanup lists them as orphans and a destructive
237-
run would delete them. Check its dry-run list against `.vapi-ignore` with the
238-
human before any `--force` run.
232+
Matched resources are skipped on pull and push, and neither push nor
233+
`npm run cleanup` deletes them. A resource that references an ignored one is a
234+
validation error.
239235

240236
---
241237

‎docs/guides/workflows.md‎

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -186,11 +186,10 @@ npm run cleanup -- <org>
186186
npm run cleanup -- <org> --force --confirm <org>
187187
```
188188

189-
**Check the dry-run list before a destructive run.** Cleanup treats every
190-
platform resource that isn't in `.vapi-state.<org>.json` as an orphan, and it
191-
does not read `.vapi-ignore`: resources you've excluded with `.vapi-ignore`
192-
are never in the state file, so they appear in the list and `--force` would
193-
delete them.
189+
Cleanup treats every platform resource that isn't in
190+
`.vapi-state.<org>.json` as an orphan, except those matched by
191+
`resources/<org>/.vapi-ignore`: it lists those as retained and never deletes
192+
them. Still read the dry-run list before a destructive run.
194193

195194
**When the list includes Vapi's built-in fixtures** (for example the stock
196195
simulation personalities, which can't be deleted — see

‎improvements.md‎

Lines changed: 12 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -87,7 +87,7 @@ you which stack PR closes the row.**
8787
| 33 | `npm run sim` reported every run as passed | A failing suite exited 0 — false green | None | RESOLVED 2026-10-01 |
8888
| 34 | No pre-merge simulation signal; simulations only tested what was deployed | A PR that breaks an agent merges green | #33 | RESOLVED 2026-10-01 |
8989
| 35 | A failed promotion pushed nothing, not even state | git lost track of resources already on the platform | None | RESOLVED 2026-10-01 |
90-
| 36 | `cleanup` deletes resources excluded by `.vapi-ignore` | A destructive cleanup can delete resources another team owns | None | Open |
90+
| 36 | `cleanup` deletes resources excluded by `.vapi-ignore` | A destructive cleanup can delete resources another team owns | None | RESOLVED 2026-10-03 |
9191

9292
**Active backlog after cleanup:** `#2`, `#6`, `#8`, `#12`, `#20`, `#24–#26`, `#31`, and the open remainder of `#27` (wiring the listing-completeness verdict into push/delete/audit, and moving `cleanup.ts` onto the shared pager). Resolved entries stay in this file as historical incident notes per the maintenance directive; stale superseded backlog rows are not duplicated.
9393

@@ -1888,6 +1888,8 @@ None needed once the fix below lands.
18881888

18891889
## 36. `cleanup` deletes resources excluded by `.vapi-ignore`
18901890

1891+
**[RESOLVED 2026-10-03]**
1892+
18911893
**Discovered:** 2026-10-03, while reviewing the agent instructions for the public release.
18921894

18931895
### Problem
@@ -1914,19 +1916,20 @@ another team deletes that team's assistants, tools or squads.
19141916

19151917
### Current mitigation
19161918

1917-
The default run is a dry run, and the destructive run needs `--confirm <org>`.
1918-
The docs and `AGENTS.md` now tell people and agents to check the dry-run list
1919-
against `.vapi-ignore` first.
1919+
None needed once the fix below lands.
19201920

1921-
### Possible fix
1921+
### Possible fix (landed)
19221922

1923-
Load the org's ignore patterns in `cleanup.ts` and exclude matches from the
1924-
deletion list (printing them as retained, as push does), with a test that a
1925-
destructive cleanup keeps an ignored resource.
1923+
`src/cleanup.ts` loads the org's ignore patterns and keeps every orphan that
1924+
matches one, checking the IDs pull would give the resource (its name slug,
1925+
with and without the UUID suffix). Kept resources are listed as retained.
1926+
`tests/cleanup-ignore.test.ts` runs a destructive cleanup against a stub API:
1927+
before the fix it deleted an ignored assistant and tool along with the true
1928+
orphan; after it, only the orphan.
19261929

19271930
### Status
19281931

1929-
Open.
1932+
**RESOLVED 2026-10-03.**
19301933

19311934
---
19321935

‎resources/.vapi-ignore.example‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,8 @@
1010
# id matches `.vapi-ignore`.
1111
# - `--force` on push bypasses the load-filter (so a deliberate override
1212
# can flow through) BUT the orphan-protect still applies.
13+
# - `npm run cleanup -- <org>` never deletes a matched resource, even with
14+
# `--force --confirm <org>`; it lists it as retained.
1315
# - A resource that references an ignored resource (e.g. a squad
1416
# pointing at `assistants/foo` while `assistants/foo` is ignored) is
1517
# a validation ERROR — `--strict` push aborts before any API call.

‎src/cleanup.ts‎

Lines changed: 87 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,16 @@
11
import { resolve } from "path";
22
import { fileURLToPath } from "url";
3-
import { VAPI_BASE_URL, VAPI_ENV, VAPI_TOKEN } from "./config.ts";
3+
import {
4+
loadIgnorePatterns,
5+
matchesIgnore,
6+
VAPI_BASE_URL,
7+
VAPI_ENV,
8+
VAPI_TOKEN,
9+
} from "./config.ts";
10+
import { FOLDER_MAP } from "./resource-parse.ts";
11+
import { slugify } from "./slug-utils.ts";
412
import { loadState } from "./state.ts";
13+
import type { ResourceType } from "./types.ts";
514

615
// ─────────────────────────────────────────────────────────────────────────────
716
// Dangerous Sync - Delete everything NOT in state file
@@ -68,6 +77,30 @@ async function vapiDelete(endpoint: string): Promise<void> {
6877
interface VapiResource {
6978
id: string;
7079
name?: string;
80+
function?: { name?: string };
81+
}
82+
83+
// The .vapi-ignore pattern a platform resource matches, or null. Checks the
84+
// ids pull would give it — its name slug with the UUID suffix (what pull
85+
// writes for an untracked resource) and without it — so a pattern written
86+
// against either form protects the resource.
87+
function ignoredBy(
88+
folder: string,
89+
resource: VapiResource,
90+
patterns: string[],
91+
): string | null {
92+
if (patterns.length === 0) return null;
93+
// Truthy, not ??: pull treats an empty name as missing (extractName).
94+
const name = resource.name || resource.function?.name;
95+
const shortId = resource.id.slice(0, 8);
96+
const ids = name
97+
? [`${slugify(name)}-${shortId}`, slugify(name)]
98+
: [`resource-${shortId}`];
99+
for (const id of ids) {
100+
const matched = matchesIgnore(folder, id, patterns);
101+
if (matched) return matched;
102+
}
103+
return null;
71104
}
72105

73106
function readConfirmToken(argv: string[]): string | undefined {
@@ -149,43 +182,75 @@ async function main(): Promise<void> {
149182
}[] = [];
150183

151184
// Fetch and compare each resource type
152-
const resourceTypes = [
185+
const resourceTypes: Array<{
186+
type: ResourceType;
187+
name: string;
188+
endpoint: string;
189+
deleteEndpoint: string;
190+
}> = [
153191
{
192+
type: "assistants",
154193
name: "assistants",
155194
endpoint: "/assistant",
156195
deleteEndpoint: "/assistant",
157196
},
158-
{ name: "tools", endpoint: "/tool", deleteEndpoint: "/tool" },
159197
{
198+
type: "tools",
199+
name: "tools",
200+
endpoint: "/tool",
201+
deleteEndpoint: "/tool",
202+
},
203+
{
204+
type: "structuredOutputs",
160205
name: "structured outputs",
161206
endpoint: "/structured-output",
162207
deleteEndpoint: "/structured-output",
163208
},
164-
{ name: "squads", endpoint: "/squad", deleteEndpoint: "/squad" },
165209
{
210+
type: "squads",
211+
name: "squads",
212+
endpoint: "/squad",
213+
deleteEndpoint: "/squad",
214+
},
215+
{
216+
type: "personalities",
166217
name: "personalities",
167218
endpoint: "/eval/simulation/personality",
168219
deleteEndpoint: "/eval/simulation/personality",
169220
},
170221
{
222+
type: "scenarios",
171223
name: "scenarios",
172224
endpoint: "/eval/simulation/scenario",
173225
deleteEndpoint: "/eval/simulation/scenario",
174226
},
175227
{
228+
type: "simulations",
176229
name: "simulations",
177230
endpoint: "/eval/simulation",
178231
deleteEndpoint: "/eval/simulation",
179232
},
180233
{
234+
type: "simulationSuites",
181235
name: "simulation suites",
182236
endpoint: "/eval/simulation/suite",
183237
deleteEndpoint: "/eval/simulation/suite",
184238
},
185-
{ name: "evals", endpoint: "/eval", deleteEndpoint: "/eval" },
239+
{
240+
type: "evals",
241+
name: "evals",
242+
endpoint: "/eval",
243+
deleteEndpoint: "/eval",
244+
},
186245
];
187246

188-
for (const { name, endpoint, deleteEndpoint } of resourceTypes) {
247+
// Resources this repo must not manage (.vapi-ignore) are never written to
248+
// state, so every one of them would look like an orphan here. Keep them,
249+
// as push's orphan-protection does.
250+
const ignorePatterns = loadIgnorePatterns();
251+
let retained = 0;
252+
253+
for (const { type, name, endpoint, deleteEndpoint } of resourceTypes) {
189254
console.log(`📥 Fetching ${name}...`);
190255
try {
191256
// Enable debug for structured outputs to see response format
@@ -202,7 +267,16 @@ async function main(): Promise<void> {
202267
continue;
203268
}
204269

205-
const orphans = resources.filter((r) => !stateIds.has(r.id));
270+
const orphans: VapiResource[] = [];
271+
for (const r of resources.filter((r) => !stateIds.has(r.id))) {
272+
const matched = ignoredBy(FOLDER_MAP[type], r, ignorePatterns);
273+
if (matched) {
274+
console.log(
275+
` 🚫 ${r.name || r.function?.name || r.id} retained (matched .vapi-ignore: ${matched})`,
276+
);
277+
retained++;
278+
} else orphans.push(r);
279+
}
206280

207281
if (orphans.length > 0) {
208282
console.log(
@@ -228,6 +302,12 @@ async function main(): Promise<void> {
228302
"\n═══════════════════════════════════════════════════════════════",
229303
);
230304

305+
if (retained > 0) {
306+
console.log(
307+
`\n🚫 ${retained} resource(s) not in state were kept because they match .vapi-ignore`,
308+
);
309+
}
310+
231311
if (toDelete.length === 0) {
232312
console.log("✅ Nothing to delete - all resources match state file\n");
233313
return;

0 commit comments

Comments
 (0)