Skip to content

Commit 368c0e9

Browse files
committed
feat(agents): improve workflow diagnostics and tool execution
1 parent 7c115a1 commit 368c0e9

157 files changed

Lines changed: 7936 additions & 334 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎apps/docs/components/footer/footer.tsx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ const PRODUCT_LINKS: FooterItem[] = [
2626
{ label: 'Workflows', href: '/workflows' },
2727
{ label: 'Knowledge Base', href: '/knowledgebase' },
2828
{ label: 'Tables', href: '/tables' },
29-
{ label: 'MCP', href: '/agents/mcp' },
29+
{ label: 'MCP', href: '/mcp' },
3030
{ label: 'API', href: '/api-reference/getting-started' },
3131
{ label: 'Self Hosting', href: '/platform/self-hosting' },
3232
{ label: 'Status', href: 'https://status.sim.ai', external: true },

‎apps/docs/content/docs/agents/mcp.mdx‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,13 @@ import { Image } from '@/components/ui/image'
77
import { Video } from '@/components/ui/video'
88
import { Callout } from 'fumadocs-ui/components/callout'
99

10-
The Model Context Protocol ([MCP](https://modelcontextprotocol.com/)) is an open standard for connecting AI to external tools and data. Add an MCP server to your workspace and its tools become available to your agents — a way to integrate services Sim doesn't have a built-in integration for.
10+
The Model Context Protocol ([MCP](https://modelcontextprotocol.io/)) is an open standard for connecting AI to external tools and data. Add an MCP server to your workspace and its tools become available to your agents — a way to integrate services Sim doesn't have a built-in integration for.
11+
12+
<Callout type="info">
13+
This guide connects **external tools to Sim agents**. To connect Codex or another
14+
assistant to Sim's workspace API, use [Sim MCP](/mcp). To expose only selected
15+
workflows, use [MCP deployment](/workflows/deployment/mcp).
16+
</Callout>
1117

1218
## Adding an MCP Server as a Tool
1319

‎apps/docs/content/docs/api-reference/(generated)/workflow-runs/meta.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
{
22
"pages": [
3+
"previewWorkflowRunFromBlock",
34
"listWorkflowRunsV2",
45
"getWorkflowRunV2",
56
"downloadWorkflowRunFileV2",

‎apps/docs/content/docs/api-reference/(generated)/workflows/meta.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
"listWorkflows",
44
"createWorkflowV2",
55
"getWorkflow",
6+
"inspectWorkflow",
67
"updateWorkflowV2",
78
"deleteWorkflowV2",
89
"restoreWorkflow",
Lines changed: 261 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,261 @@
1+
---
2+
title: Agent workflow operations
3+
description: Edit bindings safely, inspect compact run evidence, and track partial retries
4+
---
5+
6+
Use [Sim CLI authentication](/cli/authentication) for shell commands. To connect
7+
an assistant over MCP, start with the [hosted Sim MCP overview](/mcp). Its
8+
[capability map](/mcp#choose-an-interface) explains which operations need the CLI
9+
or HTTP API, including binary downloads.
10+
11+
## Discover the exact operation and credential
12+
13+
```sh
14+
sim tools list --search slack
15+
sim tools get slack_message
16+
sim credentials list --search "Support connection"
17+
sim --output json credentials get "$CREDENTIAL_ID"
18+
```
19+
20+
Use the returned tool ID and parameter schema; do not derive IDs from an English
21+
verb. For example, Slack sending uses `slack_message`. Supply the intended
22+
credential ID in the tool's documented credential parameter. A workflow can use
23+
a different credential from a direct tool call even when both are started by the
24+
same Sim user.
25+
26+
`credentials get` reports the credential type/provider, stored identity metadata,
27+
and recorded scopes with their source. `verifiedLive: false` means inspection
28+
does not contact the provider or prove that a token is still valid. Unknown scope
29+
metadata is different from a known empty scope set. Custom bot credentials may
30+
have no separately stored public identity metadata; the diagnostic marks that
31+
unknown instead of decrypting their secret payload or guessing from a display name.
32+
33+
Slack access depends on the selected user's or bot's grants and conversation
34+
membership. A credential that can list some channels may lack access to another
35+
private channel or its files. Follow the diagnostic notes and the actual provider
36+
error for that credential.
37+
38+
## Download stored tool files
39+
40+
`slack_download` returns a stored file descriptor (`id`, `key`, `context`, `name`,
41+
`type`, `size`, `url`), not inline base64. The URL requires authorization.
42+
43+
For a personal output returned by a direct tool call (`context: copilot`):
44+
45+
```sh
46+
sim tools files download "$FILE_ID" -o ./attachment.pdf
47+
```
48+
49+
For a workflow execution output, use the workflow/run that produced it:
50+
51+
```sh
52+
sim workflows runs files download "$WORKFLOW_ID" "$RUN_ID" "$FILE_ID" -o ./attachment.pdf
53+
```
54+
55+
The downloader uses the configured Sim credential and streams to disk through
56+
the existing safe file writer. Direct personal outputs require their original
57+
human owner and current workspace access; workflow outputs use the run's access
58+
checks. A descriptor's URL or storage key alone grants no access. Execution and
59+
personal output files can expire; copy durable artifacts into workspace files.
60+
61+
## Use editable state for in-place edits
62+
63+
Start a diagnostic read with a compact view:
64+
65+
```sh
66+
sim --output json workflows inspect "$WORKFLOW_ID"
67+
sim --output json workflows inspect "$WORKFLOW_ID" --block-id "$BLOCK_ID" --include-code
68+
```
69+
70+
Inspection returns canonical block IDs, names, enabled states, parent IDs,
71+
connections, and bounded nonempty inputs. It omits credential fields, opaque
72+
credential-bearing values, UI metadata, output schemas, and code by default.
73+
`omittedInputs` identifies withheld fields, and `truncated` identifies inputs
74+
that exceed the diagnostic budget. Automatic redaction cannot recognize every
75+
secret embedded in arbitrary text or code. This view is for inspection, not saving.
76+
77+
`workflows export` produces a sanitized portable representation. Its credential
78+
bindings are cleared; optional reference metadata supports mapped imports into
79+
another workspace. Use [workflow import and sync](/cli/workflow-sync) for that flow.
80+
81+
For an existing workflow, start from `workflows state get`, preserve block IDs
82+
and credential/table references, and use `state replace --dry-run` before saving.
83+
Inspect the validation check coverage, skipped checks, warnings, and removed
84+
bindings. A clean static check cannot verify provider access, model output, or
85+
runtime behavior. Never replace editable state with a portable export merely
86+
because both are JSON.
87+
88+
A semantic edit with `dryRun=true` returns provisional `previewBlockIds` and an
89+
empty `mintedBlockIds`. Commit the edit and use that response's `mintedBlockIds`;
90+
preview IDs are not reserved. Inspect the saved workflow to confirm its enabled
91+
blocks and connections before executing.
92+
93+
## Discover workflows across workspaces
94+
95+
Use `workspaces list`, then `workflows list --workspace <id>` for each relevant
96+
workspace. Follow each list's `nextCursor`; a page is not the entire account.
97+
Workflow results already include the workspace ID, description, folder path,
98+
deployment state, and web URL. `--search` matches workflow names;
99+
`--folder-path` and `--deployed-only` narrow that workspace's results.
100+
101+
There is no account-wide workflow-content search. Narrow using metadata before
102+
inspecting candidate workflows; do not fetch every full draft to discover a
103+
Slack destination or integration. Inspection does not index or search secrets.
104+
105+
## Execute Functions and reference secrets
106+
107+
Read `tools get function_execute` before a direct Function call. JavaScript
108+
references a selected secret with `{{SECRET_NAME}}`, not `process.env`:
109+
110+
```javascript
111+
const token = {{SECRET_NAME}}
112+
return { configured: typeof token === 'string' && token.length > 0 }
113+
```
114+
115+
Use the tool's documented secret-selection parameters and avoid returning the
116+
secret itself. Direct execution uses the authenticated caller's current workspace
117+
permissions; it does not require creating a temporary workflow or supplying an
118+
executor origin.
119+
120+
## Inspect execution and application results separately
121+
122+
```sh
123+
sim --output json logs get "$RUN_ID" --summary
124+
sim --output json workflows runs get "$RUN_ID" --workflow "$WORKFLOW_ID" \
125+
--select-output "Verify delivery.result"
126+
```
127+
128+
`--summary` reports execution status, recorded failures (including handled
129+
errors), compact failing inputs/outputs, observed block statuses, and file
130+
metadata. It caps trace traversal and field/text output and reports `truncated`
131+
when it omits detail. Binary fields and data URLs are omitted. This is a local
132+
projection of the authorized log response; it reduces terminal/context output,
133+
not the size of the server's log read. Use `logs get --trace` or explicit output
134+
selectors when more detail is needed.
135+
136+
Selectors accept an unambiguous block name or ID followed by an optional output
137+
path. Names are resolved against the **current saved workflow**, ignoring case,
138+
spaces, and dots. Use block IDs for renamed or deleted historical blocks.
139+
Ambiguous names are rejected with candidate IDs. Missing output paths can be
140+
omitted; that does not establish that a block ran successfully.
141+
142+
A completed run can have sent a failure explanation rather than delivered a
143+
document. Have the workflow deliberately emit an `applicationOutcome` field in
144+
its final output, for example:
145+
146+
```json
147+
{
148+
"applicationOutcome": {
149+
"status": "verification_blocked",
150+
"threadId": "<provider-thread-id>",
151+
"fileIds": []
152+
}
153+
}
154+
```
155+
156+
Choose application statuses that describe the workflow's result, such as
157+
`delivered`, `clarification_requested`, or `verification_blocked`. The summary
158+
copies this field at the top level, or inside a Function block's `result` or a
159+
Response block's `data` wrapper; it does not infer it from a successful execution or a Slack
160+
message. A null outcome means the workflow did not report one. Trace entries
161+
describe observed work, and cannot prove unrecorded downstream effects or content
162+
quality. Expired trace data may leave no block-level evidence.
163+
164+
## Track long manual runs
165+
166+
Preview a partial retry before repeating external actions:
167+
168+
```sh
169+
sim --output json workflows runs preview "$WORKFLOW_ID" \
170+
--from-block "$BLOCK_ID" --source-run "$SOURCE_RUN_ID"
171+
```
172+
173+
Check `validation`, `rerunBlocks`, and `upstreamBlocks`. The preview shares the
174+
executor's graph analysis and returns cached-output availability without exposing
175+
output values. It executes no blocks and reserves no run ID. The listed rerun
176+
blocks are candidates, not a promise of execution order: conditions, disabled
177+
blocks, loops, and runtime failures still affect the path. A later run uses the
178+
draft saved at that time; preview does not lock a snapshot.
179+
180+
```sh
181+
sim workflows run "$WORKFLOW_ID" --from-block "$BLOCK_ID" \
182+
--source-run "$SOURCE_RUN_ID" --follow
183+
```
184+
185+
Manual execution uses the current saved workflow, while an ordinary run uses
186+
the active deployment. The CLI prints a run ID to stderr before sending a manual
187+
request, including partial retries. From another terminal:
188+
189+
```sh
190+
sim workflows runs get "$RUN_ID" --workflow "$WORKFLOW_ID"
191+
sim workflows runs wait "$RUN_ID" --workflow "$WORKFLOW_ID"
192+
```
193+
194+
Status becomes available after the server admits and records the run. A request
195+
rejected before admission may never have a run record. The run ID is a one-shot
196+
identifier, not a replayable idempotency key. After an uncertain network failure,
197+
inspect that same ID before considering another execution; a fresh ID can repeat
198+
external actions.
199+
200+
| Mode | Supported |
201+
| --- | --- |
202+
| Deployed run | Synchronous, `--follow`, or `--async` |
203+
| Manual draft run | Synchronous or `--follow` |
204+
| Manual `--from-block` with `--source-run` | Synchronous or `--follow` |
205+
| `--follow` with `--async` | No |
206+
| Manual execution with `--async` | No; use the printed ID to inspect the active request |
207+
208+
## Inspect an intermediate execution result
209+
210+
Use the existing output selectors with the executor's stop-after control:
211+
212+
```sh
213+
sim --output json workflows run "$WORKFLOW_ID" --stop-after "$FORMATTER_BLOCK_ID" \
214+
--select-output "$FORMATTER_BLOCK_ID.result"
215+
```
216+
217+
This executes real blocks. It stops scheduling after the target finishes;
218+
already-running parallel branches can still perform actions. It is not a dry
219+
run or a side-effect suppression mode. The control applies to this request and
220+
does not disable blocks or edit the saved graph. It is unavailable with `--async`.
221+
If a condition skips the target, the remaining path can complete normally.
222+
If the run pauses, a later explicit resume does not inherit the stop-after limit.
223+
224+
Deployed execution uses the deployment snapshot. A workflow with only one
225+
runnable Schedule trigger can run directly. When multiple entry points are
226+
ambiguous, select the intended deployed trigger with the API's
227+
`run: { "source": "deployment", "entry": { "type": "trigger", "blockId": "<block-id>" } }`.
228+
Manual trigger execution remains a separate choice that uses the saved draft.
229+
230+
## Preview a Slack message and paginate Stripe results
231+
232+
For Block Kit messages without file attachments, supply meaningful top-level `text` as the fallback for
233+
notifications and accessibility, alongside `blocks`. To check the actual visual
234+
layout, open Slack's [Block Kit Builder](https://app.slack.com/block-kit-builder)
235+
and paste the message payload into its editor. Review the rendered layout and
236+
fallback text separately. Viewing the preview does not post the message; the
237+
builder's **Send to Slack** action does. Slack may require sign-in.
238+
Slack's file-upload completion endpoint treats a block comment and an initial
239+
text comment differently; Sim preserves blocks when both are supplied with files.
240+
241+
Stripe subscription listing uses `starting_after` or `ending_before` with a
242+
subscription ID; do not send both. Check `has_more` before requesting another
243+
page. Subscription search uses a different provider cursor: pass `next_page`
244+
back as `page`, retaining the same search query. The tools expose these response
245+
fields under `metadata`. Use the exact parameter names
246+
published by `tools get stripe_list_subscriptions` and
247+
`tools get stripe_search_subscriptions`.
248+
249+
## Keep generated files organized
250+
251+
Create explicit folders for durable generated artifacts and temporary scratch
252+
work, and pass the destination to file-producing operations that support it.
253+
For example, use a reusable `/Generated/Archives` folder for archives and a
254+
separate `/Scratch` folder for temporary inputs. These are ordinary workspace
255+
folders, with the same permissions and retention as other files; naming one
256+
`Scratch` does not make it automatically expire.
257+
258+
File compression accepts `folderPath` and `onConflict`. `rename` chooses an
259+
available name; `error` refuses a collision. Inspect the returned `path` for the
260+
actual stored location. Omitting the destination retains the existing root
261+
behavior. There is no implicit workspace-wide generated-file destination.

‎apps/docs/content/docs/cli/credentials.mdx‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,24 @@ Disconnect Credential (OAuth login or personal API key required)
3737

3838
</CommandTable>
3939

40+
## Inspect credential
41+
42+
```bash
43+
sim credentials get <credentialId>
44+
```
45+
46+
Inspect Credential (OAuth login or personal API key required)
47+
48+
**Arguments**
49+
50+
<CommandTable>
51+
52+
| Argument | Required | Description |
53+
| --- | --- | --- |
54+
| `credentialId` | Yes | Selected credential to inspect. |
55+
56+
</CommandTable>
57+
4058
## List credential providers
4159

4260
```bash

‎apps/docs/content/docs/cli/index.mdx‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,9 @@ move files, search knowledge bases, and read run logs from the terminal. Use
1212
pipelines. See [Output formats](/cli/output) for commands that emit raw content
1313
or local configuration.
1414

15+
For safe edits, compact run diagnostics, and partial retries, see
16+
[Agent workflow operations](/cli/agent-workflows).
17+
1518
## Install
1619

1720
<Tabs items={['npm', 'pnpm', 'bun']}>

‎apps/docs/content/docs/cli/logs.mdx‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@ sim logs get <runId> [options]
3232
| Option | Required | Description |
3333
| --- | --- | --- |
3434
| `--trace` | No | Show expanded trace spans with inputs, outputs, errors, timing, and cost. |
35+
| `--summary` | No | Show bounded run diagnostics and the explicit application outcome; omit binary content. |
3536

3637
</CommandTable>
3738

‎apps/docs/content/docs/cli/meta.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99
"output",
1010
"scripting",
1111
"workflow-sync",
12+
"agent-workflows",
1213
"troubleshooting",
1314
"usage-data",
1415
"---Commands---",

0 commit comments

Comments
 (0)