feat(runtime): warn when a topology timeout pin is below the default it overrides (#675)

CHANGELOG.md

@@ -30,6 +30,7 @@
 ### Other changes
 
 - **[runtime] [docs]** `ultrafuzz run --run-id <id>` now fails with `RUN_ALREADY_EXISTS` before planning when the workflow engine already records a run with that ID, as it does after `ultrafuzz clean <id>` (#1258). Before, `run` rebuilt the plan and the execution snapshot, about 6 minutes and 1 GB for Damn Vulnerable DeFi, then failed at submission with `WORKFLOW_SUBMISSION_FAILED` / `DETACHED_ADMISSION_FAILED` / `RUN_EXISTS`, and left a partial run directory behind. The check asks the engine only once the project root has the engine's database, and any answer other than "this run exists" lets the launch go ahead as before. `clean` still leaves the engine record behind (#1257), so the ID stays taken.
+- **[runtime] [docs]** `validate`, `plan` and `run` now report a `TOPOLOGY_TIMEOUT_SHADOWS_DEFAULT` warning for each topology `timeout_seconds` pin, on a node or in a group's defaults, that is below the model profile `timeout_seconds` or `run.default_timeout_seconds` it overrides (#675). A pin wins even when it is the shorter window, so raising a default silently did not reach a pinned node: the packaged `goals`, `strategies`, `specialists` and `review` groups pin 7,200 seconds, and goal nodes kept timing out at two hours after a longer default was configured. The warning names the pin, the default it overrides and the nodes it applies to. Before, only a CI test on the packaged topologies checked this, so a project's own topology got no check.
 - **[config] [runtime] [prompts] [docs]** Adds `run.friction_log_enabled` (default `false`). When enabled, agent tasks are told to record Ultrafuzz, tooling, or instruction roadblocks with [Frog](https://github.com/wevm/frog) (`frog@1.1.0`, now a pinned dependency of `@ultrafuzz/runtime`) through a run-local `<run>/friction-bin/ultrafuzz-friction-log` command, which runs the Frog of the Ultrafuzz install that rendered the workflow. The command accepts only `log` and `list` with a few local options, refuses the built-in flags of incur, Frog's CLI framework, such as `--mcp`, where an option value belongs, points Frog at `<run>/friction` with `GIT_DIR` set to a path that does not exist, so entries land in `<run>/friction/.agents/friction-log/`, unsets Frog's GitHub and Postgres store variables and incur's `COMPLETE`, points `GH_CONFIG_DIR` at a path that does not exist, sets `NO_UPDATE_NOTIFIER=1`, and gives Frog `/dev/null` as stdin. It guards against misuse by mistake and enforces nothing: agents run unsandboxed (see `docs/security.md`), so `<run>/friction` is only where the command writes, and it removes no GitHub credential, because Frog falls back to `gh auth token`, which still finds a token kept in the system keyring. Every task preparation creates the directory and rewrites the command, best-effort: a friction log that cannot be prepared never fails the task. Frog is installed with the runtime even when the friction log is disabled, but runs leave it out of their trusted CLI closure and execution snapshot. Its `postgres` dependency moves the install paths of the Smithers packages that reach drizzle-orm (see the `resume` change above); the engine still runs with `SMITHERS_BACKEND=sqlite`. A disabled run renders byte-identical prompts. Entries are free-form agent text that the artifact secret scan does not cover, so check them for credentials, such as RPC URLs with API keys, before publishing anything. Read them as Markdown, or with the Frog pinned in the Ultrafuzz checkout and `GIT_DIR` set to a path that does not exist (see `docs/reference/configuration.md`), never a bare `npx frog`, and delete a malformed entry's directory, since Frog refuses every `log` and `list` while one is malformed (#172).
 - **[runtime] [docs]** A failed `ClaudeAgent` or `DeepSeekAgent` attempt now records the failure Claude Code states in its result, such as a contended OAuth refresh or a rejected DeepSeek API key, in the node's `last_error` and the attempt ledger's `failure_message`, for example `Claude run failed See https://smithers.sh/reference/errors (agent stated: Failed to refresh OAuth token: …)`. Before, the record was only the generic `Claude run failed` message, and the cause survived only in the Claude Code session transcript. `ultrafuzz inspect <run-id> --json` shows it as `data.state.nodes.<node-id>.last_error`, the dashboard shows it as the node's latest error, and a public Modal eval worker's `public-eval-diagnostics.json` carries it as the failed node's `failure_message`; `ultrafuzz status` and `ultrafuzz why` do not show it, because they relay the workflow engine's own summaries, which carry the error message but not its details. The thrown error Smithers classifies is unchanged, so quota parking, the auth disable and the retry behaviour from #1171 are unaffected; the statement travels as `details.agentStatedFailure` and gets the same secret redaction and length cap as `failure_message`. `ultrafuzz run` refuses a project whose stock `.smithers/agents` files predate this release (`CONTROLLER_SOURCE_UNTRUSTED`), so existing projects must re-run `ultrafuzz init`, which keeps an existing `ultrafuzz.toml`, topology and prompts, before their next run. A run launched by an earlier release records the statement only after `ultrafuzz resume <run-id> --refresh-controller`, which continues it with this release's workflow and adapters (#1084).
 - **[modal]** Moves `@grpc/grpc-js` to 1.14.5, past the High advisory GHSA-m9gg-hp2v-232j (`getAuthContext` could return unauthorized certificates as authorized in certain configurations).

docs/reference/configuration.md

@@ -317,6 +317,12 @@ topology node or group default.
 
 The effective agent timeout uses this precedence: a topology node or group
 timeout, then the model profile timeout, then `[run].default_timeout_seconds`.
+A topology pin wins even when it is shorter, so raising the profile or run
+default does not reach a pinned node. `validate`, `plan` and `run` report a
+`TOPOLOGY_TIMEOUT_SHADOWS_DEFAULT` warning for each node or group pin below the
+default it overrides. The packaged `goals`, `strategies`, `specialists` and `review`
+groups pin 7,200 seconds, so a `default_timeout_seconds` above that warns for
+them.
 
 Generated defaults may include `[models] synthesized_default = true` when the
 default profile was synthesized by the scaffold.

packages/runtime/src/plan-run.ts

@@ -86,6 +86,7 @@ import {
 } from "./data-governance.js";
 import { forgeGuardMetadata } from "./forge-guard.js";
 import { assertExpandedGraphRetryChains } from "./retry-chain.js";
+import { timeoutShadowingDiagnostics } from "./timeout-shadowing.js";
 import { projectArtifactSchemaDir } from "./init.js";
 import {
   assertLaunchCheckoutRevision,
@@ -194,6 +195,7 @@ export async function planRun(input: PlanRunInput, hooks: PlanRunHooks = {}) {
 
   let expandedGraph: ExpandedGraph;
   let catalog: PromptCatalog;
+  let timeoutDiagnostics: RuntimeDiagnostic[];
   try {
     const topology = transformTopologyForRun(
       loadTopology(projectRoot, {
@@ -215,6 +217,7 @@ export async function planRun(input: PlanRunInput, hooks: PlanRunHooks = {}) {
       configFingerprint: redactedConfigFingerprint
     });
     assertExpandedGraphRetryChains(resolved.config, expandedGraph);
+    timeoutDiagnostics = timeoutShadowingDiagnostics(topology, expandedGraph, resolved.config);
   } catch (error) {
     return runtimeFailure<PlanRunValue>([diagnosticFromError(error, "runtime", "RUN_PLAN_INVALID")]);
   }
@@ -568,7 +571,7 @@ export async function planRun(input: PlanRunInput, hooks: PlanRunHooks = {}) {
             }
           })
     },
-    preMaterializeDiagnostics
+    [...timeoutDiagnostics, ...preMaterializeDiagnostics]
   );
 }
 

packages/runtime/src/timeout-shadowing.ts

@@ -0,0 +1,70 @@
+import type { ResolvedConfig } from "@ultrafuzz/config";
+import type { ExpandedGraph, ProjectTopology } from "@ultrafuzz/topology";
+
+import type { RuntimeDiagnostic } from "./types.js";
+
+/**
+ * Warn about each topology `timeout_seconds` pin below the default it overrides (#675).
+ *
+ * A node's timeout resolves to its own pin, then its group's pin, then its model profile's
+ * `timeout_seconds`, then `run.default_timeout_seconds`. A pin therefore wins even when it is the
+ * shorter window, so raising the profile or run default silently does not reach a pinned node: the
+ * packaged `goals` and `strategies` pins kept high-reasoning goal nodes at 2h after #645 raised the
+ * profile default. Each pin is reported once, with the largest default it overrides and the
+ * agentic nodes it applies to.
+ */
+export function timeoutShadowingDiagnostics(
+  topology: ProjectTopology,
+  expanded: ExpandedGraph,
+  config: ResolvedConfig
+): RuntimeDiagnostic[] {
+  const topologyNodes = new Map(topology.nodes.map((node) => [node.id, node]));
+  const pins = new Map<
+    string,
+    { label: string; path: string; seconds: number; shadowed: { seconds: number; source: string }; nodes: Set<string> }
+  >();
+  for (const node of expanded.nodes) {
+    const pinned = node.timeoutSeconds;
+    const declared = topologyNodes.get(node.logicalId);
+    if (node.kind !== "agentic" || pinned === undefined || declared === undefined) continue;
+    const pin =
+      declared.timeout_seconds === undefined && declared.group !== undefined
+        ? {
+            key: `group:${declared.group}`,
+            label: `group \`${declared.group}\``,
+            path: `groups.${declared.group}.defaults.timeout_seconds`
+          }
+        : {
+            key: `node:${declared.id}`,
+            label: `node \`${declared.id}\``,
+            path: `nodes.${declared.id}.timeout_seconds`
+          };
+    // The default task compilation would apply without the pin: the profile's own timeout, else the
+    // run default. Expansion folds the run default into each fan-out entry, so read the profile itself.
+    const profileIds = node.modelFanout.length === 0 ? [undefined] : node.modelFanout.map((m) => m.modelProfileId);
+    for (const profileId of profileIds) {
+      const profileTimeout = profileId === undefined ? undefined : config.models.profiles[profileId]?.timeoutSeconds;
+      const shadowed =
+        profileId === undefined || profileTimeout === undefined
+          ? { seconds: config.run.defaultTimeoutSeconds, source: "`run.default_timeout_seconds`" }
+          : { seconds: profileTimeout, source: `model profile \`${profileId}\` \`timeout_seconds\`` };
+      if (shadowed.seconds <= pinned) continue;
+      const entry = pins.get(pin.key) ?? { ...pin, seconds: pinned, shadowed, nodes: new Set<string>() };
+      if (shadowed.seconds > entry.shadowed.seconds) entry.shadowed = shadowed;
+      entry.nodes.add(declared.id);
+      pins.set(pin.key, entry);
+    }
+  }
+  return [...pins.values()].map((pin) => {
+    const nodes = [...pin.nodes].sort();
+    const named = nodes.slice(0, 3).join(", ");
+    const applies = nodes.length > 3 ? `${named} and ${String(nodes.length - 3)} more nodes` : named;
+    return {
+      code: "TOPOLOGY_TIMEOUT_SHADOWS_DEFAULT",
+      message: `topology ${pin.label} pins timeout_seconds=${String(pin.seconds)}, below ${pin.shadowed.source}=${String(pin.shadowed.seconds)}; the pin wins, so ${applies} time out after ${String(pin.seconds)} seconds. Raise or remove the pin to use the longer default.`,
+      severity: "warning",
+      source: "topology",
+      path: pin.path
+    };
+  });
+}

packages/runtime/src/validate.ts

@@ -23,6 +23,7 @@ import type {
   ValidateProjectResult
 } from "./types.js";
 import { effectiveAuditPolicy } from "./audit-profile-policy.js";
+import { timeoutShadowingDiagnostics } from "./timeout-shadowing.js";
 import { agentRegistryRegisters, inspectAgentRegistry } from "./agent-registry.js";
 import { promptTextsForCatalog, transformPromptCatalogForRun, transformTopologyForRun } from "./topology-transform.js";
 import {
@@ -304,6 +305,7 @@ function validateTopologySurface(
       modelProfiles: config ? modelProfilesForTopology(config) : undefined,
       defaultModelProfileId: config?.retry.agents[0] ?? config?.models.default
     });
+    const timeoutDiagnostics = config === undefined ? [] : timeoutShadowingDiagnostics(topology, expanded, config);
     const selectedAgents = new Set(expanded.nodes.flatMap((node) => node.modelFanout.map((model) => model.agentRef)));
     for (const model of expanded.nodes.flatMap((node) => node.modelFanout)) {
       if (config === undefined) continue;
@@ -316,8 +318,10 @@ function validateTopologySurface(
     return {
       posture: postureFromDiagnostics(
         "topology",
-        "YAML topology v1 loads, validates, and expands",
-        executionDiagnostics
+        timeoutDiagnostics.length === 0
+          ? "YAML topology v1 loads, validates, and expands"
+          : "YAML topology v1 loads, validates, and expands, but a timeout_seconds pin is below the default it overrides",
+        [...executionDiagnostics, ...timeoutDiagnostics]
       ),
       selectedAgentRefs: [...selectedAgents].sort(),
       summary: {

packages/runtime/test/runtime.test.ts

@@ -10154,6 +10154,152 @@ test("force init rejects symlinked config and nested project files before overwr
   assert.equal(fs.readFileSync(path.join(topologyOutside, "topology.yml"), "utf8"), "outside\n");
 });
 
+// #675: a topology `timeout_seconds` pin outranks the profile and run defaults even when it is the
+// shorter window, so raising a default silently does not reach a pinned node.
+const timeoutShadowingWarnings = <T extends { code: string }>(diagnostics: readonly T[]): T[] =>
+  diagnostics.filter((diagnostic) => diagnostic.code === "TOPOLOGY_TIMEOUT_SHADOWS_DEFAULT");
+
+function setRunDefaultTimeout(project: string, seconds: number): void {
+  const configPath = path.join(project, "ultrafuzz.toml");
+  const config = fs.readFileSync(configPath, "utf8");
+  assert.match(config, /^\[run\]$/mu);
+  assert.doesNotMatch(config, /^default_timeout_seconds/mu);
+  fs.writeFileSync(
+    configPath,
+    config.replace(/^\[run\]$/mu, `[run]\ndefault_timeout_seconds = ${String(seconds)}`),
+    "utf8"
+  );
+}
+
+test("validate warns when a packaged group timeout pin is below the run default it overrides", async () => {
+  const project = tempProject();
+  initProject({ projectRoot: project, force: true });
+
+  const shipped = await validateProject({ projectRoot: project, env: {} });
+  assert.deepEqual(timeoutShadowingWarnings(shipped.diagnostics), []);
+
+  setRunDefaultTimeout(project, 14_400);
+  const raised = await validateProject({ projectRoot: project, env: {} });
+  const warnings = timeoutShadowingWarnings(raised.diagnostics);
+  assert.deepEqual(warnings.map((warning) => warning.path).sort(), [
+    "groups.goals.defaults.timeout_seconds",
+    "groups.review.defaults.timeout_seconds",
+    "groups.specialists.defaults.timeout_seconds",
+    "groups.strategies.defaults.timeout_seconds"
+  ]);
+  for (const warning of warnings) {
+    assert.equal(warning.severity, "warning");
+    assert.equal(warning.source, "topology");
+    assert.match(warning.message, /pins timeout_seconds=7200, below `run\.default_timeout_seconds`=14400/u);
+  }
+  assert.equal(raised.value?.policy_posture.topology.status, "warn");
+});
+
+// The default a pin overrides is the model profile's own `timeout_seconds` when it has one, which
+// task compilation prefers to the run default, so the warning names the profile's value.
+test("validate warns when a packaged group timeout pin is below the model profile timeout it overrides", async () => {
+  const project = tempProject();
+  initProject({ projectRoot: project, force: true });
+  const configPath = path.join(project, "ultrafuzz.toml");
+  const config = fs.readFileSync(configPath, "utf8");
+  assert.match(config, /^\[models\.default\]$/mu);
+  fs.writeFileSync(
+    configPath,
+    config.replace(/^\[models\.default\]$/mu, "[models.default]\ntimeout_seconds = 10800"),
+    "utf8"
+  );
+
+  const raised = await validateProject({ projectRoot: project, env: {} });
+  const warnings = timeoutShadowingWarnings(raised.diagnostics);
+  assert.deepEqual(warnings.map((warning) => warning.path).sort(), [
+    "groups.goals.defaults.timeout_seconds",
+    "groups.review.defaults.timeout_seconds",
+    "groups.specialists.defaults.timeout_seconds",
+    "groups.strategies.defaults.timeout_seconds"
+  ]);
+  for (const warning of warnings) {
+    assert.match(warning.message, /pins timeout_seconds=7200, below model profile `default` `timeout_seconds`=10800/u);
+  }
+
+  // A longer run default does not apply to a profile with its own timeout, so the warning keeps
+  // naming the profile.
+  setRunDefaultTimeout(project, 14_400);
+  const both = await validateProject({ projectRoot: project, env: {} });
+  const bothWarnings = timeoutShadowingWarnings(both.diagnostics);
+  assert.equal(bothWarnings.length, 4, JSON.stringify(both.diagnostics));
+  for (const warning of bothWarnings) {
+    assert.match(warning.message, /below model profile `default` `timeout_seconds`=10800/u);
+    assert.doesNotMatch(warning.message, /14400/u);
+  }
+});
+
+test("plan warns about group and node timeout pins below the default they override", async () => {
+  const project = tempProject();
+  initProject({ projectRoot: project, force: true });
+  const markdownOutput = `
+      - path: ${GENERIC_RUNTIME_MARKDOWN_PATH}
+        contract: ultrafuzz/nonempty-markdown@1
+        primary: true`;
+  fs.writeFileSync(
+    path.join(project, ".ultrafuzz", "topology.yml"),
+    `version: 2
+defaults:
+  strategy_loops: 1
+groups:
+  pinned:
+    label: Pinned
+    defaults:
+      timeout_seconds: 600
+nodes:
+  - id: __start__
+    kind: meta
+    role: start
+    depends_on: []
+  - id: grouped
+    kind: agentic
+    prompt: setup/runtime-fixture.md
+    group: pinned
+    depends_on: [__start__]
+    outputs:${markdownOutput}
+  - id: own-pin
+    kind: agentic
+    prompt: setup/runtime-fixture.md
+    timeout_seconds: 300
+    depends_on: [grouped]
+    outputs:${markdownOutput}
+  - id: long-pin
+    kind: agentic
+    prompt: setup/runtime-fixture.md
+    timeout_seconds: 7200
+    depends_on: [own-pin]
+    outputs:${markdownOutput}
+  - id: __finish__
+    kind: meta
+    role: finish
+    depends_on: [long-pin]
+`,
+    "utf8"
+  );
+  writeNeutralRuntimeFixturePrompt(project);
+
+  const plan = await planRun({ projectRoot: project, runId: "timeout-shadowing", env: {} });
+
+  assert.equal(plan.ok, true, JSON.stringify(plan.diagnostics));
+  const warnings = timeoutShadowingWarnings(plan.diagnostics);
+  assert.deepEqual(
+    warnings.map((warning) => warning.path),
+    ["groups.pinned.defaults.timeout_seconds", "nodes.own-pin.timeout_seconds"],
+    JSON.stringify(plan.diagnostics)
+  );
+  const [groupWarning, nodeWarning] = warnings;
+  assert.ok(groupWarning && nodeWarning);
+  assert.match(
+    groupWarning.message,
+    /group `pinned` pins timeout_seconds=600, below `run\.default_timeout_seconds`=3600; the pin wins, so grouped time out after 600 seconds/u
+  );
+  assert.match(nodeWarning.message, /node `own-pin` pins timeout_seconds=300/u);
+});
+
 test("plan creates run layout, graph fingerprint, and rendered prompt before Smithers submission", async () => {
   const project = tempProject();
   initProject({ projectRoot: project, force: true });