[mfrancisc]

Fixes #8111

Problem

Both forges' built-in-stage dispatch checks only ever read kill_switch
from .fullsend/config.yaml. A central operator setting
kill_switch: true in .fullsend/config.base.yaml alone had no effect
on built-in stages (triage, code, review, fix, retro, prioritize).

This was a parity gap, not a greenfield need: IsKillSwitchActive()
already layers overlay (config.yaml) → base (config.base.yaml) →
compiled default (false), and that resolution order is already
honored by the Jira polling path, the managed-config-push safety gate,
and the harness-dispatch job in this very workflow file. Only the
route-stage kill-switch checks on GitHub and GitLab were out of sync.

Fix

• GitHub (reusable-dispatch.yml): add config.base.yaml to the
  route job's sparse-checkout list, and extend the kill-switch check to
  consult it when config.yaml is silent on the key.
• GitLab (run-agent-job.sh): fetch config.base.yaml the same
  way config.yaml is fetched, and apply the same override semantics.

Both checks now follow IsKillSwitchActive()'s overlay → base →
default resolution order: an explicit kill_switch value in
config.yaml wins if set (including false), otherwise
config.base.yaml applies, otherwise false.

Known limitation — boolean spelling, not resolution order: Go's
*bool decoding (gopkg.in/yaml.v3) and GitLab's PyYAML-based check
both already treat yes/on as true and no/off as false. The
pre-PR GitHub check (yq '.kill_switch // false', no -o=json) was
case-sensitive — it did not recognize True/TRUE — but accidentally
halted on quoted kill_switch: "true"/'true' as a side effect of
yq's non-JSON output rendering a quoted string identically to a bare
boolean, not because it understood quoting. This PR's -o=json switch
fixes the null-spelling and case-variant gaps, adds yes/no
recognition, and strips a quoted scalar's surrounding quotes before
comparing, so true/"true"/'true'/yes/"yes" are all correctly
recognized on GitHub. on/off and the single-letter y/n forms
remain unhandled by deliberate choice — low real-world likelihood as
kill_switch values, and chasing exhaustive parity with go-yaml's own
lenient-but-uneven bool coercion (which separately diverges on bare
1, which Go rejects as a decode error) isn't worth the complexity
here. See
this review thread
for the full analysis.

Unchanged behavior: GitLab's pre-existing fail-open handling of a
YAML parse error (log a WARNING and treat as "no kill-switch",
continuing) is intentionally left as-is — not expanded to fail-closed.
This diverges from docs/glossary.md's "Fail-Open" entry, which
classifies kill switch as a fail-closed gate; closing that gap is left
for separate follow-up (noted in the issue).

Testing

• internal/scaffold/gitlab_job_scripts_test.go: new table-driven
  subprocess test exercising run-agent-job.sh's kill-switch check
  end-to-end — base-only activation, overlay-overrides-base, neither
  set, and the malformed-YAML fail-open path.
• e2e/behaviour/features/dispatch/fork-dispatch.feature: new scenario
  proving a config.base.yaml-only kill switch blocks GitHub
  dispatch, with supporting step definitions and cleanup/validation
  wiring in pkg/behaviourtest/.
• internal/scaffold/github_kill_switch_test.go /
  internal/scaffold/gitlab_kill_switch_test.go: added rows for every
  YAML null spelling (bare key, ~, Null), a malformed-base-survives
  case, and the yes/"yes"/no/on/off/y/n/"true"/'true'
  boolean-spelling matrix (overlay and base) on both forges, locking in
  the known limitation above.

[github-actions]

E2E tests did not run

E2E tests run automatically for org/repo members and collaborators on pull requests.

For other contributors, a maintainer with write access must add the ok-to-test label after the latest push.

See E2E testing guide for details.

[qodo-code-review]

PR Summary by Qodo

Honor base-config kill switch in GitHub and GitLab dispatch

🐞 Bug fix 🧪 Tests 🕐 20-40 Minutes

AI Description

• Block GitHub and GitLab dispatch when only the base config enables the kill switch.
• Preserve explicit overlay overrides, including false, and GitLab’s existing YAML-error handling.
• Add dispatch regression tests and clean up base-config changes after behaviour scenarios.

Diagram

graph TD
  GH["GitHub route"] --> OV["config.yaml"] --> OC{"Key present?"} -->|yes| GATE["Dispatch gate"]
  GL["GitLab job"] --> OV
  OC -->|no| BASE["config.base.yaml"] --> BC{"Key present?"} -->|yes| GATE
  BC -->|no| DEF["Default false"] --> GATE

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Shared config-resolution CLI

• ➕ Could reuse Go’s layered configuration semantics across both forges.
• ➖ Would add a CLI bootstrap dependency to early dispatch gates and broaden the scope of this fix.

Recommendation: Keep the forge-local checks for this targeted dispatch fix: both paths already read trusted configuration before agent execution. A shared resolver merits consideration only if further dispatch gates need the same logic; GitLab’s existing YAML-error policy remains a separate follow-up.

Files changed (9) +547 / -14

Bug fix (2) +48 / -11

reusable-dispatch.ymlCheck the base kill switch in GitHub route dispatch +20/-5

Check the base kill switch in GitHub route dispatch

• Adds 'config.base.yaml' to the trusted sparse checkout. The route gate now uses its kill-switch value only when 'config.yaml' omits the key, so an explicit overlay 'false' still permits dispatch.

.github/workflows/reusable-dispatch.yml

run-agent-job.shResolve the GitLab kill switch from overlay and base config +28/-6

Resolve the GitLab kill switch from overlay and base config

• Fetches both config files from the pinned default-branch commit and applies overlay-first kill-switch precedence. YAML parse failures retain the script’s existing warning-and-continue behavior.

internal/scaffold/fullsend-repo-gitlab/.gitlab/ci/scripts/run-agent-job.sh

Tests (7) +499 / -3

fork-dispatch.featureCover base-only kill-switch blocking for fork PRs +19/-0

Cover base-only kill-switch blocking for fork PRs

• Adds a behaviour scenario that enables the kill switch in 'config.base.yaml', labels a fork pull request, and asserts that its harness agent does not run.

e2e/behaviour/features/dispatch/fork-dispatch.feature

gitlab_job_scripts_test.goExercise GitLab kill-switch precedence end to end +152/-0

Exercise GitLab kill-switch precedence end to end

• Adds a configurable Git origin fixture and subprocess cases for base-only activation, an explicit overlay override, absent settings, and malformed YAML.

internal/scaffold/gitlab_job_scripts_test.go

cleanup.goValidate and clean up base-config kill-switch state +16/-1

Validate and clean up base-config kill-switch state

• Slot validation now reads layered configuration so a base-only active switch is detected. Scenario cleanup deactivates a base switch enabled by the test.

pkg/behaviourtest/steps/cleanup.go

cleanup_test.goTest base-config slot validation and cleanup +87/-2

Test base-config slot validation and cleanup

• Extends the fake SCM to return distinct base-file content. Adds coverage for base-switch cleanup, cleanup failure, stale-state detection, and a missing base file.

pkg/behaviourtest/steps/cleanup_test.go

dispatch.goAdd behaviour steps to activate a base-config kill switch +73/-0

Add behaviour steps to activate a base-config kill switch

• Registers a step that sets 'kill_switch: true' in 'config.base.yaml' without changing the overlay. Adds a matching deactivation helper and handles a base file that does not yet exist.

pkg/behaviourtest/steps/dispatch.go

dispatch_test.goTest base-config kill-switch behaviour helpers +144/-0

Test base-config kill-switch behaviour helpers

• Covers activation and deactivation, missing files, SCM failures, and missing repository identity for the new behaviour-test helpers.

pkg/behaviourtest/steps/dispatch_test.go

world.goTrack base-config activation separately in scenario state +8/-0

Track base-config activation separately in scenario state

• Adds a flag recording whether a scenario activated the base kill switch, allowing cleanup to reset it independently of the overlay switch.

pkg/behaviourtest/world/world.go

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (1) 📎 Requirement gaps (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

1. Broken base config turns off a repo's own kill switch ✗ Dismissed

Description

run-agent-job.sh parses config.yaml and config.base.yaml in one guarded Python call, so a
parse error in either file sets KILL_SWITCH to false even when the other file explicitly sets
kill_switch: true. A malformed base file can therefore bypass a valid overlay switch, and a
malformed overlay can likewise bypass a valid base switch, allowing dispatch to continue.

Code

internal/scaffold/fullsend-repo-gitlab/.gitlab/ci/scripts/run-agent-job.sh[R322-335]

+overlay = load(os.environ.get('OVERLAY_YAML', ''))
+base = load(os.environ.get('BASE_YAML', ''))
+
+if 'kill_switch' in overlay:
+    active = overlay.get('kill_switch')
+elif 'kill_switch' in base:
+    active = base.get('kill_switch')
+else:
+    active = False
+
+print('true' if str(active).lower() in ('true', 'yes', '1', 'on') else 'false')
+"); then
+    echo "WARNING: invalid .fullsend/config.yaml or .fullsend/config.base.yaml — treating as unconfigured (no kill-switch)"
    KILL_SWITCH="false"

Relevance

●● Moderate

The concern conflicts with the PR’s explicitly intentional fail-open behavior; team response remains
uncertain.

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

load() parses both YAML files before checking which value takes precedence, so an exception from
either file causes the Python call to fail and the Bash guard to set KILL_SWITCH="false".
Previously, a valid overlay kill_switch: true halted dispatch without parsing the base file; the
Go accessor likewise returns an explicitly set overlay value before consulting its parent.

internal/scaffold/fullsend-repo-gitlab/.gitlab/ci/scripts/run-agent-job.sh[315-341]
internal/scaffold/fullsend-repo-gitlab/.gitlab/ci/scripts/run-agent-job.sh[319-335]
internal/config/interfaces.go[233-244]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
A parse error in either config file currently turns off a valid `kill_switch: true` in the other file, allowing dispatch to continue.

## Fix Focus Areas
- internal/scaffold/fullsend-repo-gitlab/.gitlab/ci/scripts/run-agent-job.sh[315-341]

## Recommended Fix
Parse the overlay first, handling a parse failure with a warning that names the file. If the overlay parses and explicitly sets `kill_switch`, use that value without loading the base file. Otherwise, parse the base with its own error handling and use its value if it sets the key. Fall back to `false` only when no successfully parsed layer sets the key. Add tests for an overlay with `kill_switch: true` and malformed base YAML, and for a malformed overlay with base `kill_switch: true`.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. An empty repo config hides the base kill switch on GitHub ✓ Resolved

Description

OVERLAY_KS=$(yq '.kill_switch' .fullsend/config.yaml) prints nothing for an empty or comment-only
file, because it has no YAML document. The check [[ "$OVERLAY_KS" != "null" ]] then treats the
empty string as an explicit value, so config.base.yaml is skipped and KILL_SWITCH becomes "". In
that case, an operator's base kill_switch: true does not halt built-in stages, while
IsKillSwitchActive() (nil overlay falls through to base) would halt them.

Code

.github/workflows/reusable-dispatch.yml[R505-509]

+            OVERLAY_KS=$(yq '.kill_switch' .fullsend/config.yaml)
+          fi
+          if [[ "$OVERLAY_KS" != "null" ]]; then
+            KILL_SWITCH="$OVERLAY_KS"
+          elif [[ -f .fullsend/config.base.yaml ]]; then

Relevance

●●● Strong

Empty YAML output bypasses the intended base fallback and contradicts documented layered
configuration semantics.

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The fallback only triggers when the overlay output is exactly the string null. Any other output,
including the empty output yq gives for an empty document, is taken as the overlay's final value.
The Go loader unmarshals an empty overlay into a nil KillSwitch and inherits from the base.

.github/workflows/reusable-dispatch.yml[502-519]
internal/config/interfaces.go[734-756]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The GitHub kill-switch check treats an empty yq output (an empty or comment-only config.yaml) as an explicit overlay value, so config.base.yaml is skipped.

## Fix Focus Areas
- .github/workflows/reusable-dispatch.yml[502-519]

## Recommended Fix
Use `yq '.kill_switch // "null"'` or a similar default, or change the condition to `if [[ "$OVERLAY_KS" == "true" || "$OVERLAY_KS" == "false" ]]`, so the overlay only wins when it holds a real boolean. Apply the same rule to BASE_KS.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

3. GitLab ignores the base kill switch when the repo sets it to null ✓ Resolved

Description

The Python check uses 'kill_switch' in overlay, so an overlay with kill_switch: or `kill_switch:
null counts as set. Its value is then None`, which resolves to false, and the base layer is never
read. IsKillSwitchActive() (KillSwitch *bool, null becomes nil and falls through to the parent)
and the new GitHub yq check (!= "null") both fall back to config.base.yaml in this case, so a
central kill switch blocks GitHub dispatch but not GitLab dispatch.

Code

internal/scaffold/fullsend-repo-gitlab/.gitlab/ci/scripts/run-agent-job.sh[R325-330]

+if 'kill_switch' in overlay:
+    active = overlay.get('kill_switch')
+elif 'kill_switch' in base:
+    active = base.get('kill_switch')
+else:
+    active = False

Relevance

●●● Strong

Null overlay handling diverges from the documented layered kill-switch semantics across GitHub and
GitLab.

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

In Go, KillSwitch *bool unmarshals YAML null as nil, and loadPerRepoLayers sets the overlay's
parent to the base, so a nil value inherits from the base. On GitHub, yq '.kill_switch' prints
null for an explicit null, which hits the base fallback. In the Python check, `'kill_switch' in
{'kill_switch': None}` is True, so the base is skipped.

internal/config/config.go[796-796]
internal/config/interfaces.go[725-756]
.github/workflows/reusable-dispatch.yml[505-513]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
GitLab's kill-switch resolution treats an explicit null `kill_switch` in config.yaml as set (false). IsKillSwitchActive() and the GitHub check treat it as unset and fall back to config.base.yaml.

## Fix Focus Areas
- internal/scaffold/fullsend-repo-gitlab/.gitlab/ci/scripts/run-agent-job.sh[325-330]

## Recommended Fix
Change the checks to `if overlay.get('kill_switch') is not None:` and `elif base.get('kill_switch') is not None:`. Add a test case for an overlay of `kill_switch: null` with a base of `kill_switch: true` that expects dispatch to halt.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

4. Base kill switches stay undocumented ✓ Resolved

Description

The dispatch change makes .fullsend/config.base.yaml control GitHub and GitLab kill-switch
behavior, but the repository documentation still describes the kill switch as coming only from
config.yaml. Users following the fork-dispatch guide or configuration inventory can therefore miss
the central emergency-stop path and misunderstand which setting takes precedence.

Code

.github/workflows/reusable-dispatch.yml[R509-518]

+          elif [[ -f .fullsend/config.base.yaml ]]; then
+            BASE_KS=$(yq '.kill_switch' .fullsend/config.base.yaml)
+            if [[ "$BASE_KS" != "null" ]]; then
+              KILL_SWITCH="$BASE_KS"
            fi
          fi
+          if [[ "$KILL_SWITCH" == "true" ]]; then
+            echo "::error::Kill switch is active — all agent dispatch halted"
+            echo "::error::Set kill_switch: false in .fullsend/config.yaml or .fullsend/config.base.yaml to resume"
+            exit 1

Relevance

●●● Strong

The change alters user-facing configuration behavior, and repository precedent accepts updating
related documentation.

PR-#7056

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The changed workflow now falls back to .fullsend/config.base.yaml for kill_switch, while the
configuration inventory still says the bash check reads config.yaml only and the behaviour-testing
guide says the kill switch is set in config.yaml.

Rule 2748504: Update docs/ references when changing user-facing behavior
docs/contributing/config-source-inventory.md[100-100]
docs/guides/dev/behaviour-testing.md[392-402]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The dispatch workflows now consult `.fullsend/config.base.yaml` when `.fullsend/config.yaml` does not explicitly set `kill_switch`, but existing documentation still describes only the overlay file and the old workflow behavior.

## Fix Focus Areas
- .github/workflows/reusable-dispatch.yml[509-518]
- docs/contributing/config-source-inventory.md[100-100]
- docs/guides/dev/behaviour-testing.md[392-402]

## Recommended Fix
Update the configuration inventory and behaviour-testing guide to document overlay-over-base precedence, explicit `false` overrides, and base-only activation. Update any related configuration reference or operational guide examples so the documented dispatch behavior matches both forge implementations.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

View medium (2)

5. Base-config read failures pass slot checks ✗ Dismissed

Description

ValidateSlotClean treats every error reading config.base.yaml as an absent file rather than
distinguishing not-found from other failures. When the SCM read fails while a base-layer switch is
active, allocation proceeds without checking that state.

Code

pkg/behaviourtest/steps/cleanup.go[R70-74]

+	baseData, err := w.SCM.GetFileContent(context.Background(),
+		w.Org, w.RepoName, filepath.Join(".fullsend", "config.base.yaml"))
+	if err != nil {
+		baseData = nil
+	}

Relevance

●●● Strong

Treating non-not-found SCM failures as absent can bypass safety validation; similar behavior-test
hardening is accepted.

PR-#6171

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

All base read errors discard the data, while the effective-switch check runs only after parsing. The
scenario setup calls this validator when a repository is leased.

pkg/behaviourtest/steps/cleanup.go[51-86]
pkg/behaviourtest/steps/triage.go[43-61]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Slot validation silently skips the base layer on any SCM read error.
## Fix Focus Areas
- pkg/behaviourtest/steps/cleanup.go[67-75]
## Recommended Fix
Treat only `forge.IsNotFound(err)` as an absent base file. Return other read errors so the slot is not accepted without validation, and test that path.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

6. Large configs can bypass the GitLab switch ✗ Dismissed

Description

run-agent-job.sh passes both complete config files as environment variables when launching Python.
If a config exceeds the process environment limit, Python cannot start and the command-failure
handler treats an active switch as unconfigured.

Code

internal/scaffold/fullsend-repo-gitlab/.gitlab/ci/scripts/run-agent-job.sh[R315-317]

+if [ -n "${CONFIG_YAML}" ] || [ -n "${CONFIG_BASE_YAML}" ]; then
+  if ! KILL_SWITCH=$(OVERLAY_YAML="${CONFIG_YAML}" BASE_YAML="${CONFIG_BASE_YAML}" python3 -c "
+import os, sys, yaml

Relevance

●● Moderate

The environment-size failure mode is plausible, but no closely matching team precedent confirms
acceptance.

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The new invocation exports both fetched documents in full; its catch-all failure branch sets the
switch to false. The subsequent role check instead supplies config on stdin.

internal/scaffold/fullsend-repo-gitlab/.gitlab/ci/scripts/run-agent-job.sh[306-335]
internal/scaffold/fullsend-repo-gitlab/.gitlab/ci/scripts/run-agent-job.sh[349-353]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Passing whole YAML documents through the environment can prevent Python from starting and fail open.
## Fix Focus Areas
- internal/scaffold/fullsend-repo-gitlab/.gitlab/ci/scripts/run-agent-job.sh[315-335]
## Recommended Fix
Supply config data through stdin or temporary files instead of environment variables. Do not classify a failure to start the checker as a YAML parse error that disables the switch.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

7. A workflow change lacks CI guide sign-off 📘 Rule violation § Compliance

Description

The pull request modifies .github/workflows/reusable-dispatch.yml by changing sparse checkout and
kill-switch execution logic without stating that docs/contributing/ci-workflows.md was consulted.
This leaves the workflow change without the checklist’s required evidence for workflow security and
refactoring conventions.

Code

.github/workflows/reusable-dispatch.yml[R498-505]

+          # Mirrors internal/config's IsKillSwitchActive(): if
+          # config.yaml sets kill_switch at all, that value is final
+          # and config.base.yaml is ignored; config.base.yaml is only
+          # consulted when config.yaml is silent on the key.
+          KILL_SWITCH="false"
+          OVERLAY_KS="null"
          if [[ -f .fullsend/config.yaml ]]; then
-            KILL_SWITCH=$(yq '.kill_switch // false' .fullsend/config.yaml)
-            if [[ "$KILL_SWITCH" == "true" ]]; then
-              echo "::error::Kill switch is active — all agent dispatch halted"
-              echo "::error::Set kill_switch: false in .fullsend/config.yaml to resume"
-              exit 1
+            OVERLAY_KS=$(yq '.kill_switch' .fullsend/config.yaml)

Relevance

● Weak

Recent workflow-guide sign-off finding was rejected for this repository’s workflow changes.

PR-#7845

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The PR changes a file under .github/workflows/, including workflow checkout inputs and inline
dispatch logic, but the supplied PR description contains no reference to
docs/contributing/ci-workflows.md.

Rule 3201588: Consult CI workflow guide when modifying GitHub Actions workflows or secrets
.github/workflows/reusable-dispatch.yml[498-505]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The pull request changes a GitHub Actions workflow but provides no explicit evidence that `docs/contributing/ci-workflows.md` was consulted.

## Fix Focus Areas
- .github/workflows/reusable-dispatch.yml[498-518]

## Recommended Fix
Add an explicit note to the PR description, review checklist, or an appropriate code comment stating that `docs/contributing/ci-workflows.md` was reviewed and that the sparse-checkout and workflow-step changes follow its conventions.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Dismiss ↗ | View ↗

---

8. New e2e scenario passes without the route-job fix ✓ Resolved

Description

The new fork-PR scenario registers a custom CEL-triggered harness and only asserts that its
harness-run artifact is absent, rather than exercising the changed route-stage "Check kill switch"
step. When config.base.yaml enables the kill switch, that custom harness goes through the separate
harness-dispatch job, which already honours the base setting, so the scenario would also pass on
the pre-fix workflow without testing built-in-stage routing.

Code

e2e/behaviour/features/dispatch/fork-dispatch.feature[R93-110]

+  Scenario: Fork PR kill switch in config.base.yaml blocks all harnesses
+    Given a custom harness "fork-pr-killswitch-base" with:
+      """
+      agent: agents/triage.md
+      role: triage
+      slug: fullsend-ai-fork-pr-killswitch-base
+      model: opus
+      image: ghcr.io/fullsend-ai/fullsend-sandbox:latest
+      policy: policies/base.yaml
+      trigger: |
+        event.entity.kind == "change_proposal"
+        && event.transition.kind == "label_changed"
+        && event.transition.label.name == "ready-for-fork-killswitch-base"
+      """
+    And the kill switch is active in config.base.yaml
+    When a fork pull request is opened
+    And the fork pull request is labeled "ready-for-fork-killswitch-base"
+    Then the harness "fork-pr-killswitch-base" agent did not run

Relevance

● Weak

A closely matching artifact-only negative e2e assertion finding was rejected.

PR-#3820

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The scenario defines a custom harness with a CEL trigger: and asserts only that the
fork-pr-killswitch-base agent did not run. Custom harness agents are dispatched through the
separate harness-dispatch job, which checks out all of .fullsend/ and already applied the base
kill switch; the changed bash check is instead in the route job used by built-in stages such as
triage, code and review.

e2e/behaviour/features/dispatch/fork-dispatch.feature[93-110]
.github/workflows/reusable-dispatch.yml[494-519]
pkg/behaviourtest/steps/dispatch.go[414-430]
.github/workflows/reusable-dispatch.yml[1539-1568]
.github/workflows/reusable-dispatch.yml[1671-1698]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The new fork-PR regression scenario tests custom-harness dispatch, which already honours `config.base.yaml`, rather than the changed route-job kill-switch check for built-in stages.

## Fix Focus Areas
- e2e/behaviour/features/dispatch/fork-dispatch.feature[93-110]

## Recommended Fix
Add or replace the scenario with one that triggers a built-in stage, such as triage on an issue label, while only `config.base.yaml` sets `kill_switch: true`. Assert that the route job halts at "Check kill switch" and that the built-in agent did not run.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

Context sources

✅ Tickets: 🎫 dispatch: kill_switch check on GitHub and GitLab ignores config.base.yaml

✅ Compliance rules (platform): 89 rules

✅ Skills: 5 invoked

  writing-user-docs

  writing-adrs

  check-patch-coverage

  cutting-releases

  adr-corner

✅ Cross-repo context — repo relationships

  Explored: repo: fullsend-ai/experiments (sha: 3f1614fa) — View relationship

  Explored: repo: fullsend-ai/.fullsend (sha: 7c163cad) — View relationship

  Explored: repo: fullsend-ai/metrics (sha: 52f238e4) — View relationship

  Explored: repo: fullsend-ai/pi-xai-vertex (sha: 8f50bdda) — View relationship

  Explored: repo: fullsend-ai/scribe (sha: 0f3eb528) — View relationship

  Explored: repo: fullsend-ai/agents (sha: 56b9aa3a) — View relationship

  Explored: repo: fullsend-ai/mlflow-assessments (sha: 46f42987) — View relationship

  Explored: repo: fullsend-ai/pi-anthropic-vertex (sha: 960eff35) — View relationship

Review mode: Auto: 🧠 Deep: Multiple independent workflow, shell, Go, and end-to-end behavior paths changed.

Tip of the day

💡 Did you know, you can hide the parts of a finding you never read, like the evidence or the agent prompt

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[rh-hemartin]

/fs-review

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 7:18 AM UTC · Completed 7:29 AM UTC

Commit: a1a408e · View workflow run →

Runtime: pi · Model: sonnet → claude-sonnet-5-5 · Effort: high · Cost: $2.93

[fullsend-ai-review]

Risk Assessment: moderate (2/5)

Details

Moderate risk driven by CI workflow and protected path modifications with high historical churn in dispatch files, mitigated by strong test coverage, zero dependency changes, and complete alignment with linked issue acceptance criteria.

Previous run

Risk Assessment: moderate (2/5)

Details

Moderate risk driven by CI workflow and protected path modifications with high historical churn in dispatch files, mitigated by strong test coverage, zero dependency changes, and complete alignment with linked issue acceptance criteria.

Previous run (2)

Risk Assessment: moderate (2/5)

Details

Modifies CI dispatch workflow and runner scripts touching protected paths with high historical churn and regression fixes, but well-aligned with linked issue scope, fresh discussion, and comprehensive test coverage yielding a moderate risk score.

Previous run (3)

Risk Assessment: moderate (2/5)

Details

Modifies CI dispatch workflow and runner scripts touching protected paths with high historical churn and regression fixes, but well-aligned with linked issue scope, fresh discussion, and comprehensive test coverage yielding a moderate risk score.

[fullsend-ai-review]

Review

Findings

Medium

• [protected-path] .github/workflows/reusable-dispatch.yml — This PR modifies .github/workflows/reusable-dispatch.yml, which is under the protected path .github/. The PR links issue dispatch: kill_switch check on GitHub and GitLab ignores config.base.yaml #8111 and its description explains the rationale, so context is sufficient, but human approval is always required for protected-path changes regardless of context.

Low

• [test-inadequate] internal/scaffold/github_kill_switch_test.go:222 — The halt cases assert only that Bash returned an error. A yq error or another script failure therefore satisfies the test without proving that the kill-switch guard activated. In particular, the malformed-base case does not prove that the base was ignored.
  Remediation: For wantHalt cases, also assert that output contains the kill-switch-specific error message and that the process exits with the expected status.
• [logic-error] .github/workflows/reusable-dispatch.yml:534 — The check intentionally recognizes only true/yes, but Go's yaml.v3 decoding into *bool also accepts on and y (including their supported case variants). Thus a valid base configuration such as kill_switch: on halts custom harness dispatch through the Go accessor while allowing built-in stages through this route check. The stated limitation leaves scalar-decoding parity incomplete.
  Remediation: Recognize the remaining affirmative spellings accepted by Go's bool decoder and change the on/y test cases to expect dispatch to halt.
• [missing-test] e2e/behaviour/features/dispatch/fork-dispatch.feature:109 — The new scenario checks only a custom harness, which runs through the existing Go-based harness-dispatch path rather than the modified built-in route check. It would still pass if the GitHub workflow changes were reverted. Moreover, agent-did-not-run accepts a missing or failed dispatch run, so it does not establish that the harness dispatch completed successfully with an empty matrix.
  Remediation: Add an end-to-end assertion for a built-in stage blocked by the base kill switch, and require successful empty-matrix completion for the custom-harness denial scenario.
• [fail-open] .github/workflows/reusable-dispatch.yml:534 — The kill-switch gate still permits dispatch for unrecognized scalar values. Both OVERLAY_KS and BASE_KS are lowercased and stripped of surrounding quotes, but only true/yes halt dispatch; on/y and invalid values silently proceed. An unrecognized non-null overlay value also masks an active base value. YAML parsing errors abort under set -e, but successful parsing does not validate either layer's value. This remains a trusted-configuration misconfiguration risk rather than an external authorization bypass.
  Remediation: In a follow-up, validate the selected value against the supported boolean vocabulary and halt on invalid values rather than interpreting everything except true/yes as false.
• [fail-open] internal/scaffold/fullsend-repo-gitlab/.gitlab/ci/scripts/run-agent-job.sh:343 — A parsing failure still explicitly sets KILL_SWITCH=false and continues dispatch. A malformed overlay bypasses even an active base switch, and a malformed consulted base likewise permits dispatch. Issue dispatch: kill_switch check on GitHub and GitLab ignores config.base.yaml #8111 explicitly defers this existing limitation; the code is unchanged since the prior review.
  Remediation: Track the deferred hardening separately: terminate dispatch when a consulted configuration layer cannot be parsed or its kill_switch value cannot be validated.
• [code-organization] pkg/behaviourtest/world/world.go:138 — KillSwitchBaseActivated was added to world.World as a scenario-scoped state tracking field for cleanup, but unlike sibling scenario-level fields (such as KillSwitchActivated, RuntimeOverridden, etc.), it is not reset in resetScenarioWorld in pkg/behaviourtest/suite/init.go. World.Clone() documents that callers must call resetScenarioWorld to zero scenario fields between runs.
  Remediation: Add w.KillSwitchBaseActivated = false to resetScenarioWorld in pkg/behaviourtest/suite/init.go alongside KillSwitchActivated.

---

Next steps:

• /fs-fix — agent addresses review findings automatically
• /fs-fix <your instruction> — agent fixes with your specific guidance
• Push commits directly — review re-runs automatically on push
• /fs-fix-stop — disable automatic fix runs for this PR

Previous run

Review

Findings

Medium

• [protected-path] .github/workflows/reusable-dispatch.yml:497 — This PR modifies a protected path (sparse-checkout list and the "Check kill switch" step). It links issue dispatch: kill_switch check on GitHub and GitLab ignores config.base.yaml #8111 and explains the rationale, but human approval is always required for protected-path changes regardless of context.

Low

• [test-inadequate] internal/scaffold/github_kill_switch_test.go:222 — Every wantHalt fixture treats any subprocess error as proof the kill switch fired, so e.g. overlay true survives a malformed base file would still pass if a regression eagerly parsed the malformed base and exited on a yq error. The GitLab tests already assert the specific diagnostic.
  Remediation: For wantHalt fixtures, also require the Kill switch is active diagnostic in the output.
• [code-organization] pkg/behaviourtest/world/world.go:138 — The new scenario-scoped KillSwitchBaseActivated field is not reset in suite.resetScenarioWorld (pkg/behaviourtest/suite/init.go:139), unlike KillSwitchActivated and the other scenario flags. A stale true would make cleanup touch config.base.yaml in an unrelated scenario.
  Remediation: Add w.KillSwitchBaseActivated = false to resetScenarioWorld.
• [logic-error] .github/workflows/reusable-dispatch.yml:534 — Only true/yes are recognized, while Go's *bool decoding also accepts on/y (GitLab misses y). Predates the PR, is narrowed by it, and is documented as a deliberate known limitation; advisory.
• [fail-open] .github/workflows/reusable-dispatch.yml:535 — Non-boolean or unrecognized kill_switch values (e.g. [], garbage) are treated as inactive, and an invalid overlay masks an active base. Config is read from the trusted base SHA and this behavior predates the PR; advisory, suitable for a follow-up.
• [fail-open] internal/scaffold/fullsend-repo-gitlab/.gitlab/ci/scripts/run-agent-job.sh:343 — YAML/Python failures and git show errors fall through to "no kill switch". Pre-existing and explicitly deferred by issue dispatch: kill_switch check on GitHub and GitLab ignores config.base.yaml #8111 (diverges from docs/glossary.md's fail-closed classification); advisory.
• [missing-test] e2e/behaviour/features/dispatch/fork-dispatch.feature:111 — The new scenario uses a custom CEL harness already covered by the Go accessor path, so it likely does not exercise the changed built-in-stage route step or sparse-checkout addition, and its absence assertion is not paired with the empty-matrix step. The scaffold subprocess tests cover the changed scripts; advisory.

---

Next steps:

• /fs-fix — agent addresses review findings automatically
• /fs-fix <your instruction> — agent fixes with your specific guidance
• Push commits directly — review re-runs automatically on push
• /fs-fix-stop — disable automatic fix runs for this PR

Previous run (2)

Review

Findings

Medium

• [fail-open] .github/workflows/reusable-dispatch.yml:526 — The extracted kill_switch values are not validated. Only JSON true (case-insensitive) or the JSON string "yes" halts dispatch. A non-null but invalid overlay value (for example kill_switch: "", [] or {}) shadows a base kill_switch: true and lets built-in stages dispatch. There is also a small regression: the old yq '.kill_switch // false' printed a quoted "true" as the raw string true, so it halted, but yq -o=json prints "true" with the quotes, which no longer matches. A quoted kill_switch: "true" in config.yaml therefore no longer halts built-in stages. Exploiting this needs a maintainer to commit the odd value to the trusted default-branch config, and the PR documents the boolean-spelling divergence as a deliberate limitation, but the quoted-"true" regression is not covered by that note.
  Remediation: At minimum, also match the quoted "true" JSON string so the previous halting behavior is kept. Preferably treat any non-null kill_switch value that is not a recognized boolean spelling as an error (exit 1). Add test cases in internal/scaffold/github_kill_switch_test.go for "", "true" and [] overlay values against an active base.

• [protected-path] .github/workflows/reusable-dispatch.yml — This PR modifies a protected governance/infrastructure path (sparse-checkout list and the "Check kill switch" step). The PR links issue dispatch: kill_switch check on GitHub and GitLab ignores config.base.yaml #8111 and explains the rationale, so context is sufficient, but human approval is always required for protected-path changes regardless of context.

• [test-adequacy] e2e/behaviour/features/dispatch/fork-dispatch.feature:109 — The new e2e scenario does not exercise the dispatch path this PR fixes. It checks only the custom fork-pr-killswitch-base harness, which runs through the harness-dispatch job. That job has no dependency on route and already honors config.base.yaml through the Go accessor. The changed "Check kill switch" step lives in the route job and only runs when a built-in stage is selected, which the custom label does not do. The scenario would still pass if the route job's base-file sparse-checkout and kill-switch changes were reverted, so the PR description's claim that it proves a base-only kill switch blocks GitHub dispatch is not supported.
  Remediation: Add or adapt a scenario that triggers a built-in stage, verifies the Route job halts because of the base-file kill switch, and asserts the built-in stage job was not scheduled, with a positive control. Alternatively, correct the PR description and docs so they do not claim the scenario covers the route-job fix.

Low

• [fail-open] internal/scaffold/fullsend-repo-gitlab/.gitlab/ci/scripts/run-agent-job.sh:343 — The GitLab kill-switch gate keeps its existing fail-open handling: a YAML parse error in either config.yaml or config.base.yaml logs a WARNING and sets KILL_SWITCH=false, and valid-but-unexpected values such as kill_switch: [] are coerced to false. This is not a regression (the removed code behaved the same for config.yaml, and the base file was previously never read), and issue dispatch: kill_switch check on GitHub and GitLab ignores config.base.yaml #8111 and the PR body explicitly defer it.
  Remediation: Optional follow-up: abort dispatch on parse or type-validation errors in either consulted layer and validate kill_switch against supported boolean spellings.

---

Next steps:

• /fs-fix — agent addresses review findings automatically
• /fs-fix <your instruction> — agent fixes with your specific guidance
• Push commits directly — review re-runs automatically on push
• /fs-fix-stop — disable automatic fix runs for this PR

Previous run (3)

Review

Findings

High

• [fail-open] internal/scaffold/fullsend-repo-gitlab/.gitlab/ci/scripts/run-agent-job.sh:325 — The overlay check uses key membership ('kill_switch' in overlay), so an overlay with kill_switch: null (or a bare kill_switch:) selects None, which becomes false and suppresses an active kill_switch: true in config.base.yaml. Go's IsKillSwitchActive() decodes null into a nil *bool and falls through to the base, and the GitHub route step also treats a yq null as unset, so the new GitLab path is inconsistent with both and fails open for a valid YAML shape.
  Remediation: Treat a None overlay value as unset and inherit from the base (e.g. test overlay.get('kill_switch') is not None), while keeping explicit boolean false as an override. Add subprocess test cases for kill_switch: null and kill_switch: over a true base.

• [fail-open] internal/scaffold/fullsend-repo-gitlab/.gitlab/ci/scripts/run-agent-job.sh:323 — The base YAML is parsed (line 323) before the overlay is consulted. With a valid overlay kill_switch: true and a malformed config.base.yaml, yaml.safe_load raises, the shell's failure branch resets KILL_SWITCH to false, and dispatch continues. Before this PR a valid overlay halted dispatch regardless of the base file, so this widens fail-open behavior. It is distinct from the malformed-overlay behavior that issue dispatch: kill_switch check on GitHub and GitLab ignores config.base.yaml #8111 marks out of scope.
  Remediation: Resolve the overlay first and parse the base only when the overlay is silent on kill_switch, so a base parse failure can never deactivate an explicit overlay kill switch. Add a regression case with overlay kill_switch: true and malformed base YAML.

Medium

• [fail-open] .github/workflows/reusable-dispatch.yml:515 — The route kill-switch gate compares raw yq output only with the lowercase string true. A valid YAML boolean such as kill_switch: True in config.base.yaml (or config.yaml) does not halt built-in-stage dispatch, although Go's *bool decoding would treat it as active. The pre-existing check had the same weakness for config.yaml; this PR extends it to the base file.
  Remediation: Normalize the selected value (case-insensitive match or typed yq evaluation) so that boolean true in any spelling halts dispatch, and add route-gate tests for True, quoted strings, null inheritance and explicit false overrides.

• [test-adequacy] e2e/behaviour/features/dispatch/fork-dispatch.feature:110 — The new scenario exercises custom-harness dispatch, not the built-in-stage route guard this PR changes. The harness-dispatch job has no dependency on route and already layers config.base.yaml through Go config loading, so the negative assertion passes even if the sparse-checkout and yq changes are reverted. The GitHub fix therefore has no regression coverage.
  Remediation: Add coverage that exercises the built-in route job with base true and an unset overlay, plus explicit overlay false overriding base true. Assert the route kill-switch result or built-in-stage execution, not only custom-harness artifact absence.

• [stale-doc] docs/contributing/config-source-inventory.md:100 — The kill_switch row under "File fields a workflow reads too narrowly" says the bash check reads config.yaml only, with the pending decision "Layer the direct read so a base-file true halts dispatch". This PR implements that layering, so the row is stale and in the wrong table. The PR does not touch this doc.
  Remediation: Move kill_switch to the "File fields the Go accessor already layers" table, or update the row to say the workflow and CI direct reads consult config.base.yaml when config.yaml is silent, and set Decision to Already layered.

• [missing-doc] docs/guides/dev/behaviour-testing.md:402 — The PR adds the Gherkin step the kill switch is active in config.base.yaml (pkg/behaviourtest/steps/dispatch.go). The "Dispatch step reference" documents only the kill switch is active, and line 392 describes the kill switch as kill_switch: true in config.yaml only.
  Remediation: Add a reference entry for the kill switch is active in config.base.yaml (sets kill_switch: true in .fullsend/config.base.yaml, leaving config.yaml unset) and update line 392 to say the switch can be set in either file.

• [protected-path] .github/workflows/reusable-dispatch.yml — This PR modifies a file under the protected path .github/ (.github/workflows/reusable-dispatch.yml). The PR links issue dispatch: kill_switch check on GitHub and GitLab ignores config.base.yaml #8111 and explains the rationale, but human approval is always required for protected-path changes regardless of context.

---

Next steps:

• /fs-fix — agent addresses review findings automatically
• /fs-fix <your instruction> — agent fixes with your specific guidance
• Push commits directly — review re-runs automatically on push
• /fs-fix-stop — disable automatic fix runs for this PR

[waynesun09]

Review sweep: 2 findings (1 HIGH, 1 MEDIUM) on the kill-switch layering, inline below.

[github-actions]

Site preview

Preview: https://22a2ea4f-site.fullsend-ai.workers.dev

Commit: e141b46664b8fe55bacea13043b19594f4d1530b

[waynesun09]

Review sweep: 1 MEDIUM finding on the route kill-switch check, inline below.

[mfrancisc]

@rh-hemartin thanks for the review and approval.

How do I get the PR merged ? I'm still unclear if it's automated and goes to the merge queue after approval or if the merge is manual done by maintainers only.

[waynesun09]

Review sweep: 1 HIGH finding on the route kill-switch check, inline below.

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 10:42 AM UTC · Completed 10:54 AM UTC

Commit: f3348f9 · View workflow run →

Runtime: pi · Model: sonnet → claude-sonnet-5-5 · Effort: high · Cost: $3.53

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 11:02 AM UTC · Completed 11:13 AM UTC

Commit: 1d9e8be · View workflow run →

Runtime: pi · Model: sonnet → claude-sonnet-5-5 · Effort: high · Cost: $3.38

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 9:24 AM UTC · Completed 9:35 AM UTC

Commit: e141b46 · View workflow run →

Runtime: pi · Model: sonnet → claude-sonnet-5-5 · Effort: high · Cost: $2.90

[fullsend-ai-review]

See the review comment for full details.

[waynesun09]

LGTM. Remaining review items are tracked in #8455.