[Jheyanth CS (JYunth)]

Summary

status_of_tests split the raw command, so a heredoc line that starts with a test runner counted as a test run. It now drops heredoc bodies first, as exit_status_covers already does:

 def status_of_tests(cmd):
     ...
-    pieces, pipefail = _pieces(cmd)
+    pieces, pipefail = _pieces(_split_heredocs(cmd)[0])

Before, a turn whose last real test run failed, followed by gh pr create --body-file - <<'EOF' with pytest -q in the body, was recorded as ended green. The judge was then told "The last verification run passed."

Evidence

• Before (main 264e7a7):

  status_of_tests("cat <<EOF > ci.sh\nnpm test\nEOF")              -> "mixed"
  status_of_tests("gh pr create --body-file - <<'EOF'\npytest -q\nEOF") -> "mixed"
  status_of_tests("pytest -q <<EOF\ny\nEOF")                       -> "mixed"
  fold(pytest -q -> FAILED); fold(gh pr create <<EOF ... pytest ... EOF)
    -> verifications == 2, resolved_proxy() == True
  

  All 5 new tests in tests/test_live_core.py fail.

• After:

  heredoc bodies                  -> None (no test run)
  pytest -q <<EOF / pytest after a heredoc -> "own"
  fold(...) same sequence         -> verifications == 1, resolved_proxy() == False
  

  The new tests pass, and python -m pytest tests passes: 1246 passed, 19 skipped. The only failures in experiments/tests are the five test_model_spelling ones, which call GNU timeout. That's missing on macOS, and the same tests fail on main.

Merge Danger

Door: two-way. One line, and no on-disk state or setting changes.

Blast Radius: narrow

Only commands that contain a heredoc are classified differently. The one case that moves the other way is a script piped into a shell (bash <<EOF\npytest\nEOF). Its body no longer counts as a test run, which matches how exit_status_covers already treats it.

[Jheyanth CS (JYunth)]

@microsoft-github-policy-service agree

[Muhammad Abdullah Butt (1AbdullahButt)]

I read the change against exit_status_covers and _split_heredocs, and checked _HEREDOC_START on a few shell forms in a standalone snippet. Moving the heredoc strip into _pieces (after the bash -lc unwrap) makes sense to me, and it also stops a set -o pipefail inside a heredoc body from turning pipefail on for the outer command.

One thing that now reaches status_of_tests: _HEREDOC_START (tokeneconomizer/loopguard/shell_edits.py:16, <<-?\s*(['"]?)([A-Za-z_]\w*)\1) also matches the << inside a here-string or a shift. read -r v <<< word gives the tag word, echo $((1<<n)) gives n, and echo "a<<b" gives b. Because _split_heredocs treats an unclosed body as running to the end, every later line is dropped. So with this PR:

read -r v <<< word
pytest -q

now returns None from status_of_tests (on main it returns "own"). A failing run written this way would no longer count as the turn's last verification, which is the same ended-green mix-up this PR fixes, just in the other direction. It's rare, but cheap to close:

_HEREDOC_START = re.compile(r"(?<!<)<<(?!<)-?\s*(['\"]?)([A-Za-z_]\w*)\1")

That handles the here-string, and cat<<EOF with no space still matches. A matching test next to the new ones in tests/test_live_core.py, for example status_of_tests("read -r v <<< word\npytest -q") == "own", would pin it. Quoted text (echo "a<<b") and arithmetic ($((1<<n))) could still trip it. Masking quotes before the heredoc scan would handle that, but it may be more than this PR needs.

Note: I did not run the test suite locally. I only checked the regex in isolation.

Reviewed with help from Claude Code.