Skip to content

Python: recover queued persistent shell commands - #9047

Merged
Eduard van Valkenburg (eavanvalkenburg) merged 3 commits into
microsoft:mainfrom
fhgffy:fix/queued-shell-recovery
Oct 12, 2026
Merged

Eduard van Valkenburg (eavanvalkenburg) merged 3 commits into
microsoft:mainfrom
fhgffy:fix/queued-shell-recovery

Conversation

@fhgffy

@fhgffy fhgffy commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Motivation & Context

In persistent shell mode, a command queued behind an unrecoverable timeout or output overflow sees the process before it is closed, then fails with ShellSession is not running when its turn arrives. The next command should run in a fresh shell.

Description & Review Guide

  • What are the major changes? Check/start the process after acquiring the command lock. Keep internal process cleanup separate from explicit session shutdown, and invalidate calls queued before or during explicit shutdown. Recheck after startup so closing during process creation cannot dispatch a stale command.
  • What is the impact of these changes? Queued commands recover after timeout/overflow. Explicit LocalShellTool.close() still stops pending work without restarting a process detached from the tool. A new call after shutdown can reopen the session as before. Public signatures are unchanged.
  • What do you want reviewers to focus on? The distinction between recovery and explicit close, including startup overlapping close. The regressions use real subprocesses for timeout, overflow and both shutdown queue timings; the startup test pauses real process creation at a controlled boundary.

Validation: the two recovery cases fail on the original code. After merging current main, including the PowerShell formatted-output fix in #9043, the tools unit suite passes on Windows / Python 3.14 (225 passed, 5 skipped, 3 xfailed) and Ubuntu / Python 3.12 (220 passed, 10 skipped, 3 xfailed), with 93% package coverage on each. Package Ruff check/format and strict source Pyright pass; all five configured type checkers pass for the changed test file. Wheel and sdist builds pass. No live provider/integration tests or full monorepo suite were run.

Related Issue

Fixes #9044

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR is linked to an issue and there is no other open PR for this issue (see Related Issue above).
  • This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The locking and generation checks address the reported races with focused cross-platform regression coverage.

Review effort: Balanced
Findings: None

What changed in this PR

Fixes queued persistent shell commands failing after timeout or output-overflow recovery.

Changes:

  • Restarts failed persistent shell processes after acquiring the command lock.
  • Distinguishes internal recovery from explicit shutdown.
  • Adds concurrency regressions for recovery and shutdown races.
File Description
python/​packages/​tools/​agent_framework_tools/​shell/​_session.py Adds generation-based shutdown invalidation and recovery-only cleanup.
python/​packages/​tools/​tests/​test_local_shell_tool.py Tests queued recovery and explicit-close races.

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

@fhgffy
fhgffy temporarily deployed to github-app-auth October 5, 2026 10:06 — with GitHub Actions Inactive
@fhgffy
fhgffy temporarily deployed to github-app-auth October 5, 2026 10:06 — with GitHub Actions Inactive
@fhgffy
fhgffy temporarily deployed to github-app-auth October 8, 2026 11:46 — with GitHub Actions Inactive
@agent-framework-automation agent-framework-automation Bot added the documentation Usage: [Issues, PRs], Target: documentation in the code base and learn docs label Oct 8, 2026
@fhgffy

fhgffy commented Oct 8, 2026

Copy link
Copy Markdown
Contributor Author

Pushed 7cf0361 to address the shell failure in the Windows 3.14 job, which reported a failing test despite the job's final cancelled status.

The update cancels an exited shell's readers before starting a replacement, and closes a Windows session after a timeout even when CTRL_BREAK produces a completion sentinel. This means shell-local state is reset after a Windows timeout; normal successful commands and POSIX graceful-interrupt state retention are unchanged. Commands are not replayed.

The original unmocked Windows timeout case is retained. Added coverage for forced recovery, stale readers from inherited pipes, and the sentinel-before-exit recovery decision. The lifecycle regressions fail against the previous head and pass with this change. Linux / Python 3.12: 224 passed, 10 skipped, 3 xfailed, 93% tools coverage; Ruff, source/test Pyright, wheel and sdist builds pass. The Windows decision test uses a POSIX subprocess with a platform shim, so actual Windows CI is still needed.

Separately, the Windows 3.11 MCP assertion can be reproduced by tied checkpoint timestamps selecting an older checkpoint. That test is unchanged from the base and is not modified here.

The new workflows are awaiting approval. Could a maintainer approve them so the actual Windows matrix can run?

Merged via the queue into microsoft:main with commit 2c33eba Oct 12, 2026
47 of 48 checks passed

This branch was previously deployed

1 inactive deployment
github-app-auth — 7cf03611 Deployed Oct 8, 2026 by fhgffy via add_label #24793
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Usage: [Issues, PRs], Target: documentation in the code base and learn docs python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: [Bug]: queued persistent shell commands fail after session recovery

3 participants