From e50e0cdc96e6cfef7d1662752dc272a13ced1870 Mon Sep 17 00:00:00 2001 From: Tony Narlock Date: Wed, 12 Aug 2026 17:04:35 -0500 Subject: [PATCH 01/32] Engines(feat): Make command execution pluggable why: Connection flags and process dispatch were written four times across core with four different answers, and there was no way to run a tmux command through anything but a fork of the tmux binary. what: - Add libtmux.engines: TmuxEngine protocol, CommandRequest/CommandResult, ServerConnection, and SubprocessEngine as the default - Add Server(engine=...); an engine naming no server adopts the server's connection so it cannot dispatch to the ambient tmux server - Route Server.cmd, Server.raise_if_dead, neo.fetch_objs and the control-mode client through one ServerConnection - Escape a trailing ";" in command arguments per tmux's cmd_parse grammar; CommandSeparator marks an intentional boundary - Deprecate tmux_cmd.process; keep tmux_cmd as the declared return type - Document in docs/topics/engines.md, docs/api/libtmux.engines.md, MIGRATION and CHANGES --- CHANGES | 92 ++++ MIGRATION | 82 +++ docs/api/index.md | 7 + docs/api/libtmux.engines.md | 57 +++ docs/topics/engines.md | 235 +++++++++ docs/topics/index.md | 7 + src/libtmux/_internal/control_mode.py | 27 +- src/libtmux/common.py | 159 ++++-- src/libtmux/engines/__init__.py | 71 +++ src/libtmux/engines/base.py | 477 ++++++++++++++++++ src/libtmux/engines/connection.py | 322 ++++++++++++ src/libtmux/engines/subprocess.py | 314 ++++++++++++ src/libtmux/neo.py | 17 +- src/libtmux/pane.py | 7 +- src/libtmux/server.py | 180 +++++-- .../examples/engines/test_recording_engine.py | 73 +++ tests/test_engines.py | 313 ++++++++++++ 17 files changed, 2321 insertions(+), 119 deletions(-) create mode 100644 docs/api/libtmux.engines.md create mode 100644 docs/topics/engines.md create mode 100644 src/libtmux/engines/__init__.py create mode 100644 src/libtmux/engines/base.py create mode 100644 src/libtmux/engines/connection.py create mode 100644 src/libtmux/engines/subprocess.py create mode 100644 tests/examples/engines/test_recording_engine.py create mode 100644 tests/test_engines.py diff --git a/CHANGES b/CHANGES index 7a691b0cc9..0f0bbf46ce 100644 --- a/CHANGES +++ b/CHANGES @@ -45,8 +45,100 @@ $ uvx --from 'libtmux' --prerelease allow python _Notes on the upcoming release will go here._ +### Breaking changes + +#### A bare `";"` argument is now literal data + +tmux reads a trailing `;` on a command argument as a command boundary, so a `;` +passed to {meth}`Server.cmd() ` as a *separator* worked by +relying on that parse. Arguments are now escaped, which fixes the data case +(below) and makes the separator case explicit. Pass +{class}`~libtmux.engines.base.CommandSeparator` where you mean a boundary: + +```python +# Before +server.cmd("send-keys", "-t", pane.pane_id, "-R", ";", "clear-history") + +# After +from libtmux.engines import CommandSeparator + +server.cmd( + "send-keys", "-t", pane.pane_id, "-R", + CommandSeparator(";"), + "clear-history", +) +``` + +See {ref}`migration-0-63-command-separator`. + +#### `tmux_cmd.process` deprecated + +{attr}`libtmux.common.tmux_cmd.process` is now a deprecated property. Reading it +warns, and it raises {exc}`~libtmux.exc.LibTmuxException` under an engine that +forks no process. Use `returncode`, `stdout`, and `stderr` on the result, which +are unchanged. + +#### `raise_if_dead()` no longer echoes tmux's error + +{meth}`Server.raise_if_dead() ` previously let +tmux write its message straight to the terminal. It now captures that text onto +the raised {exc}`subprocess.CalledProcessError`. The exception type is +unchanged. + +### What's new + +#### Pluggable command engines + +Every tmux command libtmux runs now goes through an *engine* — an object that +takes a rendered argv and returns a structured result. The default, +{class}`~libtmux.engines.subprocess.SubprocessEngine`, forks the tmux binary +exactly as before, so existing code is unaffected. + +Pass `engine=` to {class}`~libtmux.Server` and every command on that server runs +through your object instead. {class}`~libtmux.engines.base.TmuxEngine` is a +{class}`typing.Protocol`, so any object with `run()` and `run_batch()` qualifies +— there is no base class to inherit. That makes it possible to drive libtmux +against a recorded or in-memory tmux with no server running, and it is the seam +the control-mode, asyncio, and native-protocol engines will plug into. + +An engine that names no tmux server of its own adopts the server's connection, +so injecting one into a socket-scoped {class}`~libtmux.Server` cannot silently +dispatch to the ambient tmux server. Engines that name a server keep it. + +{class}`~libtmux.engines.connection.ServerConnection` is now the single place +the tmux binary and the `-L`/`-S`/`-f`/`-2`/`-8` flags are computed; four +separate copies previously disagreed about which flags to emit. It is derived +from the server's public attributes on each use, so reassigning `socket_name` +takes effect on the next command, and it memoizes its {func}`shutil.which` +lookup instead of re-walking `$PATH` for every command. + +See {ref}`engines` for the guide and {ref}`engines-api` for the reference. + +### Fixes + +#### A trailing `;` in a command argument is no longer swallowed + +`pane.cmd("send-keys", "echo hello;")` sent `echo hello` — tmux consumed the +final `;` as a command boundary and the character never reached the pane. +Arguments are now escaped for tmux's parser, so the `;` arrives as typed. + +#### Listing queries honor `config_file` and `colors` + +{meth}`Server.raise_if_dead() ` and the listing +queries behind {attr}`~libtmux.Server.sessions` built their own connection flags +and emitted only `-L`/`-S`, so a server constructed with `config_file=` or +`colors=` passed those flags on some commands and not others. All paths now +share one connection. A `colors=` value other than `256` or `88` raises +{exc}`~libtmux.exc.UnknownColorOption` on those paths as well. + ### Documentation +#### Engines guide and API reference + +{ref}`engines` covers what an engine is, writing one, the optional capability +protocols, and explicit command separators. {ref}`engines-api` documents the +module. + #### Cleaner `from_env` examples (#719) The rendered examples for {meth}`Pane.from_env() ` and diff --git a/MIGRATION b/MIGRATION index cbcf307f24..6ad083b023 100644 --- a/MIGRATION +++ b/MIGRATION @@ -113,6 +113,88 @@ sections below for detailed migration examples and code samples. _Detailed migration steps for the next version will be posted here._ +(migration-0-63-command-separator)= + +## Pluggable engines: separators, and `tmux_cmd.process` + +Command execution now runs through a swappable *engine* +({class}`~libtmux.engines.base.TmuxEngine`). The default, +{class}`~libtmux.engines.subprocess.SubprocessEngine`, forks the tmux binary +exactly as before, so a `Server` built the way you build it today behaves the +same. Two details do change for callers who reached past the object API. + +### A bare `";"` argument is now literal data + +tmux reads a trailing `;` on an argument as a command boundary. libtmux now +escapes it, so a `;` you pass as *data* arrives intact — the fix described in +{ref}`changelog`. The cost is that a `;` you passed as a *separator* must now +say so explicitly with {class}`~libtmux.engines.base.CommandSeparator`. + +This only affects code calling {meth}`Server.cmd() ` (or +the `Session`/`Window`/`Pane` equivalents) with a bare `";"` to fold two tmux +commands into one dispatch: + +```python +# Before +server.cmd( + "send-keys", "-t", pane.pane_id, "-R", + ";", + "clear-history", "-t", pane.pane_id, +) + +# After +from libtmux.engines import CommandSeparator + +server.cmd( + "send-keys", "-t", pane.pane_id, "-R", + CommandSeparator(";"), + "clear-history", "-t", pane.pane_id, +) +``` + +Nothing else needs changing. A `;` anywhere other than the end of an argument +was never structural, and connection flags are untouched because tmux's +`getopt` consumes them before the command parser runs. + +### `tmux_cmd.process` is deprecated + +{attr}`libtmux.common.tmux_cmd.process` exposed the {class}`subprocess.Popen` +behind a command. Only a subprocess engine has one, so it is now a deprecated +property: reading it warns, and under an engine that forks nothing it raises +{exc}`~libtmux.exc.LibTmuxException`. + +Everything the attribute was used for is on the result itself: + +```python +# Before +proc = server.cmd("list-sessions") +code = proc.process.returncode + +# After +proc = server.cmd("list-sessions") +code = proc.returncode +``` + +`cmd`, `stdout`, `stderr`, and `returncode` are unchanged. If you need the real +process object, hold your own +{class}`~libtmux.engines.subprocess.SubprocessEngine` rather than reaching +through the result. + +### Listing queries now honor `config_file` and `colors` + +{meth}`Server.raise_if_dead() ` and the listing +queries behind {attr}`~libtmux.Server.sessions` previously built their own +connection flags and emitted only `-L`/`-S`. They now share one connection with +{meth}`Server.cmd() `, so a server constructed with +`config_file=` or `colors=` passes those flags on every command instead of only +some. A `colors=` value other than `256` or `88` now raises +{exc}`~libtmux.exc.UnknownColorOption` on those paths too, where it was +previously ignored. + +`raise_if_dead()` also captures tmux's error text instead of letting it print +to the terminal. It still raises {exc}`subprocess.CalledProcessError`, and the +message is now on the exception's `stderr`. + ## libtmux 0.62.0: Query exceptions join the hierarchy (#718) {exc}`~libtmux.exc.ObjectDoesNotExist` and diff --git a/docs/api/index.md b/docs/api/index.md index 23cd9043b1..8507cab75d 100644 --- a/docs/api/index.md +++ b/docs/api/index.md @@ -96,6 +96,12 @@ Base classes and command execution. Dataclass-based query interface. ::: +:::{grid-item-card} Engine +:link: libtmux.engines +:link-type: doc +How tmux commands are executed, and how to swap that out. +::: + :::{grid-item-card} Options :link: libtmux.options :link-type: doc @@ -173,6 +179,7 @@ Window Pane Client Common +Engine Neo Options Hooks diff --git a/docs/api/libtmux.engines.md b/docs/api/libtmux.engines.md new file mode 100644 index 0000000000..f555001585 --- /dev/null +++ b/docs/api/libtmux.engines.md @@ -0,0 +1,57 @@ +(engines-api)= + +# Engines + +An *engine* is the object that actually runs a tmux command. Every dispatch in +libtmux — {meth}`Server.cmd() `, the listing queries behind +{attr}`~libtmux.Server.sessions`, and {meth}`Server.raise_if_dead() +` — goes through one, and by default that is +{class}`~libtmux.engines.subprocess.SubprocessEngine`, which forks the tmux +binary exactly as libtmux always has. + +The engine is swappable. Pass `engine=` to {class}`~libtmux.Server` and every +command on that server runs through your object instead, which is how you drive +libtmux against a recorded or in-memory tmux without a running server. + +See {ref}`engines` for the guide, with worked examples. + +Every symbol below is re-exported from `libtmux.engines`, so +`from libtmux.engines import SubprocessEngine` works regardless of which +submodule defines it. + +## Requests and results + +A {class}`~libtmux.engines.base.CommandRequest` is a rendered tmux argv; a +{class}`~libtmux.engines.base.CommandResult` is the structured outcome. A +tmux-side failure is *data* here — it sets `returncode` and `stderr` rather than +raising. Only an engine-broken condition (missing binary, lost connection) +raises. + +{class}`~libtmux.engines.base.TmuxEngine` is a {class}`typing.Protocol`, so any +object with `run()` and `run_batch()` is an engine; there is no base class to +inherit. The `Supports*` protocols are optional capabilities an engine may +also implement. + +```{eval-rst} +.. automodule:: libtmux.engines.base + :members: +``` + +## Connections + +A {class}`~libtmux.engines.connection.ServerConnection` is the pair every engine +needs before it can dispatch anything: which tmux *binary* to run, and the +connection flags (`-L`/`-S`/`-f`/`-2`/`-8`) naming one tmux server. It is the +single place either is computed. + +```{eval-rst} +.. automodule:: libtmux.engines.connection + :members: +``` + +## The default engine + +```{eval-rst} +.. automodule:: libtmux.engines.subprocess + :members: +``` diff --git a/docs/topics/engines.md b/docs/topics/engines.md new file mode 100644 index 0000000000..f1db392755 --- /dev/null +++ b/docs/topics/engines.md @@ -0,0 +1,235 @@ +(engines)= + +# Engines + +Every tmux command libtmux runs goes through an **engine**. An engine takes a +rendered argv and returns a structured result — that is its whole job. + +By default that engine is +{class}`~libtmux.engines.subprocess.SubprocessEngine`, which forks the tmux +binary once per command. You never have to know it exists. But because it is a +seam rather than hard-wired code, you can replace it — to test without tmux +running, to record what libtmux would do, or to point one `Server` at a +different tmux binary than another. + +## The default path + +Nothing changes if you ignore engines entirely: + +```python +>>> server.cmd("display-message", "-p", "#{session_name}").stdout +['libtmux_...'] +``` + +Under that call, {class}`~libtmux.Server` built a +{class}`~libtmux.engines.connection.ServerConnection` from its own +`socket_name`, `socket_path`, `config_file`, and `colors`, handed it to a +`SubprocessEngine`, and asked the engine to run the command: + +```python +>>> from libtmux.engines import SubprocessEngine +>>> server.connection.args +('-L...',) +>>> isinstance(server.engine, SubprocessEngine) +True +``` + +The connection is *derived*, not frozen at construction, so moving a server to a +different socket is picked up on the next command: + +```python +>>> from libtmux.server import Server +>>> tmux = Server(socket_name="engines_doc_a") +>>> tmux.connection.args +('-Lengines_doc_a',) +>>> tmux.socket_name = "engines_doc_b" +>>> tmux.connection.args +('-Lengines_doc_b',) +``` + +## Requests and results + +An engine speaks two value types. +{class}`~libtmux.engines.base.CommandRequest` is the argv *after* the binary and +connection flags. {class}`~libtmux.engines.base.CommandResult` is what came +back. + +```python +>>> from libtmux.engines import CommandRequest +>>> CommandRequest.from_args("kill-window", "-t", 2) +CommandRequest(args=('kill-window', '-t', '2'), tmux_bin=None) +``` + +A tmux-side failure is **data**, not an exception. An engine sets `returncode` +and `stderr`; it does not raise. Only an engine-broken condition — a missing +binary, a dropped connection — raises: + +```python +>>> from libtmux.engines import CommandResult +>>> result = CommandResult( +... cmd=("tmux", "kill-window"), +... stderr=("no such window",), +... returncode=1, +... ) +>>> result.returncode, result.stderr +(1, ('no such window',)) +``` + +## Writing an engine + +{class}`~libtmux.engines.base.TmuxEngine` is a {class}`typing.Protocol`. There +is no base class to inherit — any object with `run()` and `run_batch()` is an +engine. + +Here is a complete one that runs nothing, records everything, and answers from a +canned script. Hand it to a server and no tmux process is involved: + +```python +>>> from libtmux.engines import CommandResult +>>> from libtmux.server import Server + +>>> class RecordingEngine: +... """Record every dispatch; answer from a canned script.""" +... +... def __init__(self, stdout=()): +... self.requests = [] +... self._stdout = tuple(stdout) +... +... def run(self, request): +... self.requests.append(request.args) +... return CommandResult(cmd=("tmux", *request.args), stdout=self._stdout) +... +... def run_batch(self, requests): +... return [self.run(request) for request in requests] + +>>> recorder = RecordingEngine(stdout=("my_session",)) +>>> offline = Server(engine=recorder) +>>> offline.cmd("display-message", "-p", "#{session_name}").stdout +['my_session'] +>>> recorder.requests +[('display-message', '-p', '#{session_name}')] +``` + +This works because the socket flags live on the *engine*, not in the request, so +your `run()` only ever sees the tmux subcommand — never a `-L` to parse back +out: + +```python +>>> from libtmux.engines import CommandResult +>>> from libtmux.server import Server + +>>> class Recorder: +... def __init__(self): +... self.requests = [] +... def run(self, request): +... self.requests.append(request.args) +... return CommandResult(cmd=("tmux", *request.args)) +... def run_batch(self, requests): +... return [self.run(request) for request in requests] + +>>> recorder = Recorder() +>>> _ = Server(socket_name="engines_doc_scoped", engine=recorder).cmd("list-sessions") +>>> recorder.requests +[('list-sessions',)] +``` + +## Injected engines and sockets + +An engine that names no tmux server of its own **adopts** the server's +connection. Without that rule, injecting a bare engine into a socket-scoped +server would silently dispatch to whichever server a flagless `tmux` reaches: + +```python +>>> from libtmux.engines import SubprocessEngine +>>> from libtmux.server import Server +>>> scoped = Server(socket_name="engines_doc_c", engine=SubprocessEngine()) +>>> scoped.engine.server_args +('-Lengines_doc_c',) +``` + +An engine that *does* name a server is left exactly as you built it: + +```python +>>> from libtmux.engines import SubprocessEngine +>>> from libtmux.server import Server +>>> pinned = SubprocessEngine.of(server_args=("-Lengines_doc_pinned",)) +>>> Server(socket_name="engines_doc_c", engine=pinned).engine.server_args +('-Lengines_doc_pinned',) +``` + +An in-memory engine has no connection at all, so neither rule applies and it is +used untouched. + +## Optional capabilities + +An engine may implement extra protocols. Each is optional; libtmux checks with +{func}`isinstance` and degrades gracefully when absent. + +{class}`~libtmux.engines.base.SupportsCommandLine` renders the argv an engine +*would* run, which is how the full command line reaches the debug log before +dispatch. {class}`~libtmux.engines.base.SupportsTmuxVersion` reports the tmux +version an engine targets, for version-gated behavior. +{class}`~libtmux.engines.base.SupportsConnection` marks an engine that +dispatches over a named server and can be rebound — the protocol behind the +adoption rule above. + +```python +>>> from libtmux.engines import ( +... SubprocessEngine, +... SupportsCommandLine, +... SupportsConnection, +... ) +>>> engine = SubprocessEngine() +>>> isinstance(engine, SupportsCommandLine), isinstance(engine, SupportsConnection) +(True, True) +``` + +An engine that implements neither simply is not matched: + +```python +>>> from libtmux.engines import CommandResult, SupportsCommandLine +>>> class Bare: +... def run(self, request): +... return CommandResult(cmd=("tmux", *request.args)) +... def run_batch(self, requests): +... return [self.run(request) for request in requests] +>>> isinstance(Bare(), SupportsCommandLine) +False +``` + +## Separators are explicit + +tmux treats a trailing `;` on an argument as a command boundary. libtmux escapes +it so your data survives, which means a `;` you *intend* as a separator must say +so with {class}`~libtmux.engines.base.CommandSeparator`: + +```python +>>> from libtmux.engines import CommandSeparator, encode_direct_argv +>>> encode_direct_argv(("send-keys", "echo hi;")) +('send-keys', 'echo hi\\;') +>>> encode_direct_argv(("send-keys", CommandSeparator(";"), "clear-history")) +('send-keys', ';', 'clear-history') +``` + +Connection flags are never escaped, because tmux's `getopt` removes them before +the command parser ever sees them: + +```python +>>> from libtmux.engines import encode_direct_argv +>>> encode_direct_argv(("-Lsock;", "display-message", "text;")) +('-Lsock;', 'display-message', 'text\\;') +``` + +Used against a live pane, a separator folds two tmux commands into one dispatch: + +```python +>>> pane = session.active_window.active_pane +>>> from libtmux.engines import CommandSeparator +>>> _ = server.cmd( +... "send-keys", "-t", pane.pane_id, "-R", +... CommandSeparator(";"), +... "clear-history", "-t", pane.pane_id, +... ) +``` + +See {ref}`migration-0-63-command-separator` for migrating existing callers. diff --git a/docs/topics/index.md b/docs/topics/index.md index c955e5857e..5b4653c3d6 100644 --- a/docs/topics/index.md +++ b/docs/topics/index.md @@ -61,6 +61,12 @@ Common patterns for scripting and automation. Automatic cleanup with temporary sessions and windows. ::: +:::{grid-item-card} Engines +:link: engines +:link-type: doc +Swap how tmux commands execute: record, fake, or retarget the binary. +::: + :::{grid-item-card} Options & Hooks :link: options_and_hooks :link-type: doc @@ -97,6 +103,7 @@ workspace_setup automation_patterns context_managers options_and_hooks +engines clients format-tokens ``` diff --git a/src/libtmux/_internal/control_mode.py b/src/libtmux/_internal/control_mode.py index 05945451eb..639ca2dec5 100644 --- a/src/libtmux/_internal/control_mode.py +++ b/src/libtmux/_internal/control_mode.py @@ -62,23 +62,16 @@ def __enter__(self) -> Self: """Spawn control-mode client and wait for registration.""" read_fd, self._write_fd = os.pipe() - tmux_bin = self.server.tmux_bin or "tmux" - - if self.server.socket_name is not None: - socket_args = ["-L", str(self.server.socket_name)] - elif self.server.socket_path is not None: - socket_args = ["-S", str(self.server.socket_path)] - else: - socket_args = [] - - cmd = [ - tmux_bin, - *socket_args, - "-C", - "attach-session", - "-t", - str(self.session.session_id), - ] + # Same connection the object API dispatches on, so the control client + # attaches to the server Server.cmd() talks to. + cmd = list( + self.server.connection.argv( + "-C", + "attach-session", + "-t", + str(self.session.session_id), + ), + ) try: try: diff --git a/src/libtmux/common.py b/src/libtmux/common.py index 2871547700..d8b6a40e62 100644 --- a/src/libtmux/common.py +++ b/src/libtmux/common.py @@ -11,17 +11,21 @@ import logging import re import shlex -import shutil -import subprocess import sys import typing as t +import warnings from . import exc from ._compat import LooseVersion +from .engines.base import CommandRequest, SupportsCommandLine +from .engines.subprocess import SubprocessEngine if t.TYPE_CHECKING: + import subprocess from collections.abc import Callable + from .engines.base import TmuxEngine + logger = logging.getLogger(__name__) @@ -281,7 +285,35 @@ def raise_if_stderr(proc: tmux_cmd, subcommand: str) -> None: class tmux_cmd: - """Run any :term:`tmux(1)` command through :py:mod:`subprocess`. + """Run any :term:`tmux(1)` command, returning list-shaped output. + + Dispatches through a :class:`~libtmux.engines.base.TmuxEngine` -- + :class:`~libtmux.engines.subprocess.SubprocessEngine` unless one is passed -- + and adapts the engine's :class:`~libtmux.engines.base.CommandResult` to the + ``list``-of-``str`` attributes libtmux's wrappers read. + + Parameters + ---------- + *args : typing.Any + tmux argv. Connection flags may be included inline (``"-Lwork"``); an + engine supplies its own, so :meth:`libtmux.Server.cmd` passes only the + subcommand. + tmux_bin : str, optional + Path to the tmux binary. Ignored when *engine* is given -- the engine + owns its binary. + engine : :class:`~libtmux.engines.base.TmuxEngine`, optional + Executor to dispatch through. + + Attributes + ---------- + cmd : list[str] + The full argv that ran, tmux binary first. + stdout : list[str] + Standard output, one line per item. + stderr : list[str] + Standard error, one line per item, blanks removed. + returncode : int + tmux exit code. Examples -------- @@ -309,66 +341,53 @@ class tmux_cmd: Renamed from ``tmux`` to ``tmux_cmd``. """ - def __init__(self, *args: t.Any, tmux_bin: str | None = None) -> None: - resolved = tmux_bin or shutil.which("tmux") - if not resolved: - raise exc.TmuxCommandNotFound - - cmd = [resolved] - cmd += args # add the command arguments to cmd - cmd = [str(c) for c in cmd] - - self.cmd = cmd + def __init__( + self, + *args: t.Any, + tmux_bin: str | None = None, + engine: TmuxEngine | None = None, + ) -> None: + runner: TmuxEngine = ( + engine if engine is not None else SubprocessEngine.of(tmux_bin) + ) + request = CommandRequest.from_args(*args) if logger.isEnabledFor(logging.DEBUG): - cmd_str = shlex.join(cmd) logger.debug( "tmux command dispatched", - extra={"tmux_cmd": cmd_str}, - ) - - try: - self.process = subprocess.Popen( - cmd, - stdout=subprocess.PIPE, - stderr=subprocess.PIPE, - text=True, - encoding="utf-8", - errors="backslashreplace", - ) - stdout, stderr = self.process.communicate() - returncode = self.process.returncode - except FileNotFoundError: - raise exc.TmuxCommandNotFound from None - except Exception: - logger.error( # noqa: TRY400 - "tmux subprocess failed", extra={ - "tmux_cmd": shlex.join(cmd), + "tmux_cmd": shlex.join( + runner.command_line(request) + if isinstance(runner, SupportsCommandLine) + else request.args, + ), + "tmux_subcommand": request.subcommand, }, ) - raise - - self.returncode = returncode - - stdout_split = stdout.split("\n") - # remove trailing newlines from stdout - while stdout_split and stdout_split[-1] == "": - stdout_split.pop() - - stderr_split = stderr.split("\n") - self.stderr = list(filter(None, stderr_split)) # filter empty values - if "has-session" in cmd and len(self.stderr) and not stdout_split: - self.stdout = [self.stderr[0]] - else: - self.stdout = stdout_split + result = runner.run(request) + + self.cmd = list(result.cmd) + self.returncode = result.returncode + self.stderr = list(result.stderr) + self._process = result.process + + # tmux writes ``has-session``'s answer to stderr; the wrappers have + # always read it off stdout. Adapted here, not in an engine, so every + # engine stays a plain executor. + stdout = list(result.stdout) + self.stdout = ( + [self.stderr[0]] + if "has-session" in self.cmd and self.stderr and not stdout + else stdout + ) if logger.isEnabledFor(logging.DEBUG): logger.debug( "tmux command completed", extra={ - "tmux_cmd": shlex.join(cmd), + "tmux_cmd": shlex.join(self.cmd), + "tmux_subcommand": request.subcommand, "tmux_exit_code": self.returncode, "tmux_stdout": self.stdout[:100], "tmux_stderr": self.stderr[:100], @@ -377,6 +396,46 @@ def __init__(self, *args: t.Any, tmux_bin: str | None = None) -> None: }, ) + @property + def process(self) -> subprocess.Popen[str]: + """Return the finished :class:`subprocess.Popen`. + + Returns + ------- + subprocess.Popen + The process the default engine forked. + + Raises + ------ + :exc:`~libtmux.exc.LibTmuxException` + The engine that ran the command never forked a process. + + Examples + -------- + >>> import warnings + >>> proc = server.cmd("display-message", "-p", "hi") + >>> with warnings.catch_warnings(record=True) as caught: + ... warnings.simplefilter("always") + ... returncode = proc.process.returncode + >>> returncode + 0 + >>> caught[0].category.__name__ + 'DeprecationWarning' + + .. deprecated:: 0.63 + Read :attr:`returncode`, :attr:`stdout` and :attr:`stderr` instead. + Only engines that fork an OS process can supply this. + """ + warnings.warn( + "tmux_cmd.process is deprecated; use .returncode, .stdout, .stderr", + DeprecationWarning, + stacklevel=2, + ) + if self._process is None: + msg = "engine did not fork a subprocess; tmux_cmd.process is unavailable" + raise exc.LibTmuxException(msg) + return self._process + class _TmuxVersionUnavailable(Exception): """Internal signal: this tmux predates the ``-V`` flag (pre-1.7).""" diff --git a/src/libtmux/engines/__init__.py b/src/libtmux/engines/__init__.py new file mode 100644 index 0000000000..2b9867027e --- /dev/null +++ b/src/libtmux/engines/__init__.py @@ -0,0 +1,71 @@ +"""Engines: the seam between libtmux's object API and tmux itself. + +An *engine* answers one question -- how does a tmux command actually get run? +:class:`~libtmux.engines.subprocess.SubprocessEngine` is the default and forks the +tmux CLI, which is what libtmux has always done. Because +:class:`~libtmux.engines.base.TmuxEngine` is a +:class:`typing.Protocol`, an in-memory fake, a recorder, or a control-mode client +can take its place: + +>>> from libtmux.engines import CommandRequest, CommandResult, TmuxEngine +>>> class RecordingEngine: +... def __init__(self): +... self.seen: list[tuple[str, ...]] = [] +... +... def run(self, request): +... self.seen.append(request.args) +... return CommandResult(cmd=("tmux", *request.args), stdout=("$1",)) +... +... def run_batch(self, requests): +... return [self.run(request) for request in requests] +>>> engine = RecordingEngine() +>>> isinstance(engine, TmuxEngine) +True + +Injection happens at the :class:`~libtmux.Server` boundary: + +>>> from libtmux.server import Server +>>> Server(socket_name="engine_docs", engine=engine).cmd("list-sessions").stdout +['$1'] +>>> engine.seen +[('list-sessions',)] + +The connection flags (``-L``/``-S``/``-f``/``-2``/``-8``) are *not* part of a +request: they belong to the engine's +:class:`~libtmux.engines.connection.ServerConnection`, so every engine sees the +same request regardless of which tmux server it targets. +""" + +from __future__ import annotations + +from libtmux.engines.base import ( + CommandRequest, + CommandResult, + CommandSeparator, + DirectArgv, + SupportsCommandLine, + SupportsConnection, + SupportsTmuxVersion, + TmuxEngine, + encode_direct_argv, + is_command_separator, + split_direct_argv, +) +from libtmux.engines.connection import ServerConnection +from libtmux.engines.subprocess import SubprocessEngine + +__all__ = ( + "CommandRequest", + "CommandResult", + "CommandSeparator", + "DirectArgv", + "ServerConnection", + "SubprocessEngine", + "SupportsCommandLine", + "SupportsConnection", + "SupportsTmuxVersion", + "TmuxEngine", + "encode_direct_argv", + "is_command_separator", + "split_direct_argv", +) diff --git a/src/libtmux/engines/base.py b/src/libtmux/engines/base.py new file mode 100644 index 0000000000..c610732b42 --- /dev/null +++ b/src/libtmux/engines/base.py @@ -0,0 +1,477 @@ +"""Core engine values: requests, results, argv encoding, and the protocols. + +A :class:`CommandRequest` is a tmux argv (the subcommand and its arguments, +*without* connection flags); a :class:`CommandResult` is the structured outcome. +:class:`TmuxEngine` is a :class:`typing.Protocol`, so any object with ``run`` and +``run_batch`` is an engine -- an in-memory fake, a control-mode client, a +recorder -- without inheriting a base class. + +The argv encoders live here because they describe tmux's own parser, not any one +transport: tmux strips client-global options with ``getopt`` before handing the +remainder to ``cmd_parse_from_arguments``, where a trailing ``;`` is a command +boundary rather than data. +""" + +from __future__ import annotations + +import typing as t +from dataclasses import dataclass, field + +if t.TYPE_CHECKING: + import pathlib + import subprocess + from collections.abc import Sequence + + from typing_extensions import Self + +# tmux parses these options with getopt before its command argv reaches +# cmd_parse_from_arguments. Only values in the latter have structural +# trailing-semicolon semantics. +_GLOBAL_OPTIONS_WITH_VALUE = frozenset({"c", "f", "L", "S", "T"}) +_GLOBAL_OPTIONS_WITHOUT_VALUE = frozenset( + {"2", "8", "C", "D", "d", "h", "l", "N", "q", "u", "U", "v", "V"}, +) + + +class CommandSeparator(str): + """A caller-authored command boundary, distinct from a literal ``";"``. + + Examples + -------- + >>> CommandSeparator(";") + ';' + >>> CommandSeparator("kill-server") + Traceback (most recent call last): + ... + ValueError: a command separator must be exactly ';' + """ + + def __new__(cls, value: str) -> Self: + """Construct the one legal structural token. + + Parameters + ---------- + value : str + Must be ``";"``. + + Returns + ------- + CommandSeparator + The structural token. + + Examples + -------- + >>> str(CommandSeparator(";")) + ';' + """ + if value != ";": + msg = "a command separator must be exactly ';'" + raise ValueError(msg) + return super().__new__(cls, value) + + +def is_command_separator(token: str) -> bool: + """Return whether *token* is an intentional tmux command boundary. + + Parameters + ---------- + token : str + A single argv token. + + Returns + ------- + bool + ``True`` only for a :class:`CommandSeparator`, never for a plain + ``";"`` a caller meant as data. + + Examples + -------- + >>> is_command_separator(CommandSeparator(";")) + True + >>> is_command_separator(";") + False + """ + return type(token) is CommandSeparator and token == ";" + + +class DirectArgv(t.NamedTuple): + """The client-global and command portions of direct tmux argv. + + Attributes + ---------- + global_args : tuple[str, ...] + Leading options consumed by tmux's client-level ``getopt`` parser. + command_argv : tuple[str, ...] + The subcommand and arguments passed to ``cmd_parse_from_arguments``. + """ + + global_args: tuple[str, ...] + command_argv: tuple[str, ...] + + +def _global_option_consumes_next(token: str) -> bool | None: + """Return a global option's separate-value arity, or ``None`` if unknown. + + Examples + -------- + >>> _global_option_consumes_next("-L") + True + >>> _global_option_consumes_next("-Lwork") + False + >>> _global_option_consumes_next("list-sessions") is None + True + """ + if not token.startswith("-") or token in {"-", "--"}: + return None + cluster = token[1:] + if not cluster: + return None + for index, option in enumerate(cluster): + if option in _GLOBAL_OPTIONS_WITH_VALUE: + return index == len(cluster) - 1 + if option not in _GLOBAL_OPTIONS_WITHOUT_VALUE: + return None + return False + + +def split_direct_argv(argv: Sequence[str]) -> DirectArgv: + """Split raw tmux argv at the client-global/command parser boundary. + + The split follows tmux's leading short-option ``getopt`` grammar, including + attached values and ``--``. Global values remain byte-for-byte data because + tmux removes them before parsing command separators. + + Parameters + ---------- + argv : Sequence[str] + tmux argv after the binary. + + Returns + ------- + DirectArgv + The global and command halves. + + Raises + ------ + ValueError + A token contains a NUL byte, which no tmux transport can carry. + + Examples + -------- + >>> split_direct_argv(("-L", "socket;", "display-message", "text;")) + DirectArgv(global_args=('-L', 'socket;'), command_argv=('display-message', 'text;')) + >>> split_direct_argv(("-Lsocket;", "--", "display-message")) + DirectArgv(global_args=('-Lsocket;', '--'), command_argv=('display-message',)) + """ + args = tuple(argv) + if any("\0" in token for token in args): + msg = "tmux command arguments cannot contain NUL" + raise ValueError(msg) + + index = 0 + while index < len(args): + token = args[index] + if token == "--": + index += 1 + break + consumes_next = _global_option_consumes_next(token) + if consumes_next is None: + break + index += 2 if consumes_next and index + 1 < len(args) else 1 + return DirectArgv(global_args=args[:index], command_argv=args[index:]) + + +def _encode_command_argv(argv: Sequence[str]) -> tuple[str, ...]: + r"""Escape literal separators in argv already known to be command-scoped. + + Examples + -------- + >>> _encode_command_argv(("display-message", "literal;")) + ('display-message', 'literal\\;') + """ + return tuple( + f"{token[:-1]}\\;" + if not is_command_separator(token) and token.endswith(";") + else str(token) + for token in argv + ) + + +def encode_direct_argv(argv: Sequence[str]) -> tuple[str, ...]: + r"""Encode literal arguments for tmux's direct argv parser. + + tmux first removes client-global options, then routes only the remaining + command argv through ``cmd_parse_from_arguments``, where a final ``;`` is + structural. Prefixing that final byte with one backslash preserves it as + data. Global option values are left alone, and a :class:`CommandSeparator` + stays structural. + + Parameters + ---------- + argv : Sequence[str] + tmux argv after the binary. + + Returns + ------- + tuple[str, ...] + argv safe to hand to ``execve``. + + Examples + -------- + >>> encode_direct_argv(("send-keys", "text;")) + ('send-keys', 'text\\;') + >>> encode_direct_argv(("-L", "socket;", "send-keys", "text;")) + ('-L', 'socket;', 'send-keys', 'text\\;') + >>> encode_direct_argv(("a", CommandSeparator(";"), "b")) + ('a', ';', 'b') + """ + direct = split_direct_argv(argv) + return (*direct.global_args, *_encode_command_argv(direct.command_argv)) + + +@dataclass(frozen=True) +class CommandRequest: + """A tmux command, ready for an engine to execute. + + Carries the subcommand and its arguments only. Connection flags + (``-L``/``-S``/``-f``/``-2``/``-8``) belong to the engine's + :class:`~libtmux.engines.connection.ServerConnection`, so every engine sees + the same request no matter which tmux server it targets. + + Attributes + ---------- + args : tuple[str, ...] + The tmux argv (e.g. ``("split-window", "-t", "%1")``). + tmux_bin : str or None + Override the tmux binary for this one request; ``None`` lets the engine + decide. + + Examples + -------- + >>> CommandRequest.from_args("split-window", "-t", "%1") + CommandRequest(args=('split-window', '-t', '%1'), tmux_bin=None) + >>> CommandRequest.from_args("kill-window", "-t", 2).args + ('kill-window', '-t', '2') + """ + + args: tuple[str, ...] + tmux_bin: str | None = None + + def __post_init__(self) -> None: + r"""Reject arguments that cannot survive tmux's C-string transports. + + Examples + -------- + >>> CommandRequest(args=("display-message", "a\0b")) + Traceback (most recent call last): + ... + ValueError: tmux command arguments cannot contain NUL + """ + normalized = tuple( + arg if is_command_separator(arg) else str.__str__(arg) for arg in self.args + ) + if any("\0" in arg for arg in normalized): + msg = "tmux command arguments cannot contain NUL" + raise ValueError(msg) + object.__setattr__(self, "args", normalized) + + @classmethod + def from_args( + cls, + *args: t.Any, + tmux_bin: str | pathlib.Path | None = None, + ) -> CommandRequest: + """Build a request from arbitrary tokens, stringifying each. + + Parameters + ---------- + *args : typing.Any + Tokens; non-strings are rendered with :func:`str`, matching what + :class:`~libtmux.common.tmux_cmd` has always accepted. + tmux_bin : str or pathlib.Path, optional + Per-request tmux binary override. + + Returns + ------- + CommandRequest + The request. + + Examples + -------- + >>> CommandRequest.from_args("resize-pane", "-t", "%3", "-x", 80).args + ('resize-pane', '-t', '%3', '-x', '80') + """ + return cls( + args=tuple(arg if isinstance(arg, str) else str(arg) for arg in args), + tmux_bin=str(tmux_bin) if tmux_bin is not None else None, + ) + + @property + def subcommand(self) -> str: + """Return the tmux subcommand, or ``""`` for an empty request. + + Returns + ------- + str + First argv token. + + Examples + -------- + >>> CommandRequest.from_args("list-sessions", "-F#S").subcommand + 'list-sessions' + >>> CommandRequest.from_args().subcommand + '' + """ + return self.args[0] if self.args else "" + + +@dataclass(frozen=True) +class CommandResult: + """The structured outcome of executing a :class:`CommandRequest`. + + A tmux-side failure (nonzero exit, message on stderr) is *data* here: it + sets ``returncode`` and ``stderr`` rather than raising. Only a broken engine + (missing binary, lost connection) raises. + + Attributes + ---------- + cmd : tuple[str, ...] + The full argv that ran, including the tmux binary and connection flags. + stdout : tuple[str, ...] + Captured standard-output lines, trailing blanks removed. + stderr : tuple[str, ...] + Captured standard-error lines, blanks removed. + returncode : int + tmux exit code. + process : subprocess.Popen or None + The OS process, when the engine forked one. ``None`` for engines that + never touch the operating system, which is why + :attr:`libtmux.common.tmux_cmd.process` can only be a best-effort + accessor. Excluded from equality and :func:`repr`. + + Examples + -------- + >>> CommandResult(cmd=("tmux", "display-message", "-p", "hi"), stdout=("hi",)) + CommandResult(cmd=('tmux', 'display-message', '-p', 'hi'), stdout=('hi',), + stderr=(), returncode=0) + """ + + cmd: tuple[str, ...] + stdout: tuple[str, ...] = () + stderr: tuple[str, ...] = () + returncode: int = 0 + process: subprocess.Popen[str] | None = field( + default=None, + compare=False, + repr=False, + ) + + +@t.runtime_checkable +class TmuxEngine(t.Protocol): + """A synchronous executor of tmux commands. + + Structural: an object is an engine when it has ``run`` and ``run_batch``. + + Examples + -------- + >>> from libtmux.engines import CommandRequest, CommandResult, TmuxEngine + >>> class EchoEngine: + ... def run(self, request): + ... return CommandResult(cmd=("tmux", *request.args), stdout=("ok",)) + ... + ... def run_batch(self, requests): + ... return [self.run(request) for request in requests] + >>> isinstance(EchoEngine(), TmuxEngine) + True + >>> EchoEngine().run(CommandRequest.from_args("list-sessions")).stdout + ('ok',) + """ + + def run(self, request: CommandRequest) -> CommandResult: + """Execute one tmux command and return its structured result.""" + ... + + def run_batch(self, requests: Sequence[CommandRequest]) -> list[CommandResult]: + """Execute requests in order, returning one result per request. + + Persistent-connection engines override this to pipeline; stateless + engines implement it as a loop over :meth:`run`. + """ + ... + + +@t.runtime_checkable +class SupportsCommandLine(t.Protocol): + """An engine that can render the argv it *would* run, without running it. + + Optional capability. :class:`~libtmux.common.tmux_cmd` uses it to log the + full command line before dispatch; engines without a command line (in-memory + fakes) simply do not implement it. + + Examples + -------- + >>> from libtmux.engines import SupportsCommandLine, SubprocessEngine + >>> isinstance(SubprocessEngine.for_server(server), SupportsCommandLine) + True + """ + + def command_line(self, request: CommandRequest) -> tuple[str, ...]: + """Return the full argv, binary first, that *request* would run as.""" + ... + + +@t.runtime_checkable +class SupportsConnection(t.Protocol): + """An engine that dispatches over a named tmux server and can be rebound. + + Optional capability. :attr:`Server.engine ` reads it + so an injected engine that names no server of its own adopts the server's + connection instead of silently reaching the ambient tmux server. In-memory + engines have no connection and simply do not implement it. + + Examples + -------- + >>> from libtmux.engines import SubprocessEngine, SupportsConnection + >>> isinstance(SubprocessEngine(), SupportsConnection) + True + + An engine with no notion of a socket does not implement it: + + >>> class InMemoryEngine: + ... def run(self, request): + ... return CommandResult(cmd=("tmux", *request.args)) + ... def run_batch(self, requests): + ... return [self.run(r) for r in requests] + >>> isinstance(InMemoryEngine(), SupportsConnection) + False + """ + + @property + def connection(self) -> t.Any: + """Return the tmux binary and flags this engine dispatches over.""" + ... + + def with_connection(self, connection: t.Any) -> TmuxEngine: + """Return an equivalent engine bound to *connection*.""" + ... + + +@t.runtime_checkable +class SupportsTmuxVersion(t.Protocol): + """An engine that can report the tmux version it targets. + + Optional capability, for version-gated rendering. Engines that cannot know + their version -- in-memory fakes -- do not implement it, and callers fall + back to "assume latest". + + Examples + -------- + >>> from libtmux.engines import SubprocessEngine, SupportsTmuxVersion + >>> isinstance(SubprocessEngine.for_server(server), SupportsTmuxVersion) + True + """ + + def tmux_version(self) -> str | None: + """Return the engine's tmux version string, or ``None`` if unknown.""" + ... diff --git a/src/libtmux/engines/connection.py b/src/libtmux/engines/connection.py new file mode 100644 index 0000000000..6b9d395a1e --- /dev/null +++ b/src/libtmux/engines/connection.py @@ -0,0 +1,322 @@ +"""The connection an engine talks to: which tmux binary, which tmux server. + +Every engine needs the same two things before it can dispatch anything: a tmux +*binary* to exec, and the *connection flags* (``-L``/``-S``/``-f``/``-2``/``-8``) +that point at one particular tmux server. :class:`ServerConnection` is that pair +as one frozen value, and it is the only place in libtmux where either is +computed -- :meth:`libtmux.Server.cmd`, :meth:`libtmux.Server.raise_if_dead` and +:func:`libtmux.neo.fetch_objs` all read their flags from here. + +:meth:`ServerConnection.resolve_bin` is the single door to a tmux binary path: it +memoizes :func:`shutil.which` and raises +:exc:`~libtmux.exc.TmuxCommandNotFound` when tmux is absent, so no engine ships +an unguarded ``shutil.which("tmux")`` of its own. +""" + +from __future__ import annotations + +import shutil +import typing as t +from dataclasses import dataclass, field + +from libtmux import exc + +if t.TYPE_CHECKING: + import pathlib + from collections.abc import Sequence + + +class _BinaryResolver: + """Memoized tmux-binary resolution and ``tmux -V`` probe. + + Owned by a :class:`ServerConnection`; never constructed by engines. Holding + the mutable cache here keeps :class:`ServerConnection` a frozen, comparable + value. + """ + + __slots__ = ("_declared", "_resolved", "_version", "_version_probed") + + def __init__(self, tmux_bin: str | None = None) -> None: + self._declared = tmux_bin + self._resolved: str | None = None + self._version: str | None = None + self._version_probed = False + + def resolve(self) -> str: + """Return the tmux binary path, memoized for this connection. + + An explicit binary wins. Otherwise :func:`shutil.which` walks ``$PATH`` + once and the answer is cached. A *failure* is not cached, so a tmux + installed after the miss is picked up. + """ + if self._declared is not None: + return self._declared + if self._resolved is None: + resolved = shutil.which("tmux") + if resolved is None: + raise exc.TmuxCommandNotFound + self._resolved = resolved + return self._resolved + + def version(self) -> str | None: + """Return the tmux version string, memoized; ``None`` when unknowable. + + ``None`` (missing binary, unparseable output) lets version resolution + degrade to "assume latest" rather than exploding. + """ + if not self._version_probed: + self._version_probed = True + # Imported here, not at module scope: libtmux.common's tmux_cmd + # dispatches through this package, so a module-level import would + # close an import cycle. + from libtmux.common import get_version + + try: + self._version = str(get_version(self.resolve())) + except exc.LibTmuxException: + self._version = None + return self._version + + +@dataclass(frozen=True) +class ServerConnection: + """Which tmux binary, and which tmux server, an engine talks to. + + Attributes + ---------- + tmux_bin : str or None + An explicit tmux binary. ``None`` means "resolve from ``$PATH``", which + :meth:`resolve_bin` does once and memoizes. + args : tuple[str, ...] + Connection flags placed before the tmux subcommand (e.g. ``("-Lwork",)``). + _resolver : _BinaryResolver + Memoized resolver for the binary path and tmux version. Built in + ``__post_init__``; excluded from equality, hashing and :func:`repr`. + + Examples + -------- + The default connection targets the ambient tmux server: + + >>> ServerConnection() + ServerConnection(tmux_bin=None, args=()) + + :meth:`from_server` reads the flags off a live :class:`libtmux.Server`: + + >>> conn = ServerConnection.from_server(server) + >>> conn.args[0].startswith(("-L", "-S")) + True + + It duck-types, so any object with the same attributes works: + + >>> import types + >>> ServerConnection.from_server( + ... types.SimpleNamespace(socket_name="work", colors=256) + ... ) + ServerConnection(tmux_bin=None, args=('-2', '-Lwork')) + + :meth:`argv` prepends the binary and the flags to a command: + + >>> ServerConnection.of(tmux_bin="tmux", args=("-Lwork",)).argv( + ... "kill-window", "-t", "@1" + ... ) + ('tmux', '-Lwork', 'kill-window', '-t', '@1') + """ + + tmux_bin: str | None = None + args: tuple[str, ...] = () + _resolver: _BinaryResolver = field( + init=False, + repr=False, + compare=False, + hash=False, + ) + + def __post_init__(self) -> None: + """Normalize *args* and build the connection's binary resolver. + + Examples + -------- + >>> ServerConnection(args=["-Lwork"]).args + ('-Lwork',) + """ + object.__setattr__(self, "args", tuple(self.args)) + object.__setattr__(self, "_resolver", _BinaryResolver(self.tmux_bin)) + + @classmethod + def of( + cls, + tmux_bin: str | pathlib.Path | None = None, + args: Sequence[str] = (), + ) -> ServerConnection: + """Build a connection, stringifying a :class:`pathlib.Path` binary. + + Parameters + ---------- + tmux_bin : str or pathlib.Path, optional + Explicit tmux binary. + args : Sequence[str] + Connection flags. + + Returns + ------- + ServerConnection + The connection. + + Examples + -------- + >>> import pathlib + >>> ServerConnection.of(pathlib.Path("/usr/bin/tmux")).tmux_bin + '/usr/bin/tmux' + >>> ServerConnection.of(args=["-L", "test"]).args + ('-L', 'test') + """ + return cls( + tmux_bin=str(tmux_bin) if tmux_bin is not None else None, + args=tuple(args), + ) + + @classmethod + def from_server(cls, server: t.Any) -> ServerConnection: + """Build the connection a live :class:`libtmux.Server` talks over. + + Flags are emitted in tmux's documented order of significance and in the + order :meth:`libtmux.Server.cmd` has always emitted them: color depth, + ``-f`` config file, ``-S`` socket path, ``-L`` socket name. + + Parameters + ---------- + server : typing.Any + Any object exposing ``socket_name``, ``socket_path``, + ``config_file``, ``colors`` and ``tmux_bin``. Missing attributes are + treated as unset. + + Returns + ------- + ServerConnection + The connection. + + Raises + ------ + :exc:`~libtmux.exc.UnknownColorOption` + ``colors`` is truthy but is neither ``256`` nor ``88``. + + Examples + -------- + >>> import types + >>> ServerConnection.from_server( + ... types.SimpleNamespace(socket_path="/tmp/s", config_file="/tmp/c") + ... ) + ServerConnection(tmux_bin=None, args=('-f/tmp/c', '-S/tmp/s')) + + >>> from libtmux import exc + >>> try: + ... ServerConnection.from_server(types.SimpleNamespace(colors=16)) + ... except exc.UnknownColorOption as e: + ... print(e) + Server.colors must equal 88 or 256 + """ + args: list[str] = [] + + colors = getattr(server, "colors", None) + if colors: + if colors == 256: + args.append("-2") + elif colors == 88: + args.append("-8") + else: + raise exc.UnknownColorOption + + if getattr(server, "config_file", None): + args.append(f"-f{server.config_file}") + if getattr(server, "socket_path", None): + args.append(f"-S{server.socket_path}") + if getattr(server, "socket_name", None): + args.append(f"-L{server.socket_name}") + + return cls.of(tmux_bin=getattr(server, "tmux_bin", None), args=args) + + @property + def is_unconfigured(self) -> bool: + """Whether this connection names no server and no binary of its own. + + An unconfigured connection targets whichever tmux server ``tmux`` would + reach with no flags. :attr:`Server.engine ` reads + this to decide whether an injected engine should adopt the server's + connection: an engine that already names a server is left alone, and one + that names none is bound, so it cannot silently dispatch to the ambient + server. + + Returns + ------- + bool + + Examples + -------- + >>> ServerConnection().is_unconfigured + True + >>> ServerConnection.of(args=("-Lwork",)).is_unconfigured + False + >>> ServerConnection.of(tmux_bin="/usr/bin/tmux").is_unconfigured + False + """ + return not self.args and self.tmux_bin is None + + def resolve_bin(self) -> str: + """Return the tmux binary path (memoized). + + Returns + ------- + str + Path to tmux. + + Raises + ------ + :exc:`~libtmux.exc.TmuxCommandNotFound` + tmux is not on ``$PATH`` and none was declared. + + Examples + -------- + >>> ServerConnection.of(tmux_bin="/usr/bin/tmux").resolve_bin() + '/usr/bin/tmux' + """ + return self._resolver.resolve() + + def tmux_version(self) -> str | None: + """Return this connection's tmux version (memoized), or ``None``. + + Returns + ------- + str or None + Version string, e.g. ``"3.5"``; ``None`` when tmux is missing or + its version cannot be parsed. + + Examples + -------- + >>> ServerConnection().tmux_version() is not None + True + """ + return self._resolver.version() + + def argv(self, *args: str, tmux_bin: str | None = None) -> tuple[str, ...]: + """Render a full command line: binary, connection flags, then *args*. + + Parameters + ---------- + *args : str + The tmux subcommand and its arguments. + tmux_bin : str, optional + Override this connection's binary for one command. + + Returns + ------- + tuple[str, ...] + The full argv. + + Examples + -------- + >>> ServerConnection.of("tmux", ("-Lwork",)).argv("list-sessions") + ('tmux', '-Lwork', 'list-sessions') + >>> ServerConnection.of("tmux").argv("list-sessions", tmux_bin="/opt/tmux") + ('/opt/tmux', 'list-sessions') + """ + return (tmux_bin or self.resolve_bin(), *self.args, *args) diff --git a/src/libtmux/engines/subprocess.py b/src/libtmux/engines/subprocess.py new file mode 100644 index 0000000000..2c3bf792b5 --- /dev/null +++ b/src/libtmux/engines/subprocess.py @@ -0,0 +1,314 @@ +"""The default engine: one ``fork``/``exec`` of the tmux CLI per command. + +Mirrors the output handling libtmux has always had -- ``backslashreplace`` +decoding, trailing-blank stripping on stdout, blank filtering on stderr. A +tmux-side failure comes back as data (nonzero ``returncode`` plus ``stderr``); +only a missing binary raises. +""" + +from __future__ import annotations + +import logging +import shlex +import subprocess +import typing as t + +from libtmux import exc +from libtmux.engines.base import CommandResult, encode_direct_argv +from libtmux.engines.connection import ServerConnection + +if t.TYPE_CHECKING: + import pathlib + from collections.abc import Sequence + + from libtmux.engines.base import CommandRequest + +logger = logging.getLogger(__name__) + + +class SubprocessEngine: + """Execute tmux commands by forking the tmux CLI binary. + + Parameters + ---------- + connection : ServerConnection, optional + The tmux binary and connection flags to dispatch through. Defaults to + the ambient tmux server on ``$PATH``. + + Examples + -------- + >>> from libtmux.engines import CommandRequest, SubprocessEngine + >>> engine = SubprocessEngine.for_server(server) + >>> engine.run(CommandRequest.from_args("display-message", "-p", "hi")).stdout + ('hi',) + """ + + def __init__(self, connection: ServerConnection | None = None) -> None: + self._conn = connection if connection is not None else ServerConnection() + + @classmethod + def of( + cls, + tmux_bin: str | pathlib.Path | None = None, + server_args: Sequence[str] = (), + ) -> SubprocessEngine: + """Build an engine from a binary path and raw connection flags. + + Parameters + ---------- + tmux_bin : str or pathlib.Path, optional + Explicit tmux binary; resolved from ``$PATH`` when ``None``. + server_args : Sequence[str] + Connection flags, e.g. ``("-Lwork",)``. + + Returns + ------- + SubprocessEngine + The engine. + + Examples + -------- + >>> SubprocessEngine.of(server_args=["-Lwork"]).server_args + ('-Lwork',) + """ + return cls(ServerConnection.of(tmux_bin, server_args)) + + def with_connection(self, connection: ServerConnection) -> SubprocessEngine: + """Return an equivalent engine dispatching over *connection*. + + Engines are immutable with respect to their connection, so this returns + a new engine rather than rebinding this one. + :attr:`Server.engine ` calls it to bind an engine + that names no server of its own. + + Parameters + ---------- + connection : ServerConnection + The connection the returned engine dispatches over. + + Returns + ------- + SubprocessEngine + A new engine; this one is left untouched. + + Examples + -------- + >>> from libtmux.engines import ServerConnection + >>> engine = SubprocessEngine() + >>> engine.server_args + () + >>> engine.with_connection(ServerConnection.of(args=("-Lwork",))).server_args + ('-Lwork',) + >>> engine.server_args + () + """ + return type(self)(connection) + + @classmethod + def for_server(cls, server: t.Any) -> SubprocessEngine: + """Build an engine bound to a live :class:`libtmux.Server`'s socket. + + Parameters + ---------- + server : typing.Any + Any object shaped like a :class:`libtmux.Server`. + + Returns + ------- + SubprocessEngine + An engine reaching the same tmux server as the object API. + + Examples + -------- + >>> SubprocessEngine.for_server(server).server_args[0].startswith("-L") + True + """ + return cls(ServerConnection.from_server(server)) + + @property + def connection(self) -> ServerConnection: + """The tmux binary + connection flags this engine dispatches through. + + Returns + ------- + ServerConnection + The connection. + + Examples + -------- + >>> SubprocessEngine.of("tmux").connection.tmux_bin + 'tmux' + """ + return self._conn + + @property + def tmux_bin(self) -> str | None: + """The explicitly configured tmux binary, if any. + + Returns + ------- + str or None + The declared binary; ``None`` when resolved from ``$PATH``. + + Examples + -------- + >>> SubprocessEngine.of("/usr/bin/tmux").tmux_bin + '/usr/bin/tmux' + """ + return self._conn.tmux_bin + + @property + def server_args(self) -> tuple[str, ...]: + """Connection flags placed before every tmux subcommand. + + Returns + ------- + tuple[str, ...] + The flags. + + Examples + -------- + >>> SubprocessEngine.of(server_args=("-Ltest",)).server_args + ('-Ltest',) + """ + return self._conn.args + + def tmux_version(self) -> str | None: + """Report this engine's tmux version (``tmux -V``), memoized. + + Returns + ------- + str or None + ``None`` when the binary is missing or its version cannot be + parsed, so version resolution degrades to "assume latest". + + Examples + -------- + >>> SubprocessEngine.for_server(server).tmux_version() is not None + True + """ + return self._conn.tmux_version() + + def command_line(self, request: CommandRequest) -> tuple[str, ...]: + r"""Return the full argv *request* would run as, without running it. + + Parameters + ---------- + request : CommandRequest + The command. + + Returns + ------- + tuple[str, ...] + Binary, connection flags, then the encoded command argv. + + Examples + -------- + >>> from libtmux.engines import CommandRequest + >>> SubprocessEngine.of("tmux", ("-Lwork",)).command_line( + ... CommandRequest.from_args("send-keys", "echo hi;") + ... ) + ('tmux', '-Lwork', 'send-keys', 'echo hi\\;') + """ + return self._conn.argv( + *encode_direct_argv(request.args), + tmux_bin=request.tmux_bin, + ) + + def run(self, request: CommandRequest) -> CommandResult: + """Execute one tmux command via :mod:`subprocess` and return its result. + + Parameters + ---------- + request : CommandRequest + The command. + + Returns + ------- + CommandResult + Structured output, carrying the :class:`subprocess.Popen` that ran. + + Raises + ------ + :exc:`~libtmux.exc.TmuxCommandNotFound` + The tmux binary is missing or not executable. + + Examples + -------- + >>> from libtmux.engines import CommandRequest + >>> engine = SubprocessEngine.for_server(server) + >>> engine.run(CommandRequest.from_args("has-session", "-t", "nope")).returncode + 1 + """ + cmd = self.command_line(request) + + try: + process = subprocess.Popen( + cmd, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + text=True, + encoding="utf-8", + errors="backslashreplace", + ) + stdout, stderr = process.communicate() + returncode = process.returncode + except FileNotFoundError: + raise exc.TmuxCommandNotFound from None + except Exception: + logger.error( # noqa: TRY400 + "tmux subprocess failed", + extra={"tmux_cmd": shlex.join(cmd)}, + ) + raise + + stdout_lines = stdout.split("\n") + while stdout_lines and stdout_lines[-1] == "": + stdout_lines.pop() + + result = CommandResult( + cmd=cmd, + stdout=tuple(stdout_lines), + stderr=tuple(line for line in stderr.split("\n") if line), + returncode=returncode, + process=process, + ) + if logger.isEnabledFor(logging.DEBUG): + logger.debug( + "tmux subprocess completed", + extra={ + "tmux_cmd": shlex.join(cmd), + "tmux_subcommand": request.subcommand, + "tmux_exit_code": returncode, + "tmux_stdout_len": len(result.stdout), + "tmux_stderr_len": len(result.stderr), + }, + ) + return result + + def run_batch(self, requests: Sequence[CommandRequest]) -> list[CommandResult]: + """Execute each request in order, one fork per command. + + Parameters + ---------- + requests : Sequence[CommandRequest] + Commands to run. + + Returns + ------- + list[CommandResult] + One result per request, in order. + + Examples + -------- + >>> from libtmux.engines import CommandRequest + >>> results = SubprocessEngine.for_server(server).run_batch( + ... [ + ... CommandRequest.from_args("display-message", "-p", "one"), + ... CommandRequest.from_args("display-message", "-p", "two"), + ... ] + ... ) + >>> [result.stdout[0] for result in results] + ['one', 'two'] + """ + return [self.run(request) for request in requests] diff --git a/src/libtmux/neo.py b/src/libtmux/neo.py index 98ece86fa5..feb8bb0e47 100644 --- a/src/libtmux/neo.py +++ b/src/libtmux/neo.py @@ -1098,17 +1098,7 @@ def fetch_objs( tmux_version = str(get_version(tmux_bin=server.tmux_bin)) _fields, format_string = get_output_format(list_cmd, tmux_version) - cmd_args: list[str | int] = [] - - if server.socket_name: - cmd_args.insert(0, f"-L{server.socket_name}") - if server.socket_path: - cmd_args.insert(0, f"-S{server.socket_path}") - - tmux_cmds = [ - *cmd_args, - list_cmd, - ] + tmux_cmds: list[str | int] = [list_cmd] if list_extra_args is not None and isinstance(list_extra_args, Iterable): tmux_cmds.extend(list(list_extra_args)) @@ -1130,10 +1120,7 @@ def fetch_objs( }, ) - proc = tmux_cmd( - *tmux_cmds, - tmux_bin=server.tmux_bin, - ) + proc = tmux_cmd(*tmux_cmds, engine=server.engine) raise_if_stderr(proc, list_cmd) diff --git a/src/libtmux/pane.py b/src/libtmux/pane.py index e0c2f59619..69215a9563 100644 --- a/src/libtmux/pane.py +++ b/src/libtmux/pane.py @@ -23,6 +23,7 @@ PaneDirection, ResizeAdjustmentDirection, ) +from libtmux.engines.base import CommandSeparator from libtmux.formats import FORMAT_SEPARATOR from libtmux.hooks import HooksMixin from libtmux.neo import Obj, fetch_obj @@ -2613,7 +2614,9 @@ def reset(self) -> Pane: Sends ``send-keys -R`` and ``clear-history`` to the pane in one targeted tmux command sequence so output cannot land in the freshly-cleared grid between the terminal-state reset and the - history clear. + history clear. The boundary between the two is a + :class:`~libtmux.engines.base.CommandSeparator`, which marks it + structural: a plain ``";"`` argument is data, and reaches tmux escaped. Examples -------- @@ -2625,7 +2628,7 @@ def reset(self) -> Pane: "-t", self.pane_id, "-R", - ";", + CommandSeparator(";"), "clear-history", "-t", self.pane_id, diff --git a/src/libtmux/server.py b/src/libtmux/server.py index e650557c34..1d66023dda 100644 --- a/src/libtmux/server.py +++ b/src/libtmux/server.py @@ -10,7 +10,6 @@ import logging import os import pathlib -import shutil import subprocess import typing as t import warnings @@ -21,6 +20,9 @@ from libtmux.client import Client from libtmux.common import get_version, has_gte_version, raise_if_stderr, tmux_cmd from libtmux.constants import OptionScope +from libtmux.engines.base import CommandRequest, SupportsConnection +from libtmux.engines.connection import ServerConnection +from libtmux.engines.subprocess import SubprocessEngine from libtmux.hooks import HooksMixin from libtmux.neo import fetch_objs, get_output_format, parse_output from libtmux.pane import Pane @@ -43,6 +45,7 @@ from typing_extensions import Self from libtmux._internal.types import StrPath + from libtmux.engines.base import TmuxEngine DashLiteral: TypeAlias = t.Literal["-"] @@ -108,6 +111,10 @@ class Server( on_init : callable, optional socket_name_factory : callable, optional tmux_bin : str or pathlib.Path, optional + engine : :class:`~libtmux.engines.base.TmuxEngine`, optional + Executor every tmux command runs through. Defaults to + :class:`~libtmux.engines.subprocess.SubprocessEngine` bound to this + server's :attr:`connection`. Examples -------- @@ -167,6 +174,17 @@ class Server( tmux_bin: str | None = None """Custom path to tmux binary. Falls back to ``shutil.which("tmux")``.""" + _engine: TmuxEngine | None = None + """Caller-supplied executor, or ``None`` for the default subprocess engine.""" + _default_engine: SubprocessEngine | None = None + """Lazily built default engine, rebuilt whenever :attr:`connection` changes.""" + _connection: ServerConnection | None = None + """Cached connection, valid while :attr:`_connection_key` still matches.""" + _connection_key: tuple[t.Any, ...] | None = None + """Snapshot of the public connection attributes the cache was built from.""" + _adopted_engine: tuple[ServerConnection, TmuxEngine] | None = None + """Injected engine rebound to :attr:`connection`, with the connection it used.""" + def __init__( self, socket_name: str | None = None, @@ -176,10 +194,16 @@ def __init__( on_init: t.Callable[[Server], None] | None = None, socket_name_factory: t.Callable[[], str] | None = None, tmux_bin: str | pathlib.Path | None = None, + engine: TmuxEngine | None = None, **kwargs: t.Any, ) -> None: EnvironmentMixin.__init__(self, "-g") self.tmux_bin = str(tmux_bin) if tmux_bin is not None else None + self._engine = engine + self._default_engine = None + self._adopted_engine = None + self._connection = None + self._connection_key = None self._windows: list[WindowDict] = [] self._panes: list[PaneDict] = [] @@ -199,6 +223,115 @@ def __init__( if on_init is not None: on_init(self) + @property + def connection(self) -> ServerConnection: + """Return the tmux binary and connection flags this server dispatches on. + + :attr:`socket_name`, :attr:`socket_path`, :attr:`config_file`, + :attr:`colors` and :attr:`tmux_bin` are public and writable, and + :meth:`__eq__` reads two of them, so a connection captured once at + construction would silently keep pointing at the old socket after a + write. The connection is therefore *derived*, and cached against a + snapshot of exactly those five attributes: reassigning any of them + invalidates the cache on the next command, while an unchanged server + keeps one memoized :func:`shutil.which` lookup for its whole life. + + Returns + ------- + :class:`~libtmux.engines.connection.ServerConnection` + Flags in the order tmux receives them: color depth, ``-f``, ``-S``, + ``-L``. + + Raises + ------ + :exc:`~libtmux.exc.UnknownColorOption` + :attr:`colors` is set to something other than ``256`` or ``88``. + + Examples + -------- + >>> tmux = Server(socket_name="engine_conn_docs") + >>> tmux.connection.args + ('-Lengine_conn_docs',) + + A later write is picked up: + + >>> tmux.socket_name = "engine_conn_docs_moved" + >>> tmux.connection.args + ('-Lengine_conn_docs_moved',) + + .. versionadded:: 0.63 + """ + key = ( + self.socket_name, + None if self.socket_path is None else str(self.socket_path), + self.config_file, + self.colors, + self.tmux_bin, + ) + if self._connection is None or self._connection_key != key: + self._connection = ServerConnection.from_server(self) + self._connection_key = key + return self._connection + + @property + def engine(self) -> TmuxEngine: + """Return the executor every tmux command on this server runs through. + + With no ``engine=``, a + :class:`~libtmux.engines.subprocess.SubprocessEngine` is built from + :attr:`connection` and rebuilt whenever that connection changes. + + A caller-supplied ``engine=`` that already names a tmux server is + returned untouched. One that names none -- a bare + ``SubprocessEngine()`` -- *adopts* this server's :attr:`connection`, + because returning it untouched would dispatch to whichever server a + flagless ``tmux`` reaches rather than to this one. Engines with no + connection at all, such as in-memory fakes, are always returned + untouched. + + Returns + ------- + :class:`~libtmux.engines.base.TmuxEngine` + The engine. + + Examples + -------- + >>> from libtmux.engines import SubprocessEngine + >>> isinstance(server.engine, SubprocessEngine) + True + + An injected engine that names no server adopts this one's socket: + + >>> tmux = Server(socket_name="engine_adopt_docs", engine=SubprocessEngine()) + >>> tmux.engine.server_args + ('-Lengine_adopt_docs',) + + An engine that names a server keeps it: + + >>> pinned = SubprocessEngine.of(server_args=("-Lelsewhere",)) + >>> Server(socket_name="engine_adopt_docs", engine=pinned).engine.server_args + ('-Lelsewhere',) + + .. versionadded:: 0.63 + """ + connection = self.connection + engine = self._engine + if engine is not None: + if not isinstance(engine, SupportsConnection): + return engine + if connection.is_unconfigured or not engine.connection.is_unconfigured: + return engine + adopted = self._adopted_engine + if adopted is None or adopted[0] is not connection: + adopted = (connection, engine.with_connection(connection)) + self._adopted_engine = adopted + return adopted[1] + default = self._default_engine + if default is None or default.connection is not connection: + default = SubprocessEngine(connection) + self._default_engine = default + return default + @classmethod def from_env(cls, env: t.Mapping[str, str] | None = None) -> Server: """Return the tmux server this process's pane is attached to. @@ -317,22 +450,9 @@ def raise_if_dead(self) -> None: ... print(type(e)) """ - resolved = self.tmux_bin or shutil.which("tmux") - if resolved is None: - raise exc.TmuxCommandNotFound - - cmd_args: list[str] = ["list-sessions"] - if self.socket_name: - cmd_args.insert(0, f"-L{self.socket_name}") - if self.socket_path: - cmd_args.insert(0, f"-S{self.socket_path}") - if self.config_file: - cmd_args.insert(0, f"-f{self.config_file}") - - try: - subprocess.check_call([resolved, *cmd_args]) - except FileNotFoundError: - raise exc.TmuxCommandNotFound from None + result = self.engine.run(CommandRequest.from_args("list-sessions")) + if result.returncode != 0: + raise subprocess.CalledProcessError(result.returncode, list(result.cmd)) # # Command @@ -386,29 +506,19 @@ def cmd( Notes ----- + Dispatches through :attr:`Server.engine`; the connection flags come + from :attr:`Server.connection`, so this method and every other tmux + call on this server target the same socket. + .. versionchanged:: 0.8 Renamed from ``.tmux`` to ``.cmd``. """ - svr_args: list[str | int] = [cmd] - cmd_args: list[str | int] = [] - if self.socket_name: - svr_args.insert(0, f"-L{self.socket_name}") - if self.socket_path: - svr_args.insert(0, f"-S{self.socket_path}") - if self.config_file: - svr_args.insert(0, f"-f{self.config_file}") - if self.colors: - if self.colors == 256: - svr_args.insert(0, "-2") - elif self.colors == 88: - svr_args.insert(0, "-8") - else: - raise exc.UnknownColorOption - - cmd_args = ["-t", str(target), *args] if target is not None else [*args] + cmd_args: list[str | int] = ( + ["-t", str(target), *args] if target is not None else [*args] + ) - return tmux_cmd(*svr_args, *cmd_args, tmux_bin=self.tmux_bin) + return tmux_cmd(cmd, *cmd_args, engine=self.engine) @property def attached_sessions(self) -> list[Session]: diff --git a/tests/examples/engines/test_recording_engine.py b/tests/examples/engines/test_recording_engine.py new file mode 100644 index 0000000000..61ae16ea3b --- /dev/null +++ b/tests/examples/engines/test_recording_engine.py @@ -0,0 +1,73 @@ +"""Drive libtmux with no tmux server, using a custom engine. + +The engine seam's payoff is that :class:`~libtmux.Server` will dispatch through +any object satisfying :class:`~libtmux.engines.base.TmuxEngine`. That makes it +possible to assert on the tmux commands a piece of code *would* run, and to +answer them from a script, without a tmux server anywhere. +""" + +from __future__ import annotations + +import typing as t + +from libtmux.engines import CommandResult +from libtmux.server import Server + +if t.TYPE_CHECKING: + from collections.abc import Sequence + + from libtmux.engines import CommandRequest + + +class RecordingEngine: + """Record every dispatch and answer from a canned script. + + Attributes + ---------- + requests : list[tuple[str, ...]] + The argv of every request, in dispatch order. + """ + + def __init__(self, stdout: Sequence[str] = ()) -> None: + self.requests: list[tuple[str, ...]] = [] + self._stdout = tuple(stdout) + + def run(self, request: CommandRequest) -> CommandResult: + """Record *request* and return the canned result.""" + self.requests.append(request.args) + return CommandResult(cmd=("tmux", *request.args), stdout=self._stdout) + + def run_batch(self, requests: Sequence[CommandRequest]) -> list[CommandResult]: + """Run each request in order.""" + return [self.run(request) for request in requests] + + +def test_engine_records_dispatch_without_tmux() -> None: + """A custom engine sees the tmux argv and supplies the answer.""" + engine = RecordingEngine(stdout=("my_session",)) + server = Server(engine=engine) + + result = server.cmd("display-message", "-p", "#{session_name}") + + assert result.stdout == ["my_session"] + assert engine.requests == [("display-message", "-p", "#{session_name}")] + + +def test_engine_sees_no_connection_flags() -> None: + """Connection flags live on the engine, so a request carries only the command. + + An engine implementing its own transport never has to parse ``-L``/``-S`` + back out of the argv it is handed. + """ + engine = RecordingEngine() + Server(socket_name="example_recording", engine=engine).cmd("list-sessions") + + assert engine.requests == [("list-sessions",)] + + +def test_target_is_rendered_into_the_request() -> None: + """``target=`` reaches the engine as the ``-t`` flag tmux expects.""" + engine = RecordingEngine() + Server(engine=engine).cmd("kill-window", target="@3") + + assert engine.requests == [("kill-window", "-t", "@3")] diff --git a/tests/test_engines.py b/tests/test_engines.py new file mode 100644 index 0000000000..656dd08ade --- /dev/null +++ b/tests/test_engines.py @@ -0,0 +1,313 @@ +"""Tests for :mod:`libtmux.engines`, the tmux command execution seam.""" + +from __future__ import annotations + +import subprocess +import typing as t +import warnings + +import pytest + +from libtmux import exc +from libtmux.common import tmux_cmd +from libtmux.engines import ( + CommandRequest, + CommandResult, + CommandSeparator, + ServerConnection, + SubprocessEngine, + SupportsCommandLine, + SupportsTmuxVersion, + TmuxEngine, + encode_direct_argv, + split_direct_argv, +) +from libtmux.neo import fetch_objs +from libtmux.server import Server +from libtmux.test.retry import retry_until + +if t.TYPE_CHECKING: + from collections.abc import Sequence + + from libtmux.session import Session + + +class CannedEngine: + """An in-memory engine: records requests, replays canned stdout. + + Satisfies :class:`~libtmux.engines.base.TmuxEngine` structurally, without + inheritance and without a tmux binary. + """ + + def __init__(self, stdout: Sequence[str] = ()) -> None: + self.requests: list[CommandRequest] = [] + self._stdout = tuple(stdout) + + def run(self, request: CommandRequest) -> CommandResult: + """Record *request* and return the canned result.""" + self.requests.append(request) + return CommandResult( + cmd=("canned-tmux", *request.args), + stdout=self._stdout, + ) + + def run_batch(self, requests: Sequence[CommandRequest]) -> list[CommandResult]: + """Run each request in order.""" + return [self.run(request) for request in requests] + + +def test_canned_engine_satisfies_protocol() -> None: + """A plain class with run/run_batch is a TmuxEngine.""" + assert isinstance(CannedEngine(), TmuxEngine) + assert not isinstance(CannedEngine(), SupportsCommandLine) + assert not isinstance(CannedEngine(), SupportsTmuxVersion) + + +def test_server_drives_injected_engine_without_tmux() -> None: + """``Server(engine=...)`` routes ``cmd()`` through the injected engine. + + No tmux fixture: the point is that an injected engine never forks tmux, so + the canned stdout is what ``Server.cmd`` returns. + """ + engine = CannedEngine(stdout=("$9",)) + server = Server(socket_name="canned_never_started", engine=engine) + + proc = server.cmd("new-session", "-P", "-F#{session_id}") + + assert proc.stdout == ["$9"] + assert proc.returncode == 0 + assert proc.cmd == ["canned-tmux", "new-session", "-P", "-F#{session_id}"] + assert [request.args for request in engine.requests] == [ + ("new-session", "-P", "-F#{session_id}"), + ] + assert server.engine is engine + + +def test_injected_engine_receives_target_flag() -> None: + """``target=`` is rendered into the request, not the connection.""" + engine = CannedEngine() + server = Server(socket_name="canned_target", engine=engine) + + server.cmd("kill-window", target="@3") + + assert engine.requests[0].args == ("kill-window", "-t", "@3") + + +def test_process_raises_on_engine_without_subprocess() -> None: + """``.process`` is unavailable when no OS process was forked.""" + server = Server(socket_name="canned_process", engine=CannedEngine()) + proc = server.cmd("list-sessions") + + with warnings.catch_warnings(record=True) as caught: + warnings.simplefilter("always") + with pytest.raises(exc.LibTmuxException): + _ = proc.process + + assert any(issubclass(entry.category, DeprecationWarning) for entry in caught) + + +def test_process_is_popen_under_default_engine(session: Session) -> None: + """``.process`` still resolves to the Popen, with a DeprecationWarning.""" + proc = session.server.cmd("display-message", "-p", "hi") + + with pytest.deprecated_call(): + process = proc.process + + assert isinstance(process, subprocess.Popen) + assert process.returncode == 0 + + +def test_connection_follows_socket_name_mutation() -> None: + """A post-construction write to ``socket_name`` changes the flags used. + + ``Server.socket_name`` is public and writable, so the connection is derived + per command rather than captured at construction. + """ + server = Server(socket_name="mutation_before") + assert server.connection.args == ("-Lmutation_before",) + first = server.connection + + server.socket_name = "mutation_after" + + assert server.connection.args == ("-Lmutation_after",) + assert server.connection is not first + assert server.cmd("has-session", "-t", "nothing").cmd[1] == "-Lmutation_after" + + +def test_connection_is_cached_while_unchanged(server: Server) -> None: + """An untouched server reuses one connection, and so one binary lookup.""" + assert server.connection is server.connection + assert server.engine is server.engine + + +def test_default_engine_rebuilt_after_mutation() -> None: + """The default engine is rebuilt when the connection it wraps changes.""" + server = Server(socket_name="engine_rebuild_before") + first = server.engine + + server.socket_name = "engine_rebuild_after" + second = server.engine + + assert first is not second + assert isinstance(second, SubprocessEngine) + assert second.server_args == ("-Lengine_rebuild_after",) + + +def test_injected_engine_survives_mutation() -> None: + """An injected engine is user-owned: libtmux never swaps it out.""" + engine = CannedEngine() + server = Server(socket_name="injected_before", engine=engine) + + server.socket_name = "injected_after" + + assert server.engine is engine + + +class ArgvRecordingEngine: + """Render argv against a real connection, record it, run nothing. + + Lets a test read the command line each dispatch path *would* have used, + without a tmux server and without special-casing any one path. + """ + + def __init__(self, connection: ServerConnection) -> None: + self.connection = connection + self.command_lines: list[tuple[str, ...]] = [] + + def run(self, request: CommandRequest) -> CommandResult: + """Record the rendered argv and return an empty success.""" + cmd = (self.connection.tmux_bin or "tmux", *self.connection.args, *request.args) + self.command_lines.append(cmd) + return CommandResult(cmd=cmd) + + def run_batch(self, requests: Sequence[CommandRequest]) -> list[CommandResult]: + """Run each request in order.""" + return [self.run(request) for request in requests] + + +def test_flag_builders_agree() -> None: + """cmd(), raise_if_dead() and fetch_objs() emit identical flags. + + All three paths formerly built ``-L``/``-S``/``-f``/``-2`` themselves, from + three different rules. They now read one + :class:`~libtmux.engines.connection.ServerConnection`. + """ + attrs: dict[str, t.Any] = { + "socket_name": "flag_agreement", + "config_file": "/dev/null", + "colors": 256, + } + expected = Server(**attrs).connection.args + assert expected == ("-2", "-f/dev/null", "-Lflag_agreement") + + engine = ArgvRecordingEngine(Server(**attrs).connection) + server = Server(**attrs, engine=engine) + + server.cmd("list-sessions") + server.raise_if_dead() + fetch_objs(server=server, list_cmd="list-sessions") + + assert len(engine.command_lines) == 3 + assert {line[1 : 1 + len(expected)] for line in engine.command_lines} == {expected} + + +def test_unknown_color_raises_on_every_path() -> None: + """An unknown ``colors`` value raises, matching ``Server.cmd``'s contract.""" + server = Server(socket_name="bad_colors") + server.colors = 16 + + with pytest.raises(exc.UnknownColorOption): + server.cmd("list-sessions") + with pytest.raises(exc.UnknownColorOption): + server.raise_if_dead() + with pytest.raises(exc.UnknownColorOption): + fetch_objs(server=server, list_cmd="list-sessions") + + +def test_trailing_semicolon_is_literal(session: Session) -> None: + """A trailing ``";"`` reaches tmux as data, not as a command boundary. + + Unescaped, tmux's argv parser reads the final ``;`` as a separator and the + pane never sees it. + """ + window = session.new_window(window_name="semicolon") + pane = window.active_pane + assert pane is not None + + pane.send_keys("echo one;", literal=True, enter=False) + + def typed() -> bool: + return any(line.endswith("echo one;") for line in pane.capture_pane()) + + assert retry_until(typed, raises=False), pane.capture_pane() + + +def test_command_separator_stays_structural(session: Session) -> None: + """An explicit :class:`CommandSeparator` still separates two commands.""" + server = session.server + proc = server.cmd( + "display-message", + "-p", + "first", + CommandSeparator(";"), + "display-message", + "-p", + "second", + ) + + assert proc.stdout == ["first", "second"] + + +def test_encode_direct_argv_leaves_global_values_alone() -> None: + """Connection-flag values are data to tmux's getopt, never separators.""" + assert encode_direct_argv(("-L", "sock;", "send-keys", "text;")) == ( + "-L", + "sock;", + "send-keys", + "text\\;", + ) + assert split_direct_argv(("-2", "-f/tmp/c", "list-sessions")).command_argv == ( + "list-sessions", + ) + + +def test_command_request_rejects_nul() -> None: + """NUL cannot survive tmux's C-string argv.""" + with pytest.raises(ValueError, match="NUL"): + CommandRequest.from_args("display-message", "a\0b") + + +def test_connection_from_server_duck_types() -> None: + """``from_server`` reads any object with the five connection attributes.""" + conn = ServerConnection.from_server( + Server(socket_path="/tmp/spike-sock", config_file="/tmp/spike-conf"), + ) + assert conn.args == ("-f/tmp/spike-conf", "-S/tmp/spike-sock") + + +def test_subprocess_engine_reports_version(session: Session) -> None: + """The default engine can answer ``tmux -V`` for version gating.""" + engine = SubprocessEngine.for_server(session.server) + assert isinstance(engine, SupportsTmuxVersion) + assert engine.tmux_version() == engine.tmux_version() + assert engine.tmux_version() is not None + + +def test_missing_binary_raises_tmux_command_not_found() -> None: + """A declared-but-absent tmux binary raises, on every path.""" + engine = SubprocessEngine.of("/nonexistent/tmux") + with pytest.raises(exc.TmuxCommandNotFound): + engine.run(CommandRequest.from_args("list-sessions")) + with pytest.raises(exc.TmuxCommandNotFound): + tmux_cmd("list-sessions", tmux_bin="/nonexistent/tmux") + + +def test_run_batch_preserves_order(session: Session) -> None: + """``run_batch`` returns one result per request, in order.""" + results = SubprocessEngine.for_server(session.server).run_batch( + [ + CommandRequest.from_args("display-message", "-p", "a"), + CommandRequest.from_args("display-message", "-p", "b"), + ], + ) + assert [result.stdout[0] for result in results] == ["a", "b"] From 2589fe9e114c3ee285f16485f0e4830cf8932904 Mon Sep 17 00:00:00 2001 From: Tony Narlock Date: Wed, 12 Aug 2026 17:16:08 -0500 Subject: [PATCH 02/32] Engines(feat): Record and replay tmux traffic, and result ergonomics MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit why: The seam shipped a protocol but nothing to plug into it. Simulating tmux is not viable — listing queries ask for 136 version-gated format fields per row, and a fake that answers unknown commands optimistically reports that every session exists while none do. what: - Add RecordingEngine and ReplayEngine: record real tmux answers, serve them back with no server running; tapes round-trip through JSON - Fail closed on an unrecorded command with exc.UnscriptedCommand - Add the recording_server pytest fixture, usable as a spy - Add CommandResult.ok/raise_for_status(), mirrored on tmux_cmd - Give TmuxEngine.run_batch a default body so subclasses implement run - Declare AsyncTmuxEngine for persistent-connection engines - Validate engine= at construction, naming a missing method and rejecting an async engine that satisfies the structural check - Document in docs/topics/engines.md, the API page and CHANGES --- CHANGES | 32 +++++ docs/api/libtmux.engines.md | 7 + docs/topics/engines.md | 56 ++++++++ src/libtmux/common.py | 53 +++++++- src/libtmux/engines/__init__.py | 6 + src/libtmux/engines/base.py | 153 ++++++++++++++++++++- src/libtmux/engines/record.py | 233 ++++++++++++++++++++++++++++++++ src/libtmux/exc.py | 28 ++++ src/libtmux/pytest_plugin.py | 49 +++++++ src/libtmux/server.py | 45 +++++- 10 files changed, 652 insertions(+), 10 deletions(-) create mode 100644 src/libtmux/engines/record.py diff --git a/CHANGES b/CHANGES index 0f0bbf46ce..afc8f156d6 100644 --- a/CHANGES +++ b/CHANGES @@ -114,6 +114,38 @@ lookup instead of re-walking `$PATH` for every command. See {ref}`engines` for the guide and {ref}`engines-api` for the reference. +#### Testing without a tmux server + +{class}`~libtmux.engines.record.RecordingEngine` wraps a real engine and keeps +what tmux answered; {class}`~libtmux.engines.record.ReplayEngine` serves those +answers back with no tmux running, so the whole object API — `sessions`, +`windows`, `panes` — works offline against real recorded output. Tapes +round-trip through JSON, so recording needs tmux and running does not. A replay +engine fails closed, raising {exc}`~libtmux.exc.UnscriptedCommand` for a command +it never recorded. The `recording_server` pytest fixture provides a server that +records, which doubles as a spy over the tmux commands your code issued. + +#### Result objects report success directly + +{attr}`CommandResult.ok ` and +{meth}`CommandResult.raise_for_status() +` replace hand-written +`returncode` comparisons, and {class}`~libtmux.common.tmux_cmd` carries the same +two, so results read the same whichever layer produced them. + +#### Engines are validated where they are supplied + +{class}`~libtmux.Server` now rejects a non-engine at construction, naming the +missing method, rather than failing with an {exc}`AttributeError` inside the +first command. An async engine is rejected explicitly: it satisfies the +structural check, since a runtime-checkable {class}`typing.Protocol` compares +method names rather than signatures. + +Subclassing {class}`~libtmux.engines.base.TmuxEngine` supplies `run_batch`, so a +stateless engine only implements `run`. +{class}`~libtmux.engines.base.AsyncTmuxEngine` is declared for engines that will +hold a persistent connection. + ### Fixes #### A trailing `;` in a command argument is no longer swallowed diff --git a/docs/api/libtmux.engines.md b/docs/api/libtmux.engines.md index f555001585..f6a22d2ea6 100644 --- a/docs/api/libtmux.engines.md +++ b/docs/api/libtmux.engines.md @@ -55,3 +55,10 @@ single place either is computed. .. automodule:: libtmux.engines.subprocess :members: ``` + +## Recording and replay + +```{eval-rst} +.. automodule:: libtmux.engines.record + :members: +``` diff --git a/docs/topics/engines.md b/docs/topics/engines.md index f1db392755..493011d130 100644 --- a/docs/topics/engines.md +++ b/docs/topics/engines.md @@ -133,6 +133,62 @@ out: [('list-sessions',)] ``` +## Testing without tmux + +Writing a fake that *simulates* tmux is a trap. libtmux's listing queries ask +tmux for 136 format fields per row, and a fake that answers unknown commands +optimistically ends up reporting that every session exists (`has-session` exits +0) while no sessions exist (`list-sessions` is empty). + +So record real traffic instead, and play it back. +{class}`~libtmux.engines.record.RecordingEngine` wraps a real engine and keeps +what tmux said; {class}`~libtmux.engines.record.ReplayEngine` serves it back: + +```python +>>> from libtmux.engines import RecordingEngine, ReplayEngine, SubprocessEngine +>>> from libtmux.server import Server + +>>> recorder = RecordingEngine(SubprocessEngine.for_server(server)) +>>> live = Server(socket_name=server.socket_name, engine=recorder) +>>> _ = live.cmd("display-message", "-p", "#{session_name}") + +>>> offline = Server(engine=ReplayEngine(recorder.tape)) +>>> offline.cmd("display-message", "-p", "#{session_name}").stdout +['libtmux_...'] +``` + +Because the rows came from real tmux, the whole object API works offline — +`sessions`, `windows`, `panes` all hydrate. `to_dict()` and +{meth}`~libtmux.engines.record.ReplayEngine.from_dict` round-trip a tape through +JSON, so you can commit one next to the tests that replay it: recording needs +tmux, running does not. + +A replay engine **fails closed**. A command it never recorded raises +{exc}`~libtmux.exc.UnscriptedCommand` rather than inventing an answer: + +```python +>>> from libtmux.engines import CommandResult, ReplayEngine +>>> from libtmux.server import Server +>>> engine = ReplayEngine({("list-sessions",): CommandResult(cmd=("tmux",))}) +>>> Server(engine=engine).cmd("kill-server") +Traceback (most recent call last): +... +libtmux.exc.UnscriptedCommand: no recorded result for 'kill-server' +``` + +A recorder also works as a plain spy — `requests` is every argv in order, which +is what the `recording_server` pytest fixture is for: + +```python +>>> from libtmux.engines import RecordingEngine, SubprocessEngine +>>> from libtmux.server import Server +>>> recorder = RecordingEngine(SubprocessEngine.for_server(server)) +>>> spy = Server(socket_name=server.socket_name, engine=recorder) +>>> _ = spy.cmd("list-sessions") +>>> recorder.requests +[('list-sessions',)] +``` + ## Injected engines and sockets An engine that names no tmux server of its own **adopts** the server's diff --git a/src/libtmux/common.py b/src/libtmux/common.py index d8b6a40e62..648b5233b4 100644 --- a/src/libtmux/common.py +++ b/src/libtmux/common.py @@ -17,7 +17,7 @@ from . import exc from ._compat import LooseVersion -from .engines.base import CommandRequest, SupportsCommandLine +from .engines.base import CommandRequest, SupportsCommandLine, split_direct_argv from .engines.subprocess import SubprocessEngine if t.TYPE_CHECKING: @@ -396,6 +396,57 @@ def __init__( }, ) + @property + def ok(self) -> bool: + """Whether tmux accepted the command. + + The same accessor :attr:`CommandResult.ok + ` carries, so code reads the same + whether it holds an engine result or a wrapper's return value. + + Returns + ------- + bool + ``True`` when :attr:`returncode` is zero. + + Examples + -------- + >>> server.cmd("display-message", "-p", "hi").ok + True + """ + return self.returncode == 0 + + def raise_for_status(self) -> tmux_cmd: + """Raise when tmux rejected the command, otherwise return self. + + Returns + ------- + tmux_cmd + This object, when :attr:`ok`. + + Raises + ------ + :exc:`~libtmux.exc.LibTmuxException` + tmux exited non-zero. The message carries tmux's own stderr. + + Examples + -------- + >>> server.cmd("display-message", "-p", "hi").raise_for_status().stdout + ['hi'] + + >>> server.cmd("kill-window", "-t", "@999").raise_for_status() + Traceback (most recent call last): + ... + libtmux.exc.LibTmuxException: kill-window: can't find window: @999 + """ + if self.ok: + return self + detail = " ".join(self.stderr) or f"exited {self.returncode}" + command_argv = split_direct_argv(tuple(self.cmd[1:])).command_argv + subcommand = command_argv[0] if command_argv else "tmux" + msg = f"{subcommand}: {detail}" + raise exc.LibTmuxException(msg) + @property def process(self) -> subprocess.Popen[str]: """Return the finished :class:`subprocess.Popen`. diff --git a/src/libtmux/engines/__init__.py b/src/libtmux/engines/__init__.py index 2b9867027e..ba424f9b4d 100644 --- a/src/libtmux/engines/__init__.py +++ b/src/libtmux/engines/__init__.py @@ -39,6 +39,7 @@ from __future__ import annotations from libtmux.engines.base import ( + AsyncTmuxEngine, CommandRequest, CommandResult, CommandSeparator, @@ -52,18 +53,23 @@ split_direct_argv, ) from libtmux.engines.connection import ServerConnection +from libtmux.engines.record import RecordingEngine, ReplayEngine, Tape from libtmux.engines.subprocess import SubprocessEngine __all__ = ( + "AsyncTmuxEngine", "CommandRequest", "CommandResult", "CommandSeparator", "DirectArgv", + "RecordingEngine", + "ReplayEngine", "ServerConnection", "SubprocessEngine", "SupportsCommandLine", "SupportsConnection", "SupportsTmuxVersion", + "Tape", "TmuxEngine", "encode_direct_argv", "is_command_separator", diff --git a/src/libtmux/engines/base.py b/src/libtmux/engines/base.py index c610732b42..ccb35d6266 100644 --- a/src/libtmux/engines/base.py +++ b/src/libtmux/engines/base.py @@ -366,26 +366,105 @@ class CommandResult: repr=False, ) + @property + def ok(self) -> bool: + """Whether tmux accepted the command. + + Returns + ------- + bool + ``True`` when :attr:`returncode` is zero. + + Examples + -------- + >>> CommandResult(cmd=("tmux", "list-sessions")).ok + True + >>> CommandResult(cmd=("tmux", "kill-window"), returncode=1).ok + False + """ + return self.returncode == 0 + + def raise_for_status(self) -> CommandResult: + """Raise when tmux rejected the command, otherwise return self. + + Engines report a tmux-side failure as data so a caller can inspect it. + This turns that data back into an exception at the point a caller would + rather not continue, and returns ``self`` so it chains. + + Returns + ------- + CommandResult + This result, when :attr:`ok`. + + Raises + ------ + :exc:`~libtmux.exc.LibTmuxException` + tmux exited non-zero. The message carries tmux's own stderr. + + Examples + -------- + >>> result = CommandResult(cmd=("tmux", "list-sessions"), stdout=("a",)) + >>> result.raise_for_status().stdout + ('a',) + + The message names the tmux subcommand, not a connection flag: + + >>> CommandResult( + ... cmd=("tmux", "-Lmysocket", "kill-window", "-t", "@9"), + ... stderr=("can't find window @9",), + ... returncode=1, + ... ).raise_for_status() + Traceback (most recent call last): + ... + libtmux.exc.LibTmuxException: kill-window: can't find window @9 + """ + if self.ok: + return self + from libtmux import exc + + detail = " ".join(self.stderr) or f"exited {self.returncode}" + # cmd is the full argv: binary, then client-global flags, then the + # subcommand. Skip the flags the way tmux's own getopt does, so the + # message names "kill-window" rather than "-Lmysocket". + command_argv = split_direct_argv(self.cmd[1:]).command_argv + subcommand = command_argv[0] if command_argv else "tmux" + msg = f"{subcommand}: {detail}" + raise exc.LibTmuxException(msg) + @t.runtime_checkable class TmuxEngine(t.Protocol): """A synchronous executor of tmux commands. Structural: an object is an engine when it has ``run`` and ``run_batch``. + Writing both is only necessary when you do *not* inherit — subclassing + :class:`TmuxEngine` supplies :meth:`run_batch`, so a stateless engine needs + just :meth:`run`. Examples -------- + Inheriting is the short way: + >>> from libtmux.engines import CommandRequest, CommandResult, TmuxEngine - >>> class EchoEngine: + >>> class EchoEngine(TmuxEngine): ... def run(self, request): ... return CommandResult(cmd=("tmux", *request.args), stdout=("ok",)) + >>> EchoEngine().run(CommandRequest.from_args("list-sessions")).stdout + ('ok',) + >>> EchoEngine().run_batch([CommandRequest.from_args("list-sessions")]) + [CommandResult(cmd=('tmux', 'list-sessions'), stdout=('ok',), stderr=(), + returncode=0)] + + Duck typing works too, but then both methods are yours to write: + + >>> class Structural: + ... def run(self, request): + ... return CommandResult(cmd=("tmux", *request.args)) ... ... def run_batch(self, requests): ... return [self.run(request) for request in requests] - >>> isinstance(EchoEngine(), TmuxEngine) + >>> isinstance(Structural(), TmuxEngine) True - >>> EchoEngine().run(CommandRequest.from_args("list-sessions")).stdout - ('ok',) """ def run(self, request: CommandRequest) -> CommandResult: @@ -395,11 +474,73 @@ def run(self, request: CommandRequest) -> CommandResult: def run_batch(self, requests: Sequence[CommandRequest]) -> list[CommandResult]: """Execute requests in order, returning one result per request. - Persistent-connection engines override this to pipeline; stateless - engines implement it as a loop over :meth:`run`. + Defaults to a loop over :meth:`run`, which is correct for any stateless + engine. Persistent-connection engines override it to pipeline. + + Parameters + ---------- + requests : Sequence[CommandRequest] + Requests to run, in order. + + Returns + ------- + list[CommandResult] + One result per request. """ + return [self.run(request) for request in requests] + + +@t.runtime_checkable +class AsyncTmuxEngine(t.Protocol): + """An asynchronous executor of tmux commands. + + The async sibling of :class:`TmuxEngine`, declared here so an async engine + has a type to satisfy from the day it is written rather than after the fact. + {class}`~libtmux.Server` is synchronous and does **not** accept one; it is + for callers driving tmux on an event loop directly, and for the engines that + will hold a persistent ``tmux -C`` connection. + + Examples + -------- + >>> import asyncio + >>> from libtmux.engines import AsyncTmuxEngine, CommandRequest, CommandResult + >>> class AsyncEcho(AsyncTmuxEngine): + ... async def run(self, request): + ... return CommandResult(cmd=("tmux", *request.args), stdout=("ok",)) + >>> async def main(): + ... engine = AsyncEcho() + ... result = await engine.run(CommandRequest.from_args("list-sessions")) + ... batch = await engine.run_batch([CommandRequest.from_args("list-panes")]) + ... return result.stdout, len(batch) + >>> asyncio.run(main()) + (('ok',), 1) + """ + + async def run(self, request: CommandRequest) -> CommandResult: + """Execute one tmux command and return its structured result.""" ... + async def run_batch( + self, + requests: Sequence[CommandRequest], + ) -> list[CommandResult]: + """Execute requests in order, returning one result per request. + + Defaults to an awaited loop over :meth:`run`. A persistent-connection + engine overrides it to pipeline without waiting for each reply. + + Parameters + ---------- + requests : Sequence[CommandRequest] + Requests to run, in order. + + Returns + ------- + list[CommandResult] + One result per request. + """ + return [await self.run(request) for request in requests] + @t.runtime_checkable class SupportsCommandLine(t.Protocol): diff --git a/src/libtmux/engines/record.py b/src/libtmux/engines/record.py new file mode 100644 index 0000000000..0e6788f1ba --- /dev/null +++ b/src/libtmux/engines/record.py @@ -0,0 +1,233 @@ +"""Record tmux traffic once, then replay it with no tmux server. + +Simulating tmux is not practical: libtmux's listing queries ask for 136 format +fields per row, version-gated, so a hand-written fake drifts out of date the +moment tmux adds a field -- and a fake that answers everything with "success" +is worse than none, because ``has-session`` then reports that every session +exists while ``list-sessions`` reports none. + +Recording sidesteps that. :class:`RecordingEngine` wraps a real engine and keeps +what tmux actually said; :class:`ReplayEngine` serves those answers back. The +rows are real, so :mod:`libtmux.neo` parses them exactly as it would live, and +they stay correct for the tmux version they were taken on. + +A replay engine fails closed: a command that was never recorded raises +:exc:`~libtmux.exc.UnscriptedCommand` rather than inventing an answer. +""" + +from __future__ import annotations + +import typing as t + +from libtmux import exc +from libtmux.engines.base import CommandResult, TmuxEngine + +if t.TYPE_CHECKING: + from collections.abc import Iterator, Mapping, Sequence + + from libtmux.engines.base import CommandRequest + +#: A recorded exchange: the request argv, mapped to what tmux answered. +Tape = t.Mapping[tuple[str, ...], CommandResult] + + +class RecordingEngine(TmuxEngine): + """Run commands through another engine, keeping what tmux answered. + + Wrap the engine a live :class:`~libtmux.Server` would use, exercise your + code, then keep :attr:`tape` for :class:`ReplayEngine`. Doubles as a spy: + :attr:`requests` is every argv in dispatch order, including repeats. + + Parameters + ---------- + inner : TmuxEngine + The engine that actually talks to tmux. + + Attributes + ---------- + requests : list[tuple[str, ...]] + Every request argv, in dispatch order. + + Examples + -------- + >>> from libtmux.engines import RecordingEngine, SubprocessEngine + >>> from libtmux.server import Server + >>> recorder = RecordingEngine(SubprocessEngine.for_server(server)) + >>> live = Server(socket_name=server.socket_name, engine=recorder) + >>> _ = live.cmd("display-message", "-p", "recorded") + >>> recorder.requests + [('display-message', '-p', 'recorded')] + >>> recorder.tape[("display-message", "-p", "recorded")].stdout + ('recorded',) + """ + + def __init__(self, inner: TmuxEngine) -> None: + self._inner = inner + self._tape: dict[tuple[str, ...], CommandResult] = {} + self.requests: list[tuple[str, ...]] = [] + + @property + def tape(self) -> Tape: + """Return the recorded exchanges, keyed by request argv. + + A repeated command keeps its most recent answer, so the tape describes + the end state rather than every intermediate one. + """ + return dict(self._tape) + + def run(self, request: CommandRequest) -> CommandResult: + """Dispatch through the inner engine and record the answer.""" + result = self._inner.run(request) + self.requests.append(request.args) + # Drop the Popen: a tape outlives the process it was recorded from. + self._tape[request.args] = CommandResult( + cmd=result.cmd, + stdout=result.stdout, + stderr=result.stderr, + returncode=result.returncode, + ) + return result + + def to_dict(self) -> list[dict[str, t.Any]]: + """Return the tape as JSON-serializable data. + + Use it to commit a tape next to the tests that replay it, so a suite + that needs a real tmux to *record* needs none to *run*. + + Returns + ------- + list[dict] + One entry per recorded command. + + Examples + -------- + >>> from libtmux.engines import ( + ... CommandRequest, + ... RecordingEngine, + ... SubprocessEngine, + ... ) + >>> recorder = RecordingEngine(SubprocessEngine.for_server(server)) + >>> _ = recorder.run(CommandRequest.from_args("display-message", "-p", "x")) + >>> entry = recorder.to_dict()[0] + >>> entry["args"], entry["stdout"], entry["returncode"] + (['display-message', '-p', 'x'], ['x'], 0) + """ + return [ + { + "args": list(args), + "cmd": list(result.cmd), + "stdout": list(result.stdout), + "stderr": list(result.stderr), + "returncode": result.returncode, + } + for args, result in self._tape.items() + ] + + +class ReplayEngine(TmuxEngine): + """Answer commands from a recorded tape, touching no tmux server. + + Fails closed. A command absent from the tape raises + :exc:`~libtmux.exc.UnscriptedCommand`, because the alternative -- inventing + a plausible answer -- is how a fake reports that every session exists and no + sessions exist at the same time. + + Parameters + ---------- + tape : Mapping[tuple[str, ...], CommandResult] + Recorded exchanges, as produced by :attr:`RecordingEngine.tape`. + + Attributes + ---------- + requests : list[tuple[str, ...]] + Every request argv served, in order. + + Examples + -------- + >>> from libtmux.engines import CommandResult, ReplayEngine + >>> from libtmux.server import Server + >>> tape = {("display-message", "-p", "hi"): CommandResult( + ... cmd=("tmux", "display-message", "-p", "hi"), stdout=("hi",) + ... )} + >>> Server(engine=ReplayEngine(tape)).cmd("display-message", "-p", "hi").stdout + ['hi'] + + An unrecorded command says so, instead of guessing: + + >>> Server(engine=ReplayEngine(tape)).cmd("kill-server") + Traceback (most recent call last): + ... + libtmux.exc.UnscriptedCommand: no recorded result for 'kill-server' + """ + + def __init__(self, tape: Tape) -> None: + self._tape = dict(tape) + self.requests: list[tuple[str, ...]] = [] + + @classmethod + def from_dict(cls, entries: Sequence[Mapping[str, t.Any]]) -> ReplayEngine: + """Rebuild an engine from :meth:`RecordingEngine.to_dict` output. + + Parameters + ---------- + entries : Sequence[Mapping] + Serialized tape entries. + + Returns + ------- + ReplayEngine + + Examples + -------- + >>> from libtmux.engines import ReplayEngine + >>> from libtmux.server import Server + >>> engine = ReplayEngine.from_dict([ + ... { + ... "args": ["display-message", "-p", "hi"], + ... "cmd": ["tmux", "display-message", "-p", "hi"], + ... "stdout": ["hi"], + ... "stderr": [], + ... "returncode": 0, + ... } + ... ]) + >>> Server(engine=engine).cmd("display-message", "-p", "hi").stdout + ['hi'] + """ + return cls( + { + tuple(entry["args"]): CommandResult( + cmd=tuple(entry.get("cmd", ())), + stdout=tuple(entry.get("stdout", ())), + stderr=tuple(entry.get("stderr", ())), + returncode=int(entry.get("returncode", 0)), + ) + for entry in entries + }, + ) + + def run(self, request: CommandRequest) -> CommandResult: + """Return the recorded answer for *request*. + + Raises + ------ + :exc:`~libtmux.exc.UnscriptedCommand` + The tape holds no answer for this argv. + """ + try: + result = self._tape[request.args] + except KeyError: + raise exc.UnscriptedCommand(request.args) from None + self.requests.append(request.args) + return result + + def __contains__(self, args: object) -> bool: + """Whether the tape can answer *args*.""" + return args in self._tape + + def __len__(self) -> int: + """Return how many distinct commands the tape answers.""" + return len(self._tape) + + def __iter__(self) -> Iterator[tuple[str, ...]]: + """Iterate the argvs the tape can answer.""" + return iter(self._tape) diff --git a/src/libtmux/exc.py b/src/libtmux/exc.py index 57bb06102f..a55630a838 100644 --- a/src/libtmux/exc.py +++ b/src/libtmux/exc.py @@ -99,6 +99,34 @@ class TmuxCommandNotFound(LibTmuxException): """Application binary for tmux not found.""" +class UnscriptedCommand(LibTmuxException): + """A replay engine was asked for a command it never recorded. + + Raised by :class:`~libtmux.engines.record.ReplayEngine` instead of + fabricating a result. A fake that answers unknown commands optimistically + reports contradictory state -- ``has-session`` succeeding while + ``list-sessions`` is empty -- and the contradiction surfaces far from its + cause. + + Parameters + ---------- + args : tuple[str, ...] + The request argv with no recorded answer. + + Examples + -------- + >>> raise UnscriptedCommand(("kill-server",)) + Traceback (most recent call last): + ... + libtmux.exc.UnscriptedCommand: no recorded result for 'kill-server' + """ + + def __init__(self, args: tuple[str, ...]) -> None: + self.args_requested = tuple(args) + rendered = " ".join(self.args_requested) or "" + super().__init__(f"no recorded result for {rendered!r}") + + class NotInsideTmux(LibTmuxException): """Raised when the process is not running inside a tmux pane. diff --git a/src/libtmux/pytest_plugin.py b/src/libtmux/pytest_plugin.py index fcc3ce052d..8b636b10e2 100644 --- a/src/libtmux/pytest_plugin.py +++ b/src/libtmux/pytest_plugin.py @@ -14,6 +14,7 @@ from libtmux import exc from libtmux._internal.control_mode import ControlMode +from libtmux.engines import RecordingEngine, SubprocessEngine from libtmux.server import Server from libtmux.test.constants import TEST_SESSION_PREFIX from libtmux.test.random import get_test_session_name, namer @@ -182,6 +183,54 @@ def fin() -> None: return server +@pytest.fixture +def recording_server( + request: pytest.FixtureRequest, + monkeypatch: pytest.MonkeyPatch, + config_file: pathlib.Path, +) -> Server: + """Return a temporary :class:`libtmux.Server` that records its tmux traffic. + + Behaves exactly like the :func:`server` fixture, plus a + :class:`~libtmux.engines.record.RecordingEngine` on + ``server.engine``. Use it two ways: as a spy, asserting on which tmux + commands your code issued, and as a recorder, keeping + ``server.engine.to_dict()`` as a tape a + :class:`~libtmux.engines.record.ReplayEngine` can serve back with no tmux + running. + + >>> from libtmux.server import Server + + >>> def test_spy(recording_server: Server) -> None: + ... session = recording_server.new_session('spied') + ... session.kill() + ... issued = [argv[0] for argv in recording_server.engine.requests] + ... assert 'kill-session' in issued + + .. :: + >>> source = ''.join([e.source for e in request._pyfuncitem.dtest.examples][:2]) + >>> pytester = request.getfixturevalue('pytester') + + >>> pytester.makepyfile(**{'whatever.py': source}) + PosixPath(...) + + >>> result = pytester.runpytest('whatever.py', '--disable-warnings') + ===... + + >>> result.assert_outcomes(passed=1) + """ + socket_name = f"libtmux_test{next(namer)}" + engine = RecordingEngine(SubprocessEngine.of(server_args=(f"-L{socket_name}",))) + server = Server(socket_name=socket_name, engine=engine) + + def fin() -> None: + _reap_test_server(socket_name) + + request.addfinalizer(fin) + + return server + + @pytest.fixture def session_params() -> dict[str, t.Any]: """Return default session creation parameters. diff --git a/src/libtmux/server.py b/src/libtmux/server.py index 1d66023dda..be91842c20 100644 --- a/src/libtmux/server.py +++ b/src/libtmux/server.py @@ -7,6 +7,7 @@ from __future__ import annotations +import inspect import logging import os import pathlib @@ -20,7 +21,11 @@ from libtmux.client import Client from libtmux.common import get_version, has_gte_version, raise_if_stderr, tmux_cmd from libtmux.constants import OptionScope -from libtmux.engines.base import CommandRequest, SupportsConnection +from libtmux.engines.base import ( + CommandRequest, + SupportsConnection, + TmuxEngine, +) from libtmux.engines.connection import ServerConnection from libtmux.engines.subprocess import SubprocessEngine from libtmux.hooks import HooksMixin @@ -45,13 +50,47 @@ from typing_extensions import Self from libtmux._internal.types import StrPath - from libtmux.engines.base import TmuxEngine DashLiteral: TypeAlias = t.Literal["-"] logger = logging.getLogger(__name__) +def _validated_engine(engine: TmuxEngine | None) -> TmuxEngine | None: + """Return *engine*, rejecting anything a synchronous server cannot drive. + + :class:`~libtmux.engines.base.TmuxEngine` is a runtime-checkable + :class:`typing.Protocol`, so :func:`isinstance` confirms that ``run`` and + ``run_batch`` exist but says nothing about their signatures -- and an + :class:`~libtmux.engines.base.AsyncTmuxEngine` satisfies it too, because its + methods have the same names. Both failures otherwise surface far from the + constructor: a missing method as an :exc:`AttributeError` inside + :meth:`Server.cmd`, and a coroutine as a result object that has no + ``returncode``. Checking here names the real problem instead. + """ + if engine is None: + return None + if not isinstance(engine, TmuxEngine): + missing = [ + name + for name in ("run", "run_batch") + if not callable(getattr(engine, name, None)) + ] + msg = ( + f"{type(engine).__name__} is not a TmuxEngine: missing " + f"{', '.join(missing)}. Subclass libtmux.engines.TmuxEngine to " + f"inherit run_batch, or define both methods." + ) + raise exc.LibTmuxException(msg) + if inspect.iscoroutinefunction(engine.run): + msg = ( + f"{type(engine).__name__}.run() is async; Server is synchronous. " + f"Await an AsyncTmuxEngine directly instead of passing it to Server." + ) + raise exc.LibTmuxException(msg) + return engine + + def _is_daemon_not_up_error(stderr_text: str) -> bool: """Return True if the error indicates the tmux server is not running. @@ -199,7 +238,7 @@ def __init__( ) -> None: EnvironmentMixin.__init__(self, "-g") self.tmux_bin = str(tmux_bin) if tmux_bin is not None else None - self._engine = engine + self._engine = _validated_engine(engine) self._default_engine = None self._adopted_engine = None self._connection = None From d889adaad15febc761e7d6d90bce354e2ea33bf9 Mon Sep 17 00:00:00 2001 From: Tony Narlock Date: Wed, 12 Aug 2026 17:22:27 -0500 Subject: [PATCH 03/32] Engines(fix): Let a replay answer without a tmux binary MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit why: Version detection was the one dispatch site the seam never covered. neo.fetch_objs resolved the version-gated -F template by running tmux -V through its own engine, so a replay still needed the binary it exists to avoid — and the lenient list accessors turned the resulting TmuxCommandNotFound into an empty result rather than an error. what: - Resolve the tmux version from the engine when it reports one, falling back to tmux -V for engines that cannot - Record the tmux version in the tape; ReplayEngine reports it, so a listing query resolves with no tmux installed - Propagate exc.UnscriptedCommand through Server.sessions and .clients; leniency covers an unreachable tmux, not an untaught engine - Elide long arguments in UnscriptedCommand and name the recorded version, so a miss on a listing query is legible - Describe the format-field set as version-gated rather than counting it --- CHANGES | 10 +++ docs/topics/engines.md | 41 +++++++++-- src/libtmux/engines/record.py | 129 ++++++++++++++++++++++++---------- src/libtmux/exc.py | 43 +++++++++++- src/libtmux/neo.py | 33 ++++++++- src/libtmux/server.py | 10 +++ tests/test_engines.py | 47 +++++++++++++ 7 files changed, 267 insertions(+), 46 deletions(-) diff --git a/CHANGES b/CHANGES index afc8f156d6..dac1600681 100644 --- a/CHANGES +++ b/CHANGES @@ -125,6 +125,16 @@ engine fails closed, raising {exc}`~libtmux.exc.UnscriptedCommand` for a command it never recorded. The `recording_server` pytest fixture provides a server that records, which doubles as a spy over the tmux commands your code issued. +A tape carries the tmux version it was recorded against, and a replay engine +reports it through {class}`~libtmux.engines.base.SupportsTmuxVersion`. That is +what lets a replay serve listing queries with no tmux binary present at all: +the version-gated `-F` template is otherwise resolved by running `tmux -V`. + +{attr}`Server.sessions ` and +{attr}`Server.clients ` stay lenient when tmux cannot be +reached, but no longer swallow {exc}`~libtmux.exc.UnscriptedCommand` — an engine +that was never taught to answer is a gap in a fixture, not an unreachable tmux. + #### Result objects report success directly {attr}`CommandResult.ok ` and diff --git a/docs/topics/engines.md b/docs/topics/engines.md index 493011d130..5830062bba 100644 --- a/docs/topics/engines.md +++ b/docs/topics/engines.md @@ -135,10 +135,12 @@ out: ## Testing without tmux -Writing a fake that *simulates* tmux is a trap. libtmux's listing queries ask -tmux for 136 format fields per row, and a fake that answers unknown commands -optimistically ends up reporting that every session exists (`has-session` exits -0) while no sessions exist (`list-sessions` is empty). +Writing a fake that *simulates* tmux is a trap. A listing query asks tmux for +its whole format-field set on every row, and that set is version-gated, so a +hand-written fake goes stale as tmux gains fields. A fake that covers the gap by +answering unknown commands optimistically is worse still: it reports that every +session exists (`has-session` exits 0) while no sessions exist (`list-sessions` +is empty). So record real traffic instead, and play it back. {class}`~libtmux.engines.record.RecordingEngine` wraps a real engine and keeps @@ -158,10 +160,37 @@ what tmux said; {class}`~libtmux.engines.record.ReplayEngine` serves it back: ``` Because the rows came from real tmux, the whole object API works offline — -`sessions`, `windows`, `panes` all hydrate. `to_dict()` and +`sessions`, `windows`, `panes` all hydrate. +{meth}`~libtmux.engines.record.RecordingEngine.to_dict` and {meth}`~libtmux.engines.record.ReplayEngine.from_dict` round-trip a tape through JSON, so you can commit one next to the tests that replay it: recording needs -tmux, running does not. +tmux, running does not — not even the binary. + +That last part is why a tape carries the tmux version it was recorded on. The +`-F` template libtmux sends is version-gated, so *something* has to name a +version before a listing query can be built. A replay engine answers with the +recorded one, and a miss says which version the tape came from rather than +leaving you to guess: + +```python +>>> from libtmux.engines import ReplayEngine +>>> from libtmux.server import Server +>>> offline = Server( +... tmux_bin="/nonexistent/tmux", +... engine=ReplayEngine({}, tmux_version="3.7"), +... ) +>>> offline.sessions +Traceback (most recent call last): +... +libtmux.exc.UnscriptedCommand: no recorded result for 'list-sessions +<...-char arg>' (tape recorded on tmux 3.7) +``` + +Note that this raises rather than returning an empty list. +{attr}`Server.sessions ` is lenient by contract when +tmux cannot be reached, but an engine that was never taught to answer is a gap +in your fixture, not an unreachable tmux — reporting "no sessions" there would +hide the bug. A replay engine **fails closed**. A command it never recorded raises {exc}`~libtmux.exc.UnscriptedCommand` rather than inventing an answer: diff --git a/src/libtmux/engines/record.py b/src/libtmux/engines/record.py index 0e6788f1ba..48ef272cfa 100644 --- a/src/libtmux/engines/record.py +++ b/src/libtmux/engines/record.py @@ -1,10 +1,11 @@ """Record tmux traffic once, then replay it with no tmux server. -Simulating tmux is not practical: libtmux's listing queries ask for 136 format -fields per row, version-gated, so a hand-written fake drifts out of date the -moment tmux adds a field -- and a fake that answers everything with "success" -is worse than none, because ``has-session`` then reports that every session -exists while ``list-sessions`` reports none. +Simulating tmux is not practical. A listing query asks tmux for its whole +format-field set on every row, and that set is version-gated -- it grows as tmux +gains fields -- so a hand-written fake is stale the release after it is written. +A fake that papers over the gap by answering unknown commands optimistically is +worse than none: ``has-session`` then reports that every session exists while +``list-sessions`` reports that none do. Recording sidesteps that. :class:`RecordingEngine` wraps a real engine and keeps what tmux actually said; :class:`ReplayEngine` serves those answers back. The @@ -20,10 +21,14 @@ import typing as t from libtmux import exc -from libtmux.engines.base import CommandResult, TmuxEngine +from libtmux.engines.base import ( + CommandResult, + SupportsTmuxVersion, + TmuxEngine, +) if t.TYPE_CHECKING: - from collections.abc import Iterator, Mapping, Sequence + from collections.abc import Iterator, Mapping from libtmux.engines.base import CommandRequest @@ -66,6 +71,18 @@ def __init__(self, inner: TmuxEngine) -> None: self._tape: dict[tuple[str, ...], CommandResult] = {} self.requests: list[tuple[str, ...]] = [] + def tmux_version(self) -> str | None: + """Report the version of the tmux being recorded, if the inner engine knows. + + Recorded alongside the tape so a :class:`ReplayEngine` can answer with + it later. The ``-F`` template libtmux sends is version-gated, so a tape + replayed against a different version would miss every listing query. + """ + inner = self._inner + if isinstance(inner, SupportsTmuxVersion): + return inner.tmux_version() + return None + @property def tape(self) -> Tape: """Return the recorded exchanges, keyed by request argv. @@ -88,16 +105,21 @@ def run(self, request: CommandRequest) -> CommandResult: ) return result - def to_dict(self) -> list[dict[str, t.Any]]: + def to_dict(self) -> dict[str, t.Any]: """Return the tape as JSON-serializable data. Use it to commit a tape next to the tests that replay it, so a suite that needs a real tmux to *record* needs none to *run*. + The tmux version rides along with the commands, because the ``-F`` + template libtmux sends is version-gated: replaying a tape against a + different tmux would miss every listing query, and a bare + "no recorded result" would not explain why. + Returns ------- - list[dict] - One entry per recorded command. + dict + ``{"tmux_version": str | None, "commands": [...]}``. Examples -------- @@ -108,20 +130,26 @@ def to_dict(self) -> list[dict[str, t.Any]]: ... ) >>> recorder = RecordingEngine(SubprocessEngine.for_server(server)) >>> _ = recorder.run(CommandRequest.from_args("display-message", "-p", "x")) - >>> entry = recorder.to_dict()[0] + >>> tape = recorder.to_dict() + >>> sorted(tape) + ['commands', 'tmux_version'] + >>> entry = tape["commands"][0] >>> entry["args"], entry["stdout"], entry["returncode"] (['display-message', '-p', 'x'], ['x'], 0) """ - return [ - { - "args": list(args), - "cmd": list(result.cmd), - "stdout": list(result.stdout), - "stderr": list(result.stderr), - "returncode": result.returncode, - } - for args, result in self._tape.items() - ] + return { + "tmux_version": self.tmux_version(), + "commands": [ + { + "args": list(args), + "cmd": list(result.cmd), + "stdout": list(result.stdout), + "stderr": list(result.stderr), + "returncode": result.returncode, + } + for args, result in self._tape.items() + ], + } class ReplayEngine(TmuxEngine): @@ -160,18 +188,35 @@ class ReplayEngine(TmuxEngine): libtmux.exc.UnscriptedCommand: no recorded result for 'kill-server' """ - def __init__(self, tape: Tape) -> None: + def __init__(self, tape: Tape, *, tmux_version: str | None = None) -> None: self._tape = dict(tape) + self._tmux_version = tmux_version self.requests: list[tuple[str, ...]] = [] + def tmux_version(self) -> str | None: + """Report the tmux version the tape was recorded against. + + Satisfies :class:`~libtmux.engines.base.SupportsTmuxVersion`, which is + what lets a replay serve listing queries with no tmux installed: the + version-gated ``-F`` template is otherwise resolved by running + ``tmux -V``, and a machine replaying a tape may have no tmux at all. + + Examples + -------- + >>> from libtmux.engines import ReplayEngine + >>> ReplayEngine({}, tmux_version="3.7").tmux_version() + '3.7' + """ + return self._tmux_version + @classmethod - def from_dict(cls, entries: Sequence[Mapping[str, t.Any]]) -> ReplayEngine: + def from_dict(cls, tape: Mapping[str, t.Any]) -> ReplayEngine: """Rebuild an engine from :meth:`RecordingEngine.to_dict` output. Parameters ---------- - entries : Sequence[Mapping] - Serialized tape entries. + tape : Mapping + A serialized tape: ``{"tmux_version": ..., "commands": [...]}``. Returns ------- @@ -181,15 +226,18 @@ def from_dict(cls, entries: Sequence[Mapping[str, t.Any]]) -> ReplayEngine: -------- >>> from libtmux.engines import ReplayEngine >>> from libtmux.server import Server - >>> engine = ReplayEngine.from_dict([ - ... { - ... "args": ["display-message", "-p", "hi"], - ... "cmd": ["tmux", "display-message", "-p", "hi"], - ... "stdout": ["hi"], - ... "stderr": [], - ... "returncode": 0, - ... } - ... ]) + >>> engine = ReplayEngine.from_dict({ + ... "tmux_version": "3.7", + ... "commands": [ + ... { + ... "args": ["display-message", "-p", "hi"], + ... "cmd": ["tmux", "display-message", "-p", "hi"], + ... "stdout": ["hi"], + ... "stderr": [], + ... "returncode": 0, + ... } + ... ], + ... }) >>> Server(engine=engine).cmd("display-message", "-p", "hi").stdout ['hi'] """ @@ -201,8 +249,9 @@ def from_dict(cls, entries: Sequence[Mapping[str, t.Any]]) -> ReplayEngine: stderr=tuple(entry.get("stderr", ())), returncode=int(entry.get("returncode", 0)), ) - for entry in entries + for entry in tape.get("commands", ()) }, + tmux_version=tape.get("tmux_version"), ) def run(self, request: CommandRequest) -> CommandResult: @@ -216,7 +265,15 @@ def run(self, request: CommandRequest) -> CommandResult: try: result = self._tape[request.args] except KeyError: - raise exc.UnscriptedCommand(request.args) from None + # A listing query's -F template is version-gated, so the commonest + # cause of a miss on a tape that "should" have it is replaying + # against a different tmux than the one recorded. + hint = ( + f"tape recorded on tmux {self._tmux_version}" + if self._tmux_version + else None + ) + raise exc.UnscriptedCommand(request.args, hint) from None self.requests.append(request.args) return result diff --git a/src/libtmux/exc.py b/src/libtmux/exc.py index a55630a838..b851b71222 100644 --- a/src/libtmux/exc.py +++ b/src/libtmux/exc.py @@ -108,10 +108,22 @@ class UnscriptedCommand(LibTmuxException): ``list-sessions`` is empty -- and the contradiction surfaces far from its cause. + A listing query carries tmux's whole ``-F`` template, which runs to + thousands of characters, so long arguments are elided: the point of the + message is which command went unanswered, not the template's contents. + Parameters ---------- args : tuple[str, ...] The request argv with no recorded answer. + hint : str, optional + Extra guidance appended to the message, e.g. the tmux version a tape + was recorded against. + + Attributes + ---------- + args_requested : tuple[str, ...] + The full, un-elided argv, for a caller that wants to inspect it. Examples -------- @@ -119,12 +131,37 @@ class UnscriptedCommand(LibTmuxException): Traceback (most recent call last): ... libtmux.exc.UnscriptedCommand: no recorded result for 'kill-server' + + A long argument is summarized rather than dumped: + + >>> raise UnscriptedCommand(("list-sessions", "-F" + "#" * 99)) + Traceback (most recent call last): + ... + libtmux.exc.UnscriptedCommand: no recorded result for 'list-sessions <101-char arg>' + + >>> raise UnscriptedCommand(("list-sessions",), hint="tape recorded on 3.7") + Traceback (most recent call last): + ... + libtmux.exc.UnscriptedCommand: no recorded result for 'list-sessions' + (tape recorded on 3.7) """ - def __init__(self, args: tuple[str, ...]) -> None: + #: Arguments longer than this are replaced by a length summary. + _MAX_ARG_CHARS = 60 + + def __init__(self, args: tuple[str, ...], hint: str | None = None) -> None: self.args_requested = tuple(args) - rendered = " ".join(self.args_requested) or "" - super().__init__(f"no recorded result for {rendered!r}") + rendered = ( + " ".join( + arg if len(arg) <= self._MAX_ARG_CHARS else f"<{len(arg)}-char arg>" + for arg in self.args_requested + ) + or "" + ) + message = f"no recorded result for {rendered!r}" + if hint: + message = f"{message} ({hint})" + super().__init__(message) class NotInsideTmux(LibTmuxException): diff --git a/src/libtmux/neo.py b/src/libtmux/neo.py index feb8bb0e47..d5567d0873 100644 --- a/src/libtmux/neo.py +++ b/src/libtmux/neo.py @@ -12,6 +12,7 @@ from libtmux import exc from libtmux._compat import LooseVersion from libtmux.common import get_version, raise_if_stderr, tmux_cmd +from libtmux.engines.base import SupportsTmuxVersion from libtmux.formats import FORMAT_SEPARATOR if t.TYPE_CHECKING: @@ -1036,6 +1037,36 @@ def parse_output( return {k: v for k, v in formatter.items() if v} +def _resolve_tmux_version(server: Server) -> str: + """Return the tmux version the server's engine targets. + + The format string a listing query sends is version-gated, so something has + to name a version before any row can be requested. Ask the *engine* first: + it is the only party that knows what it is talking to, and for a replaying + or in-memory engine there may be no tmux binary to interrogate at all. + Engines that cannot answer -- the + :class:`~libtmux.engines.base.SupportsTmuxVersion` capability is optional -- + fall back to running ``tmux -V``, which is what every engine did before one + could report for itself. + + Parameters + ---------- + server : :class:`~libtmux.server.Server` + The server whose engine is asked. + + Returns + ------- + str + A tmux version string, e.g. ``"3.7"``. + """ + engine = server.engine + if isinstance(engine, SupportsTmuxVersion): + version = engine.tmux_version() + if version is not None: + return version + return str(get_version(tmux_bin=server.tmux_bin)) + + def fetch_objs( server: Server, list_cmd: ListCmd, @@ -1095,7 +1126,7 @@ def fetch_objs( >>> 'session_id' in objs[0] True """ - tmux_version = str(get_version(tmux_bin=server.tmux_bin)) + tmux_version = _resolve_tmux_version(server) _fields, format_string = get_output_format(list_cmd, tmux_version) tmux_cmds: list[str | int] = [list_cmd] diff --git a/src/libtmux/server.py b/src/libtmux/server.py index be91842c20..4802ff8075 100644 --- a/src/libtmux/server.py +++ b/src/libtmux/server.py @@ -2567,6 +2567,11 @@ def sessions(self) -> QueryList[Session]: Session(server=self, **obj) for obj in fetch_objs(server=self, list_cmd="list-sessions") ] + except exc.UnscriptedCommand: + # An incomplete tape is a bug in the caller's fixture, not a tmux + # that is merely unreachable. Leniency here would report "no rows" + # for a command the engine was never taught to answer. + raise except exc.LibTmuxException: return QueryList([]) return QueryList(sessions) @@ -2639,6 +2644,11 @@ def clients(self) -> QueryList[Client]: Client(server=self, **obj) for obj in fetch_objs(server=self, list_cmd="list-clients") ] + except exc.UnscriptedCommand: + # An incomplete tape is a bug in the caller's fixture, not a tmux + # that is merely unreachable. Leniency here would report "no rows" + # for a command the engine was never taught to answer. + raise except exc.LibTmuxException: return QueryList([]) return QueryList(clients) diff --git a/tests/test_engines.py b/tests/test_engines.py index 656dd08ade..3f003fd450 100644 --- a/tests/test_engines.py +++ b/tests/test_engines.py @@ -2,6 +2,7 @@ from __future__ import annotations +import json import subprocess import typing as t import warnings @@ -14,6 +15,8 @@ CommandRequest, CommandResult, CommandSeparator, + RecordingEngine, + ReplayEngine, ServerConnection, SubprocessEngine, SupportsCommandLine, @@ -311,3 +314,47 @@ def test_run_batch_preserves_order(session: Session) -> None: ], ) assert [result.stdout[0] for result in results] == ["a", "b"] + + +def test_replay_hydrates_objects_without_a_tmux_binary(session: Session) -> None: + """A tape answers listing queries on a machine with no tmux installed. + + The ``-F`` template is version-gated, so something must name a tmux version + before a listing query can be built. Resolving that by running ``tmux -V`` + made replay depend on the very binary it exists to avoid, and the failure + was silent: the lenient list accessors turned it into an empty result. + """ + server = session.server + recorder = RecordingEngine(SubprocessEngine.for_server(server)) + recording = Server(socket_name=server.socket_name, engine=recorder) + assert [s.session_name for s in recording.sessions] + + tape = json.loads(json.dumps(recorder.to_dict())) + assert tape["tmux_version"] + + offline = Server( + socket_name=server.socket_name, + tmux_bin="/nonexistent/tmux", + engine=ReplayEngine.from_dict(tape), + ) + assert [s.session_name for s in offline.sessions] == [ + s.session_name for s in recording.sessions + ] + + +def test_unscripted_command_is_not_swallowed_by_list_accessors() -> None: + """An incomplete tape raises rather than reporting "no sessions". + + ``Server.sessions`` is lenient by contract, but that contract covers a tmux + that cannot be reached -- not an engine that was never taught to answer. + """ + server = Server( + tmux_bin="/nonexistent/tmux", engine=ReplayEngine({}, tmux_version="3.7") + ) + with pytest.raises(exc.UnscriptedCommand) as excinfo: + _ = server.sessions + message = str(excinfo.value) + assert "list-sessions" in message + assert "3.7" in message + # The -F template must be summarized, not dumped into the message. + assert len(message) < 200 From 6ae49f811302b835215d0e937e1d25164d034878 Mon Sep 17 00:00:00 2001 From: Tony Narlock Date: Wed, 12 Aug 2026 17:27:32 -0500 Subject: [PATCH 04/32] Engines(fix): Replay recorded answers in order MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit why: A tape keyed by argv kept one answer per command, so a query whose answer changed — list-sessions before and after a session is created — replayed the end state for both. A test asserting that transition passed against the wrong value, silently. what: - Record an ordered sequence of Exchange values instead of a mapping; ReplayEngine serves each command's answers in the order recorded - Repeat an answer that never varied, so a read-only query stays usable more often than it was recorded - Raise once a varying answer is exhausted, naming how many times it was recorded and requested - Build from_dict as a list, not a dict comprehension, which was collapsing duplicates back to the last answer - Accept a mapping for a hand-written tape with one answer per command --- CHANGES | 6 ++ docs/topics/engines.md | 29 ++++++- src/libtmux/engines/__init__.py | 8 +- src/libtmux/engines/record.py | 131 +++++++++++++++++++++++--------- tests/test_engines.py | 45 +++++++++++ 5 files changed, 183 insertions(+), 36 deletions(-) diff --git a/CHANGES b/CHANGES index dac1600681..794b481128 100644 --- a/CHANGES +++ b/CHANGES @@ -125,6 +125,12 @@ engine fails closed, raising {exc}`~libtmux.exc.UnscriptedCommand` for a command it never recorded. The `recording_server` pytest fixture provides a server that records, which doubles as a spy over the tmux commands your code issued. +A tape keeps every exchange in order rather than one answer per command, and a +replay serves them in order. A command whose answer never varied while recording +replays as often as needed; one that varied raises once its answers run out, +because repeating the final answer would report the end state for an earlier +step. + A tape carries the tmux version it was recorded against, and a replay engine reports it through {class}`~libtmux.engines.base.SupportsTmuxVersion`. That is what lets a replay serve listing queries with no tmux binary present at all: diff --git a/docs/topics/engines.md b/docs/topics/engines.md index 5830062bba..c3b5fb9114 100644 --- a/docs/topics/engines.md +++ b/docs/topics/engines.md @@ -186,7 +186,34 @@ libtmux.exc.UnscriptedCommand: no recorded result for 'list-sessions <...-char arg>' (tape recorded on tmux 3.7) ``` -Note that this raises rather than returning an empty list. +A tape keeps every exchange in order, not one answer per command. That matters +whenever state changes underneath a repeated query — `list-sessions` before and +after a session is created — because a tape that remembered only the last answer +would replay the end state for both, and a test asserting the transition would +pass against the wrong value. Replay serves the recorded answers in order. + +A command whose answer never varied while recording may be replayed as often as +you like. One that *did* vary has no defensible reply once its answers run out, +so it raises rather than repeating the final one: + +```python +>>> from libtmux.engines import CommandResult, Exchange, ReplayEngine +>>> from libtmux.server import Server +>>> tape = [ +... Exchange(("list-sessions",), CommandResult(cmd=("tmux",), stdout=("one",))), +... Exchange(("list-sessions",), CommandResult(cmd=("tmux",), stdout=("two",))), +... ] +>>> replay = Server(engine=ReplayEngine(tape)) +>>> replay.cmd("list-sessions").stdout, replay.cmd("list-sessions").stdout +(['one'], ['two']) +>>> replay.cmd("list-sessions") +Traceback (most recent call last): +... +libtmux.exc.UnscriptedCommand: no recorded result for 'list-sessions' +(answered 2 times while recording, asked 3 times now) +``` + +Note that a missing command raises rather than returning an empty list. {attr}`Server.sessions ` is lenient by contract when tmux cannot be reached, but an engine that was never taught to answer is a gap in your fixture, not an unreachable tmux — reporting "no sessions" there would diff --git a/src/libtmux/engines/__init__.py b/src/libtmux/engines/__init__.py index ba424f9b4d..a1c914d8b7 100644 --- a/src/libtmux/engines/__init__.py +++ b/src/libtmux/engines/__init__.py @@ -53,7 +53,12 @@ split_direct_argv, ) from libtmux.engines.connection import ServerConnection -from libtmux.engines.record import RecordingEngine, ReplayEngine, Tape +from libtmux.engines.record import ( + Exchange, + RecordingEngine, + ReplayEngine, + Tape, +) from libtmux.engines.subprocess import SubprocessEngine __all__ = ( @@ -62,6 +67,7 @@ "CommandResult", "CommandSeparator", "DirectArgv", + "Exchange", "RecordingEngine", "ReplayEngine", "ServerConnection", diff --git a/src/libtmux/engines/record.py b/src/libtmux/engines/record.py index 48ef272cfa..44f212e72c 100644 --- a/src/libtmux/engines/record.py +++ b/src/libtmux/engines/record.py @@ -19,6 +19,7 @@ from __future__ import annotations import typing as t +from collections.abc import Mapping from libtmux import exc from libtmux.engines.base import ( @@ -28,12 +29,30 @@ ) if t.TYPE_CHECKING: - from collections.abc import Iterator, Mapping + from collections.abc import Iterator from libtmux.engines.base import CommandRequest -#: A recorded exchange: the request argv, mapped to what tmux answered. -Tape = t.Mapping[tuple[str, ...], CommandResult] + +class Exchange(t.NamedTuple): + """One recorded command and the answer tmux gave it. + + Attributes + ---------- + args : tuple[str, ...] + The request argv, without the binary or connection flags. + result : CommandResult + What tmux answered. + """ + + args: tuple[str, ...] + result: CommandResult + + +#: A recorded conversation. A sequence preserves order, which matters when the +#: same command is asked twice and answered differently. A mapping is accepted +#: too, for a hand-written tape where each command has one fixed answer. +Tape: t.TypeAlias = "t.Sequence[Exchange] | t.Mapping[tuple[str, ...], CommandResult]" class RecordingEngine(TmuxEngine): @@ -62,13 +81,15 @@ class RecordingEngine(TmuxEngine): >>> _ = live.cmd("display-message", "-p", "recorded") >>> recorder.requests [('display-message', '-p', 'recorded')] - >>> recorder.tape[("display-message", "-p", "recorded")].stdout + >>> recorder.tape[0].args + ('display-message', '-p', 'recorded') + >>> recorder.tape[0].result.stdout ('recorded',) """ def __init__(self, inner: TmuxEngine) -> None: self._inner = inner - self._tape: dict[tuple[str, ...], CommandResult] = {} + self._exchanges: list[Exchange] = [] self.requests: list[tuple[str, ...]] = [] def tmux_version(self) -> str | None: @@ -84,24 +105,31 @@ def tmux_version(self) -> str | None: return None @property - def tape(self) -> Tape: - """Return the recorded exchanges, keyed by request argv. + def tape(self) -> tuple[Exchange, ...]: + """Return every recorded exchange, in the order it happened. - A repeated command keeps its most recent answer, so the tape describes - the end state rather than every intermediate one. + Order is kept rather than collapsed per command, because the same + command asked twice is often answered differently -- ``list-sessions`` + before and after a session is created -- and a tape that remembered only + the last answer would replay the end state for both. """ - return dict(self._tape) + return tuple(self._exchanges) def run(self, request: CommandRequest) -> CommandResult: """Dispatch through the inner engine and record the answer.""" result = self._inner.run(request) self.requests.append(request.args) # Drop the Popen: a tape outlives the process it was recorded from. - self._tape[request.args] = CommandResult( - cmd=result.cmd, - stdout=result.stdout, - stderr=result.stderr, - returncode=result.returncode, + self._exchanges.append( + Exchange( + args=request.args, + result=CommandResult( + cmd=result.cmd, + stdout=result.stdout, + stderr=result.stderr, + returncode=result.returncode, + ), + ), ) return result @@ -147,7 +175,7 @@ def to_dict(self) -> dict[str, t.Any]: "stderr": list(result.stderr), "returncode": result.returncode, } - for args, result in self._tape.items() + for args, result in self._exchanges ], } @@ -162,8 +190,10 @@ class ReplayEngine(TmuxEngine): Parameters ---------- - tape : Mapping[tuple[str, ...], CommandResult] - Recorded exchanges, as produced by :attr:`RecordingEngine.tape`. + tape : Sequence[Exchange] or Mapping[tuple[str, ...], CommandResult] + Recorded exchanges, as produced by :attr:`RecordingEngine.tape`. A + sequence replays in order; a mapping gives each command one fixed + answer, which is convenient for a hand-written tape. Attributes ---------- @@ -189,7 +219,15 @@ class ReplayEngine(TmuxEngine): """ def __init__(self, tape: Tape, *, tmux_version: str | None = None) -> None: - self._tape = dict(tape) + exchanges = ( + [Exchange(args, result) for args, result in tape.items()] + if isinstance(tape, Mapping) + else list(tape) + ) + self._answers: dict[tuple[str, ...], list[CommandResult]] = {} + for args, result in exchanges: + self._answers.setdefault(args, []).append(result) + self._served: dict[tuple[str, ...], int] = {} self._tmux_version = tmux_version self.requests: list[tuple[str, ...]] = [] @@ -241,16 +279,22 @@ def from_dict(cls, tape: Mapping[str, t.Any]) -> ReplayEngine: >>> Server(engine=engine).cmd("display-message", "-p", "hi").stdout ['hi'] """ + # A list, not a dict comprehension: the same command recorded twice + # with different answers must stay two exchanges, or the tape replays + # the end state for the earlier step. return cls( - { - tuple(entry["args"]): CommandResult( - cmd=tuple(entry.get("cmd", ())), - stdout=tuple(entry.get("stdout", ())), - stderr=tuple(entry.get("stderr", ())), - returncode=int(entry.get("returncode", 0)), + [ + Exchange( + args=tuple(entry["args"]), + result=CommandResult( + cmd=tuple(entry.get("cmd", ())), + stdout=tuple(entry.get("stdout", ())), + stderr=tuple(entry.get("stderr", ())), + returncode=int(entry.get("returncode", 0)), + ), ) for entry in tape.get("commands", ()) - }, + ], tmux_version=tape.get("tmux_version"), ) @@ -262,9 +306,8 @@ def run(self, request: CommandRequest) -> CommandResult: :exc:`~libtmux.exc.UnscriptedCommand` The tape holds no answer for this argv. """ - try: - result = self._tape[request.args] - except KeyError: + answers = self._answers.get(request.args) + if not answers: # A listing query's -F template is version-gated, so the commonest # cause of a miss on a tape that "should" have it is replaying # against a different tmux than the one recorded. @@ -274,17 +317,37 @@ def run(self, request: CommandRequest) -> CommandResult: else None ) raise exc.UnscriptedCommand(request.args, hint) from None - self.requests.append(request.args) - return result + + served = self._served.get(request.args, 0) + if served < len(answers): + self._served[request.args] = served + 1 + self.requests.append(request.args) + return answers[served] + + if len(answers) == 1: + # The answer never varied while recording, so repeating it cannot + # misreport a state change. This keeps a read-only query usable more + # often than it was recorded. + self.requests.append(request.args) + return answers[0] + + # It *did* vary, so there is no defensible answer for the extra call: + # replaying the last one would report the end state for a step that + # happened earlier. + hint = ( + f"answered {len(answers)} times while recording, " + f"asked {served + 1} times now" + ) + raise exc.UnscriptedCommand(request.args, hint) from None def __contains__(self, args: object) -> bool: """Whether the tape can answer *args*.""" - return args in self._tape + return args in self._answers def __len__(self) -> int: """Return how many distinct commands the tape answers.""" - return len(self._tape) + return len(self._answers) def __iter__(self) -> Iterator[tuple[str, ...]]: """Iterate the argvs the tape can answer.""" - return iter(self._tape) + return iter(self._answers) diff --git a/tests/test_engines.py b/tests/test_engines.py index 3f003fd450..17a8f90399 100644 --- a/tests/test_engines.py +++ b/tests/test_engines.py @@ -15,6 +15,7 @@ CommandRequest, CommandResult, CommandSeparator, + Exchange, RecordingEngine, ReplayEngine, ServerConnection, @@ -358,3 +359,47 @@ def test_unscripted_command_is_not_swallowed_by_list_accessors() -> None: assert "3.7" in message # The -F template must be summarized, not dumped into the message. assert len(message) < 200 + + +def test_replay_preserves_answers_that_changed(session: Session) -> None: + """A command answered differently twice replays both answers, in order. + + A tape keyed only by argv would remember the last answer and report the end + state for the earlier step, so a test asserting a transition would pass + against the wrong value. + """ + server = session.server + recorder = RecordingEngine(SubprocessEngine.for_server(server)) + recording = Server(socket_name=server.socket_name, engine=recorder) + + before = len(recording.sessions) + recording.new_session("replay_seq_extra") + after = len(recording.sessions) + assert after == before + 1 + + offline = Server( + socket_name=server.socket_name, + tmux_bin="/nonexistent/tmux", + engine=ReplayEngine.from_dict(json.loads(json.dumps(recorder.to_dict()))), + ) + assert len(offline.sessions) == before + assert len(offline.sessions) == after + + # The answer demonstrably varied, so there is no defensible reply to a + # third call; guessing one is what this design exists to prevent. + with pytest.raises(exc.UnscriptedCommand, match="asked 3 times now"): + _ = offline.sessions + + +def test_replay_repeats_an_answer_that_never_varied() -> None: + """A command recorded once may be replayed any number of times.""" + tape = [ + Exchange( + ("display-message", "-p", "hi"), + CommandResult(cmd=("tmux",), stdout=("hi",)), + ), + ] + server = Server(engine=ReplayEngine(tape)) + assert [server.cmd("display-message", "-p", "hi").stdout[0] for _ in range(4)] == [ + "hi", + ] * 4 From 55c0356c1935db37240cefa34e91e799caad9313 Mon Sep 17 00:00:00 2001 From: Tony Narlock Date: Wed, 12 Aug 2026 17:31:49 -0500 Subject: [PATCH 05/32] Engines(feat): Scope an engine to a block why: Swapping an engine meant rebuilding the Server, so every caller that wanted to record or fake a section of code rebuilt one by hand and had no way back. Recording in particular needs the engine already in use, which a constructor argument cannot express. what: - Add Server.using(engine): dispatch through engine for a block, restore on the way out including on exception, nesting in reverse - Validate the engine there exactly as the constructor does - Add Server.recording(): wrap the engine already in use, yield the recorder, restore afterwards - Document both in docs/topics/engines.md and CHANGES --- CHANGES | 5 +++ docs/topics/engines.md | 21 +++++++++-- src/libtmux/server.py | 82 ++++++++++++++++++++++++++++++++++++++++++ tests/test_engines.py | 73 +++++++++++++++++++++++++++++++++++++ 4 files changed, 179 insertions(+), 2 deletions(-) diff --git a/CHANGES b/CHANGES index 794b481128..92777591f2 100644 --- a/CHANGES +++ b/CHANGES @@ -125,6 +125,11 @@ engine fails closed, raising {exc}`~libtmux.exc.UnscriptedCommand` for a command it never recorded. The `recording_server` pytest fixture provides a server that records, which doubles as a spy over the tmux commands your code issued. +{meth}`Server.using() ` swaps the engine for a block and +restores it afterwards, including when the block raises; scopes nest. +{meth}`Server.recording() ` builds on it to record +everything a block issues against the engine already in use. + A tape keeps every exchange in order rather than one answer per command, and a replay serves them in order. A command whose answer never varied while recording replays as often as needed; one that varied raises once its answers run out, diff --git a/docs/topics/engines.md b/docs/topics/engines.md index c3b5fb9114..917407b925 100644 --- a/docs/topics/engines.md +++ b/docs/topics/engines.md @@ -143,8 +143,25 @@ session exists (`has-session` exits 0) while no sessions exist (`list-sessions` is empty). So record real traffic instead, and play it back. -{class}`~libtmux.engines.record.RecordingEngine` wraps a real engine and keeps -what tmux said; {class}`~libtmux.engines.record.ReplayEngine` serves it back: +{meth}`Server.recording() ` is the short way — it +records everything the block issues, against the engine the server already uses, +and restores that engine afterwards: + +```python +>>> from libtmux.engines import ReplayEngine +>>> from libtmux.server import Server + +>>> with server.recording() as tape: +... _ = server.cmd("display-message", "-p", "#{session_name}") + +>>> offline = Server(engine=ReplayEngine(tape.tape)) +>>> offline.cmd("display-message", "-p", "#{session_name}").stdout +['libtmux_...'] +``` + +`tape.to_dict()` serializes that for a file. The engines behind it, +{class}`~libtmux.engines.record.RecordingEngine` and +{class}`~libtmux.engines.record.ReplayEngine`, can also be wired by hand: ```python >>> from libtmux.engines import RecordingEngine, ReplayEngine, SubprocessEngine diff --git a/src/libtmux/server.py b/src/libtmux/server.py index 4802ff8075..105f5a374e 100644 --- a/src/libtmux/server.py +++ b/src/libtmux/server.py @@ -7,6 +7,7 @@ from __future__ import annotations +import contextlib import inspect import logging import os @@ -27,6 +28,7 @@ TmuxEngine, ) from libtmux.engines.connection import ServerConnection +from libtmux.engines.record import RecordingEngine from libtmux.engines.subprocess import SubprocessEngine from libtmux.hooks import HooksMixin from libtmux.neo import fetch_objs, get_output_format, parse_output @@ -371,6 +373,86 @@ def engine(self) -> TmuxEngine: self._default_engine = default return default + @contextlib.contextmanager + def using(self, engine: TmuxEngine) -> t.Iterator[Server]: + """Dispatch through *engine* for the duration of the block. + + The previous engine is restored on the way out, including when the block + raises. Scopes nest, unwinding in reverse. + + Parameters + ---------- + engine : :class:`~libtmux.engines.base.TmuxEngine` + The engine to use inside the block. Validated exactly as + ``Server(engine=...)`` validates. + + Yields + ------ + :class:`~libtmux.Server` + This server, so the block can name it. + + Examples + -------- + >>> from libtmux.engines import CommandResult, ReplayEngine + >>> tape = {("display-message", "-p", "x"): CommandResult( + ... cmd=("tmux",), stdout=("canned",) + ... )} + >>> with server.using(ReplayEngine(tape)): + ... server.cmd("display-message", "-p", "x").stdout + ['canned'] + + Outside the block the server is back on its own engine: + + >>> server.cmd("display-message", "-p", "x").stdout + ['x'] + + .. versionadded:: 0.63 + """ + previous_engine = self._engine + previous_adopted = self._adopted_engine + self._engine = _validated_engine(engine) + self._adopted_engine = None + try: + yield self + finally: + self._engine = previous_engine + self._adopted_engine = previous_adopted + + @contextlib.contextmanager + def recording(self) -> t.Iterator[RecordingEngine]: + """Record every tmux command the block issues. + + Wraps whichever engine the server is already using, so the commands + still run for real; the recorder just keeps what tmux answered. Hand the + result to a :class:`~libtmux.engines.record.ReplayEngine` to replay the + same conversation with no tmux running. + + Yields + ------ + :class:`~libtmux.engines.record.RecordingEngine` + The recorder, live during the block and complete after it. + + Examples + -------- + >>> with server.recording() as recorder: + ... _ = server.cmd("display-message", "-p", "hello") + >>> recorder.requests + [('display-message', '-p', 'hello')] + + The tape replays without a tmux server: + + >>> from libtmux.engines import ReplayEngine + >>> from libtmux.server import Server + >>> replayed = Server(engine=ReplayEngine(recorder.tape)) + >>> replayed.cmd("display-message", "-p", "hello").stdout + ['hello'] + + .. versionadded:: 0.63 + """ + recorder = RecordingEngine(self.engine) + with self.using(recorder): + yield recorder + @classmethod def from_env(cls, env: t.Mapping[str, str] | None = None) -> Server: """Return the tmux server this process's pane is attached to. diff --git a/tests/test_engines.py b/tests/test_engines.py index 17a8f90399..8076748f47 100644 --- a/tests/test_engines.py +++ b/tests/test_engines.py @@ -403,3 +403,76 @@ def test_replay_repeats_an_answer_that_never_varied() -> None: assert [server.cmd("display-message", "-p", "hi").stdout[0] for _ in range(4)] == [ "hi", ] * 4 + + +def test_using_scopes_an_engine_to_a_block(session: Session) -> None: + """``Server.using()`` swaps the engine for a block, then restores it.""" + server = session.server + original = server.engine + canned = CannedEngine(stdout=("scoped",)) + + with server.using(canned) as scoped: + assert scoped is server + assert server.engine is canned + assert server.cmd("display-message", "-p", "x").stdout == ["scoped"] + + assert server.engine is original + assert server.cmd("display-message", "-p", "x").stdout != ["scoped"] + + +def test_using_restores_on_exception(session: Session) -> None: + """A raise inside the block still restores the previous engine.""" + server = session.server + original = server.engine + + with pytest.raises(ValueError, match="boom"), server.using(CannedEngine()): + msg = "boom" + raise ValueError(msg) + + assert server.engine is original + + +def test_using_nests(session: Session) -> None: + """Nested scopes unwind in order.""" + server = session.server + outer, inner = CannedEngine(stdout=("outer",)), CannedEngine(stdout=("inner",)) + + with server.using(outer): + assert server.cmd("x").stdout == ["outer"] + with server.using(inner): + assert server.cmd("x").stdout == ["inner"] + assert server.cmd("x").stdout == ["outer"] + + +def test_using_validates_like_the_constructor(session: Session) -> None: + """A non-engine is rejected where it is supplied, not on first command.""" + + class OnlyRun: + def run(self, request: CommandRequest) -> CommandResult: + return CommandResult(cmd=("tmux",)) + + with ( + pytest.raises(exc.LibTmuxException, match="run_batch"), + session.server.using(OnlyRun()), # type: ignore[arg-type] + ): + pass + + +def test_recording_captures_a_block(session: Session) -> None: + """``Server.recording()`` records the block's traffic and restores after.""" + server = session.server + original = server.engine + + with server.recording() as recorder: + server.new_session("recording_ctx") + assert [s.session_name for s in server.sessions] + + assert server.engine is original + assert any(argv[0] == "new-session" for argv in recorder.requests) + + offline = Server( + socket_name=server.socket_name, + tmux_bin="/nonexistent/tmux", + engine=ReplayEngine.from_dict(recorder.to_dict()), + ) + assert [s.session_name for s in offline.sessions] From dc8a515533abcc836de4bc5f9db1345f8bde2738 Mon Sep 17 00:00:00 2001 From: Tony Narlock Date: Wed, 12 Aug 2026 17:32:16 -0500 Subject: [PATCH 06/32] Engines(docs[run_batch]): Say why the batch hook exists why: Nothing in libtmux calls run_batch, which twice read as dead weight worth deleting. It is the override point a persistent-connection engine uses to pipeline down one connection, so removing it would have to be undone as a breaking protocol change. what: - Record the rationale on TmuxEngine.run_batch --- src/libtmux/engines/base.py | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/src/libtmux/engines/base.py b/src/libtmux/engines/base.py index ccb35d6266..097d7c728e 100644 --- a/src/libtmux/engines/base.py +++ b/src/libtmux/engines/base.py @@ -475,7 +475,13 @@ def run_batch(self, requests: Sequence[CommandRequest]) -> list[CommandResult]: """Execute requests in order, returning one result per request. Defaults to a loop over :meth:`run`, which is correct for any stateless - engine. Persistent-connection engines override it to pipeline. + engine. Nothing in libtmux calls it yet -- :meth:`Server.cmd + ` dispatches one command at a time -- but it is not + dead weight: it is the hook a persistent-connection engine overrides to + pipeline a batch down one ``tmux -C`` connection without waiting for + each reply, which is where the round-trip savings live. Removing it + would have to be undone as a breaking protocol change the moment such an + engine lands. Parameters ---------- From 56da09cb72ba9d0631791cad5b01dc9995467fa0 Mon Sep 17 00:00:00 2001 From: Tony Narlock Date: Wed, 12 Aug 2026 17:35:15 -0500 Subject: [PATCH 07/32] Engines(feat[CommandResult]): Make output list-compatible why: Moving Server.cmd() to CommandResult means callers meet tuples where tmux_cmd gave lists. Two patterns break on that: equality against a list (17 sites in-repo, unknown downstream) and isinstance(stderr, list), which gates three error paths that would otherwise stop raising silently. what: - Add Lines, a read-only list subclass that also compares equal to a tuple, and normalize CommandResult's cmd/stdout/stderr to it - Annotate those fields Sequence[str], which is what they now are: indexable, iterable, and not mutable - Update the doctests that showed tuple reprs Proposal branch: Server.cmd() still returns tmux_cmd. This is the enabling step, measured separately so the flip can be judged on its own. --- docs/topics/engines.md | 2 +- src/libtmux/engines/base.py | 82 +++++++++++++++++++++++++++---- src/libtmux/engines/record.py | 2 +- src/libtmux/engines/subprocess.py | 2 +- tests/test_result_compat.py | 36 ++++++++++++++ 5 files changed, 111 insertions(+), 13 deletions(-) create mode 100644 tests/test_result_compat.py diff --git a/docs/topics/engines.md b/docs/topics/engines.md index 917407b925..a13e40d2b6 100644 --- a/docs/topics/engines.md +++ b/docs/topics/engines.md @@ -72,7 +72,7 @@ binary, a dropped connection — raises: ... returncode=1, ... ) >>> result.returncode, result.stderr -(1, ('no such window',)) +(1, ['no such window']) ``` ## Writing an engine diff --git a/src/libtmux/engines/base.py b/src/libtmux/engines/base.py index 097d7c728e..cc3e9d06bd 100644 --- a/src/libtmux/engines/base.py +++ b/src/libtmux/engines/base.py @@ -15,12 +15,12 @@ from __future__ import annotations import typing as t +from collections.abc import Sequence from dataclasses import dataclass, field if t.TYPE_CHECKING: import pathlib import subprocess - from collections.abc import Sequence from typing_extensions import Self @@ -325,6 +325,61 @@ def subcommand(self) -> str: return self.args[0] if self.args else "" +class Lines(list[str]): + """Command output: a list that also compares equal to a tuple. + + Transitional. :class:`~libtmux.common.tmux_cmd` has always exposed + ``stdout``/``stderr``/``cmd`` as lists, and callers rely on that in two ways + a plain tuple would break -- ``result.stdout == ["x"]``, and + ``isinstance(result.stderr, list)``, which gates whether libtmux raises on a + tmux error at all. Staying a list keeps both working while + :class:`CommandResult` becomes the single result type; comparing equal to a + tuple lets code already written against the engine API keep working too. + + Read-only, because a result describes something that already happened. + + Examples + -------- + >>> lines = Lines(["a", "b"]) + >>> lines == ["a", "b"], lines == ("a", "b"), ["a", "b"] == lines + (True, True, True) + >>> isinstance(lines, list) + True + >>> lines.append("c") + Traceback (most recent call last): + ... + TypeError: command output is read-only + """ + + __slots__ = () + + def __eq__(self, other: object) -> bool: + """Compare equal to any list or tuple with the same items.""" + if isinstance(other, (list, tuple)): + return tuple(self) == tuple(other) + return NotImplemented + + def __ne__(self, other: object) -> bool: + """Negate :meth:`__eq__`, preserving ``NotImplemented``.""" + result = self.__eq__(other) + if result is NotImplemented: + return NotImplemented + return not result + + def __hash__(self) -> int: # type: ignore[override] + """Hash as the tuple this will eventually be.""" + return hash(tuple(self)) + + def _read_only(self, *args: t.Any, **kwargs: t.Any) -> t.NoReturn: + """Reject every mutation.""" + msg = "command output is read-only" + raise TypeError(msg) + + append = extend = insert = remove = _read_only + pop = sort = reverse = clear = _read_only + __setitem__ = __delitem__ = __iadd__ = __imul__ = _read_only + + @dataclass(frozen=True) class CommandResult: """The structured outcome of executing a :class:`CommandRequest`. @@ -352,13 +407,13 @@ class CommandResult: Examples -------- >>> CommandResult(cmd=("tmux", "display-message", "-p", "hi"), stdout=("hi",)) - CommandResult(cmd=('tmux', 'display-message', '-p', 'hi'), stdout=('hi',), - stderr=(), returncode=0) + CommandResult(cmd=['tmux', 'display-message', '-p', 'hi'], stdout=['hi'], + stderr=[], returncode=0) """ - cmd: tuple[str, ...] - stdout: tuple[str, ...] = () - stderr: tuple[str, ...] = () + cmd: Sequence[str] + stdout: Sequence[str] = () + stderr: Sequence[str] = () returncode: int = 0 process: subprocess.Popen[str] | None = field( default=None, @@ -366,6 +421,13 @@ class CommandResult: repr=False, ) + def __post_init__(self) -> None: + """Normalize output to :class:`Lines`, whatever an engine passed in.""" + for name in ("cmd", "stdout", "stderr"): + value = getattr(self, name) + if not isinstance(value, Lines): + object.__setattr__(self, name, Lines(value)) + @property def ok(self) -> bool: """Whether tmux accepted the command. @@ -405,7 +467,7 @@ def raise_for_status(self) -> CommandResult: -------- >>> result = CommandResult(cmd=("tmux", "list-sessions"), stdout=("a",)) >>> result.raise_for_status().stdout - ('a',) + ['a'] The message names the tmux subcommand, not a connection flag: @@ -450,9 +512,9 @@ class TmuxEngine(t.Protocol): ... def run(self, request): ... return CommandResult(cmd=("tmux", *request.args), stdout=("ok",)) >>> EchoEngine().run(CommandRequest.from_args("list-sessions")).stdout - ('ok',) + ['ok'] >>> EchoEngine().run_batch([CommandRequest.from_args("list-sessions")]) - [CommandResult(cmd=('tmux', 'list-sessions'), stdout=('ok',), stderr=(), + [CommandResult(cmd=['tmux', 'list-sessions'], stdout=['ok'], stderr=[], returncode=0)] Duck typing works too, but then both methods are yours to write: @@ -519,7 +581,7 @@ class AsyncTmuxEngine(t.Protocol): ... batch = await engine.run_batch([CommandRequest.from_args("list-panes")]) ... return result.stdout, len(batch) >>> asyncio.run(main()) - (('ok',), 1) + (['ok'], 1) """ async def run(self, request: CommandRequest) -> CommandResult: diff --git a/src/libtmux/engines/record.py b/src/libtmux/engines/record.py index 44f212e72c..a00a147c58 100644 --- a/src/libtmux/engines/record.py +++ b/src/libtmux/engines/record.py @@ -84,7 +84,7 @@ class RecordingEngine(TmuxEngine): >>> recorder.tape[0].args ('display-message', '-p', 'recorded') >>> recorder.tape[0].result.stdout - ('recorded',) + ['recorded'] """ def __init__(self, inner: TmuxEngine) -> None: diff --git a/src/libtmux/engines/subprocess.py b/src/libtmux/engines/subprocess.py index 2c3bf792b5..fe9c88a168 100644 --- a/src/libtmux/engines/subprocess.py +++ b/src/libtmux/engines/subprocess.py @@ -40,7 +40,7 @@ class SubprocessEngine: >>> from libtmux.engines import CommandRequest, SubprocessEngine >>> engine = SubprocessEngine.for_server(server) >>> engine.run(CommandRequest.from_args("display-message", "-p", "hi")).stdout - ('hi',) + ['hi'] """ def __init__(self, connection: ServerConnection | None = None) -> None: diff --git a/tests/test_result_compat.py b/tests/test_result_compat.py new file mode 100644 index 0000000000..06def5d208 --- /dev/null +++ b/tests/test_result_compat.py @@ -0,0 +1,36 @@ +"""Tests written before the implementation exists.""" + +from __future__ import annotations + +import pytest + +from libtmux.engines import CommandResult + + +def test_output_is_a_list_so_error_paths_keep_raising() -> None: + """``isinstance(..., list)`` gates three error paths in src/; keep them live.""" + r = CommandResult(cmd=("tmux",), stderr=("boom",)) + assert isinstance(r.stderr, list) + assert isinstance(r.stdout, list) + assert isinstance(r.cmd, list) + + +def test_output_compares_equal_to_both_list_and_tuple() -> None: + r = CommandResult(cmd=("tmux",), stdout=("a", "b")) + assert r.stdout == ["a", "b"] + assert r.stdout == ("a", "b") # type: ignore[comparison-overlap] + assert r.stdout == ["a", "b"] + assert r.stdout != ["a", "z"] + + +def test_output_is_read_only() -> None: + r = CommandResult(cmd=("tmux",), stdout=("a",)) + with pytest.raises(TypeError): + r.stdout.append("b") # type: ignore[attr-defined] + + +def test_result_is_hashable_and_comparable() -> None: + a = CommandResult(cmd=("tmux",), stdout=("a",)) + b = CommandResult(cmd=("tmux",), stdout=("a",)) + assert a == b + assert len({a, b}) == 1 From 94d87bc91f4fdc9b7e565c85ec8ae8b1a9a85a1a Mon Sep 17 00:00:00 2001 From: Tony Narlock Date: Wed, 12 Aug 2026 17:41:26 -0500 Subject: [PATCH 08/32] Engines(feat): Resolve engines by name why: An application that picks its tmux transport from a config file or a CLI flag had to import the implementing class, so a packaged engine could not be selected without libtmux knowing about it. what: - Add create_engine/available_engines/register_engine/unregister_engine, with "subprocess" and "replay" registered - Read the libtmux.engines entry-point group on first use, not at import; scanning costs several ms against a ~50ms import and most programs never resolve by name - Skip a distribution whose engine fails to load rather than making every other engine unresolvable - Name the registered engines in the unknown-name error --- CHANGES | 7 ++ docs/api/libtmux.engines.md | 12 +++ src/libtmux/engines/__init__.py | 12 +++ src/libtmux/engines/registry.py | 176 ++++++++++++++++++++++++++++++++ tests/test_engine_registry.py | 64 ++++++++++++ 5 files changed, 271 insertions(+) create mode 100644 src/libtmux/engines/registry.py create mode 100644 tests/test_engine_registry.py diff --git a/CHANGES b/CHANGES index 92777591f2..86e7c371c9 100644 --- a/CHANGES +++ b/CHANGES @@ -125,6 +125,13 @@ engine fails closed, raising {exc}`~libtmux.exc.UnscriptedCommand` for a command it never recorded. The `recording_server` pytest fixture provides a server that records, which doubles as a spy over the tmux commands your code issued. +Engines resolve by name through {func}`~libtmux.engines.registry.create_engine`, +with {func}`~libtmux.engines.registry.available_engines` listing what is +registered. A packaged engine joins that list by advertising itself in the +`libtmux.engines` entry-point group, which is read on first use rather than at +import. A distribution whose engine fails to load is skipped rather than +breaking resolution for every other engine. + {meth}`Server.using() ` swaps the engine for a block and restores it afterwards, including when the block raises; scopes nest. {meth}`Server.recording() ` builds on it to record diff --git a/docs/api/libtmux.engines.md b/docs/api/libtmux.engines.md index f6a22d2ea6..55b9167dd8 100644 --- a/docs/api/libtmux.engines.md +++ b/docs/api/libtmux.engines.md @@ -62,3 +62,15 @@ single place either is computed. .. automodule:: libtmux.engines.record :members: ``` + +## Resolving an engine by name + +An application that reads its transport from a config file or a CLI flag can +name it instead of importing it. A third-party distribution adds a name by +advertising it in the `libtmux.engines` entry-point group; entry points are read +on first use, not at import. + +```{eval-rst} +.. automodule:: libtmux.engines.registry + :members: +``` diff --git a/src/libtmux/engines/__init__.py b/src/libtmux/engines/__init__.py index a1c914d8b7..7eeda2c8ea 100644 --- a/src/libtmux/engines/__init__.py +++ b/src/libtmux/engines/__init__.py @@ -59,9 +59,17 @@ ReplayEngine, Tape, ) +from libtmux.engines.registry import ( + ENGINE_ENTRY_POINT_GROUP, + available_engines, + create_engine, + register_engine, + unregister_engine, +) from libtmux.engines.subprocess import SubprocessEngine __all__ = ( + "ENGINE_ENTRY_POINT_GROUP", "AsyncTmuxEngine", "CommandRequest", "CommandResult", @@ -77,7 +85,11 @@ "SupportsTmuxVersion", "Tape", "TmuxEngine", + "available_engines", + "create_engine", "encode_direct_argv", "is_command_separator", + "register_engine", "split_direct_argv", + "unregister_engine", ) diff --git a/src/libtmux/engines/registry.py b/src/libtmux/engines/registry.py new file mode 100644 index 0000000000..5af1fb7f17 --- /dev/null +++ b/src/libtmux/engines/registry.py @@ -0,0 +1,176 @@ +"""Resolve engines by name, so a caller can pick one from configuration. + +An application that reads its tmux transport from a config file or a CLI flag +should not have to import the class that implements it. :func:`create_engine` +maps a name to a factory, and the ``libtmux.engines`` entry-point group lets a +third-party distribution add a name without libtmux knowing about it -- the same +shape tmuxp uses for workspace builders. + +Entry points are read on first use rather than at import. Scanning installed +distributions costs a few milliseconds, which is a meaningful fraction of +importing libtmux at all, and most programs never resolve an engine by name. +""" + +from __future__ import annotations + +import typing as t +from importlib import metadata + +from libtmux import exc +from libtmux.engines.record import ReplayEngine +from libtmux.engines.subprocess import SubprocessEngine + +if t.TYPE_CHECKING: + from libtmux.engines.base import TmuxEngine + +ENGINE_ENTRY_POINT_GROUP = "libtmux.engines" +"""Entry-point group a packaged engine registers under.""" + +EngineFactory = t.Callable[..., "TmuxEngine"] + +_registry: dict[str, EngineFactory] = {} +_entry_points_loaded = False + + +def register_engine(name: str, factory: EngineFactory) -> None: + """Register *factory* under *name*, replacing any previous registration. + + Parameters + ---------- + name : str + The name :func:`create_engine` will accept. + factory : Callable[..., TmuxEngine] + Called with whatever keyword arguments :func:`create_engine` is given. + An engine class is usually its own factory. + + Examples + -------- + >>> from libtmux.engines import CommandResult, available_engines, register_engine + >>> class NullEngine: + ... def run(self, request): + ... return CommandResult(cmd=("tmux", *request.args)) + ... def run_batch(self, requests): + ... return [self.run(r) for r in requests] + >>> register_engine("null-doc", NullEngine) + >>> "null-doc" in available_engines() + True + >>> unregister_engine("null-doc") + """ + _registry[name] = factory + + +def unregister_engine(name: str) -> None: + """Remove a registration, failing closed on an unknown name. + + Parameters + ---------- + name : str + A registered engine name. + + Raises + ------ + :exc:`~libtmux.exc.LibTmuxException` + Nothing is registered under *name*. + + Examples + -------- + >>> from libtmux.engines import unregister_engine + >>> unregister_engine("never-registered") + Traceback (most recent call last): + ... + libtmux.exc.LibTmuxException: unknown tmux engine 'never-registered' + (registered: ...) + """ + _load_entry_points() + if name not in _registry: + raise exc.LibTmuxException(_unknown_message(name)) + del _registry[name] + + +def _unknown_message(name: str) -> str: + """Build an error naming what the caller could have said instead.""" + known = ", ".join(available_engines()) or "none" + return f"unknown tmux engine {name!r} (registered: {known})" + + +def _load_entry_points() -> None: + """Read the entry-point group once, on first use. + + A distribution that advertises a broken engine should not make every other + engine unresolvable, so a failed load is skipped rather than raised. An + explicit :func:`register_engine` always wins over an entry point of the same + name. + """ + global _entry_points_loaded + if _entry_points_loaded: + return + _entry_points_loaded = True + for entry_point in metadata.entry_points(group=ENGINE_ENTRY_POINT_GROUP): + if entry_point.name in _registry: + continue + try: + _registry[entry_point.name] = entry_point.load() + except Exception: # noqa: BLE001 - a third party's import is not ours to trust + continue + + +def available_engines() -> tuple[str, ...]: + """Return every registered engine name, sorted. + + Returns + ------- + tuple[str, ...] + Built-in names plus any contributed through the entry-point group. + + Examples + -------- + >>> from libtmux.engines import available_engines + >>> "subprocess" in available_engines() + True + """ + _load_entry_points() + return tuple(sorted(_registry)) + + +def create_engine(name: str, **kwargs: t.Any) -> TmuxEngine: + """Build the engine registered under *name*. + + Parameters + ---------- + name : str + A name from :func:`available_engines`. + **kwargs : typing.Any + Passed through to the factory. + + Returns + ------- + :class:`~libtmux.engines.base.TmuxEngine` + A new engine. + + Raises + ------ + :exc:`~libtmux.exc.LibTmuxException` + No engine is registered under *name*. The message lists what is. + + Examples + -------- + >>> from libtmux.engines import SubprocessEngine, create_engine + >>> isinstance(create_engine("subprocess"), SubprocessEngine) + True + >>> create_engine("subprocess", server_args=("-Lwork",)).server_args + ('-Lwork',) + >>> create_engine("nope") + Traceback (most recent call last): + ... + libtmux.exc.LibTmuxException: unknown tmux engine 'nope' (registered: ...) + """ + _load_entry_points() + try: + factory = _registry[name] + except KeyError: + raise exc.LibTmuxException(_unknown_message(name)) from None + return factory(**kwargs) + + +register_engine("subprocess", SubprocessEngine.of) +register_engine("replay", ReplayEngine) diff --git a/tests/test_engine_registry.py b/tests/test_engine_registry.py new file mode 100644 index 0000000000..22c8831acf --- /dev/null +++ b/tests/test_engine_registry.py @@ -0,0 +1,64 @@ +"""Tests for name-based engine resolution.""" + +from __future__ import annotations + +import typing as t + +import pytest + +from libtmux import exc +from libtmux.engines import ( + CommandResult, + SubprocessEngine, + available_engines, + create_engine, + register_engine, +) + +if t.TYPE_CHECKING: + from libtmux.engines import CommandRequest + + +def test_builtin_engines_are_registered() -> None: + assert "subprocess" in available_engines() + assert isinstance(create_engine("subprocess"), SubprocessEngine) + + +def test_available_engines_is_sorted() -> None: + names = available_engines() + assert list(names) == sorted(names) + + +def test_unknown_engine_fails_closed_and_lists_options() -> None: + with pytest.raises(exc.LibTmuxException) as excinfo: + create_engine("does-not-exist") + message = str(excinfo.value) + assert "does-not-exist" in message + assert "subprocess" in message # names what you could have said + + +def test_factory_receives_kwargs() -> None: + engine = create_engine("subprocess", server_args=("-Lfromregistry",)) + assert engine.server_args == ("-Lfromregistry",) # type: ignore[attr-defined] + + +def test_third_party_can_register() -> None: + class Custom: + def run(self, request: CommandRequest) -> CommandResult: + return CommandResult(cmd=("tmux", *request.args)) + + def run_batch( + self, + requests: t.Sequence[CommandRequest], + ) -> list[CommandResult]: + return [self.run(r) for r in requests] + + register_engine("custom-for-test", Custom) + try: + assert "custom-for-test" in available_engines() + assert isinstance(create_engine("custom-for-test"), Custom) + finally: + from libtmux.engines.registry import unregister_engine + + unregister_engine("custom-for-test") + assert "custom-for-test" not in available_engines() From 78bfb63ef0bac2d3fba0351f261717ec05d0971c Mon Sep 17 00:00:00 2001 From: Tony Narlock Date: Wed, 12 Aug 2026 17:47:46 -0500 Subject: [PATCH 09/32] Engines(feat[cmd]): Return CommandResult from cmd() why: Two result types described the same thing. tmux_cmd could only ever wrap a subprocess, so it raised for .process on an engine that forks nothing, and its output could be mutated after the command had finished. what: - Server.cmd() and the Session/Window/Pane counterparts return CommandResult; stdout/stderr/cmd read as before and stay list-typed - Output is read-only, so mutating a finished command's result raises - .process is the Popen or None, rather than raising when absent - Widen Hooks.from_stdout to Sequence[str]; convert at the boundaries that promise list[str] - tmux_cmd is unchanged and still constructible directly - Document in MIGRATION and CHANGES --- CHANGES | 12 ++++++ MIGRATION | 59 ++++++++++++++++++++++++++++++ README.md | 2 +- src/libtmux/_internal/constants.py | 3 +- src/libtmux/common.py | 32 +++++++++++++--- src/libtmux/hooks.py | 2 +- src/libtmux/neo.py | 2 +- src/libtmux/options.py | 6 +-- src/libtmux/pane.py | 10 ++--- src/libtmux/server.py | 25 +++++++------ src/libtmux/session.py | 6 +-- src/libtmux/window.py | 7 ++-- tests/test_engines.py | 25 +++++-------- tests/test_hooks.py | 2 +- tests/test_session.py | 9 ++++- 15 files changed, 149 insertions(+), 53 deletions(-) diff --git a/CHANGES b/CHANGES index 86e7c371c9..ea99398a64 100644 --- a/CHANGES +++ b/CHANGES @@ -71,6 +71,18 @@ server.cmd( See {ref}`migration-0-63-command-separator`. +#### Commands return `CommandResult` + +{meth}`Server.cmd() ` and its `Session`, `Window`, and `Pane` +counterparts return {class}`~libtmux.engines.base.CommandResult` rather than +{class}`~libtmux.common.tmux_cmd`. `stdout`, `stderr`, `cmd`, and `returncode` +read exactly as before, and the sequences still compare equal to lists — they are +now read-only, so a caller that mutated a result in place gets +{exc}`TypeError`. {attr}`~libtmux.engines.base.CommandResult.ok` and +{meth}`~libtmux.engines.base.CommandResult.raise_for_status` come with it. +`tmux_cmd` is unchanged and still constructible directly. See +{ref}`migration-0-63-command-result`. + #### `tmux_cmd.process` deprecated {attr}`libtmux.common.tmux_cmd.process` is now a deprecated property. Reading it diff --git a/MIGRATION b/MIGRATION index 6ad083b023..209bcd4c51 100644 --- a/MIGRATION +++ b/MIGRATION @@ -113,6 +113,65 @@ sections below for detailed migration examples and code samples. _Detailed migration steps for the next version will be posted here._ +(migration-0-63-command-result)= + +## Commands return `CommandResult` + +{meth}`Server.cmd() ` and its `Session`, `Window`, and +`Pane` counterparts now return +{class}`~libtmux.engines.base.CommandResult` instead of +{class}`~libtmux.common.tmux_cmd`. The attributes you read are unchanged: + +```python +proc = server.cmd("list-sessions") +proc.stdout # list[str], as before +proc.stderr # list[str], as before +proc.returncode # int, as before +proc.cmd # list[str], as before +``` + +`stdout`, `stderr`, and `cmd` are lists, compare equal to lists exactly as +before, and additionally compare equal to tuples. They are read-only: code that +mutated a result in place -- appending to `proc.stdout`, sorting it -- now +raises {exc}`TypeError`. Read it, copy it with `list(...)`, but do not edit it. + +Two additions come with the change: + +```python +proc.ok # True when tmux exited zero +proc.raise_for_status() # raise LibTmuxException on failure, else return self +``` + +### `.process` reports absence instead of raising + +`tmux_cmd.process` could only ever be a {class}`subprocess.Popen`, so it raised +when there was none. A `CommandResult` describes any engine, including ones that +fork nothing, so `.process` is now simply `None` in that case: + +```python +# Before +proc.process # DeprecationWarning; raised under a non-subprocess engine + +# After +proc.process # the Popen, or None +``` + +Prefer `returncode`, `stdout`, and `stderr`; `.process` exists for the rare +caller that needs the OS process and knows it is using the default engine. + +### `tmux_cmd` still works + +{class}`~libtmux.common.tmux_cmd` is unchanged and still constructible, so code +that builds one directly keeps working: + +```python +from libtmux.common import tmux_cmd + +proc = tmux_cmd(f"-L{server.socket_name}", "list-sessions") +``` + +Only what `Server.cmd()` *returns* changed. + (migration-0-63-command-separator)= ## Pluggable engines: separators, and `tmux_cmd.process` diff --git a/README.md b/README.md index 82594dcaaa..c72c472bc6 100644 --- a/README.md +++ b/README.md @@ -127,7 +127,7 @@ Every object has a `.cmd()` escape hatch that honors socket name and path: ```python >>> server = Server(socket_name='libtmux_doctest') >>> server.cmd('display-message', 'hello world') - +CommandResult(cmd=[...], ...) ``` Create a new session: diff --git a/src/libtmux/_internal/constants.py b/src/libtmux/_internal/constants.py index df4d9843f2..6bd0b02ac4 100644 --- a/src/libtmux/_internal/constants.py +++ b/src/libtmux/_internal/constants.py @@ -4,6 +4,7 @@ import io import typing as t +from collections.abc import Sequence from dataclasses import dataclass, field from libtmux._internal.dataclasses import SkipDefaultFieldsReprMixin @@ -1094,7 +1095,7 @@ class Hooks( command_error: SparseArray[str] = field(default_factory=SparseArray) @classmethod - def from_stdout(cls, value: list[str]) -> Hooks: + def from_stdout(cls, value: Sequence[str]) -> Hooks: """Parse raw tmux hook output into a Hooks instance. The parsing pipeline: diff --git a/src/libtmux/common.py b/src/libtmux/common.py index 648b5233b4..afc7eefdde 100644 --- a/src/libtmux/common.py +++ b/src/libtmux/common.py @@ -17,7 +17,12 @@ from . import exc from ._compat import LooseVersion -from .engines.base import CommandRequest, SupportsCommandLine, split_direct_argv +from .engines.base import ( + CommandRequest, + CommandResult, + SupportsCommandLine, + split_direct_argv, +) from .engines.subprocess import SubprocessEngine if t.TYPE_CHECKING: @@ -44,7 +49,7 @@ class CmdProtocol(t.Protocol): """Command protocol for tmux command.""" - def __call__(self, cmd: str, *args: t.Any, **kwargs: t.Any) -> tmux_cmd: + def __call__(self, cmd: str, *args: t.Any, **kwargs: t.Any) -> CommandResult: """Wrap tmux_cmd.""" ... @@ -60,7 +65,7 @@ class EnvironmentMixin: _add_option = None - cmd: Callable[[t.Any, t.Any], tmux_cmd] + cmd: Callable[[t.Any, t.Any], CommandResult] def __init__(self, add_option: str | None = None) -> None: self._add_option = add_option @@ -246,7 +251,7 @@ def getenv(self, name: str) -> str | bool | None: return opts_dict.get(name) -def raise_if_stderr(proc: tmux_cmd, subcommand: str) -> None: +def raise_if_stderr(proc: CommandResult, subcommand: str) -> None: """Raise :exc:`LibTmuxException` tagged with the tmux subcommand on stderr. Centralizes the ``if proc.stderr: raise exc.LibTmuxException(proc.stderr)`` @@ -396,6 +401,21 @@ def __init__( }, ) + @property + def result(self) -> CommandResult: + """Return this command's outcome as a :class:`CommandResult`. + + Carries the ``has-session`` stdout adaptation already applied, so the + engine-native result and this wrapper never disagree. + """ + return CommandResult( + cmd=self.cmd, + stdout=self.stdout, + stderr=self.stderr, + returncode=self.returncode, + process=self._process, + ) + @property def ok(self) -> bool: """Whether tmux accepted the command. @@ -464,7 +484,9 @@ def process(self) -> subprocess.Popen[str]: Examples -------- >>> import warnings - >>> proc = server.cmd("display-message", "-p", "hi") + >>> proc = tmux_cmd( + ... f"-L{server.socket_name}", "display-message", "-p", "hi" + ... ) >>> with warnings.catch_warnings(record=True) as caught: ... warnings.simplefilter("always") ... returncode = proc.process.returncode diff --git a/src/libtmux/hooks.py b/src/libtmux/hooks.py index d9ad24d407..267aabb423 100644 --- a/src/libtmux/hooks.py +++ b/src/libtmux/hooks.py @@ -366,7 +366,7 @@ def _show_hook( if len(cmd.stderr): handle_option_error(cmd.stderr[0]) - return cmd.stdout + return list(cmd.stdout) def show_hook( self, diff --git a/src/libtmux/neo.py b/src/libtmux/neo.py index d5567d0873..32d087d34a 100644 --- a/src/libtmux/neo.py +++ b/src/libtmux/neo.py @@ -1153,7 +1153,7 @@ def fetch_objs( proc = tmux_cmd(*tmux_cmds, engine=server.engine) - raise_if_stderr(proc, list_cmd) + raise_if_stderr(proc.result, list_cmd) outputs = [parse_output(line, list_cmd, tmux_version) for line in proc.stdout] diff --git a/src/libtmux/options.py b/src/libtmux/options.py index ff8bb2239d..96cb55bb77 100644 --- a/src/libtmux/options.py +++ b/src/libtmux/options.py @@ -82,6 +82,7 @@ OptionScope, _DefaultOptionScope, ) +from libtmux.engines.base import CommandResult from . import exc @@ -91,7 +92,6 @@ from typing_extensions import Self from libtmux._internal.constants import TerminalFeatures - from libtmux.common import tmux_cmd TerminalOverride = dict[str, str | None] @@ -809,7 +809,7 @@ def _show_options_raw( include_inherited: bool | None = None, quiet: bool | None = None, values_only: bool | None = None, - ) -> tmux_cmd: + ) -> CommandResult: """Return a dict of options for the target. Parameters @@ -1051,7 +1051,7 @@ def _show_option_raw( ignore_errors: bool | None = None, include_hooks: bool | None = None, include_inherited: bool | None = None, - ) -> tmux_cmd: + ) -> CommandResult: """Return raw option output for target. Parameters diff --git a/src/libtmux/pane.py b/src/libtmux/pane.py index 69215a9563..78ae49262a 100644 --- a/src/libtmux/pane.py +++ b/src/libtmux/pane.py @@ -15,7 +15,7 @@ from libtmux import exc from libtmux._internal.env import pane_id_from_env -from libtmux.common import get_version_str, has_gte_version, raise_if_stderr, tmux_cmd +from libtmux.common import get_version_str, has_gte_version, raise_if_stderr from libtmux.constants import ( PANE_DIRECTION_FLAG_MAP, RESIZE_ADJUSTMENT_DIRECTION_FLAG_MAP, @@ -23,7 +23,7 @@ PaneDirection, ResizeAdjustmentDirection, ) -from libtmux.engines.base import CommandSeparator +from libtmux.engines.base import CommandResult, CommandSeparator from libtmux.formats import FORMAT_SEPARATOR from libtmux.hooks import HooksMixin from libtmux.neo import Obj, fetch_obj @@ -312,7 +312,7 @@ def cmd( cmd: str, *args: t.Any, target: str | int | None = None, - ) -> tmux_cmd: + ) -> CommandResult: """Execute tmux subcommand within pane context. Automatically binds target by adding ``-t`` for object's pane ID to the @@ -690,7 +690,7 @@ def capture_pane( proc = self.cmd(*cmd) if to_buffer is not None: return None - return proc.stdout + return list(proc.stdout) def send_keys( self, @@ -1012,7 +1012,7 @@ def display_message( ) if get_text: - return proc.stdout + return list(proc.stdout) return None diff --git a/src/libtmux/server.py b/src/libtmux/server.py index 105f5a374e..6082e8b561 100644 --- a/src/libtmux/server.py +++ b/src/libtmux/server.py @@ -24,6 +24,7 @@ from libtmux.constants import OptionScope from libtmux.engines.base import ( CommandRequest, + CommandResult, SupportsConnection, TmuxEngine, ) @@ -583,13 +584,13 @@ def cmd( cmd: str, *args: t.Any, target: str | int | None = None, - ) -> tmux_cmd: + ) -> CommandResult: """Execute tmux command respective of socket name and file, return output. Examples -------- >>> server.cmd('display-message', 'hi') - + CommandResult(cmd=[...], stdout=[], stderr=[], returncode=0) New session: @@ -639,7 +640,7 @@ def cmd( ["-t", str(target), *args] if target is not None else [*args] ) - return tmux_cmd(cmd, *cmd_args, engine=self.engine) + return tmux_cmd(cmd, *cmd_args, engine=self.engine).result @property def attached_sessions(self) -> list[Session]: @@ -846,7 +847,7 @@ def run_shell( if background: return None - return proc.stdout + return list(proc.stdout) def wait_for( self, @@ -1027,7 +1028,7 @@ def list_keys( raise_if_stderr(proc, "list-keys") - return proc.stdout + return list(proc.stdout) def list_commands(self, *, command_name: str | None = None) -> list[str]: """List tmux commands via ``$ tmux list-commands``. @@ -1057,7 +1058,7 @@ def list_commands(self, *, command_name: str | None = None) -> list[str]: raise_if_stderr(proc, "list-commands") - return proc.stdout + return list(proc.stdout) def lock_server(self) -> None: """Lock the tmux server via ``$ tmux lock-server``. @@ -1143,7 +1144,7 @@ def server_access( raise_if_stderr(proc, "server-access") if list_access: - return proc.stdout + return list(proc.stdout) return None def refresh_client( @@ -1784,7 +1785,7 @@ def show_messages( raise_if_stderr(proc, "show-messages") - return proc.stdout + return list(proc.stdout) @t.overload def display_message( @@ -1957,7 +1958,7 @@ def display_message( ) if get_text: - return proc.stdout + return list(proc.stdout) return None @@ -2002,7 +2003,7 @@ def show_prompt_history( raise_if_stderr(proc, "show-prompt-history") - return proc.stdout + return list(proc.stdout) def clear_prompt_history( self, @@ -2280,7 +2281,7 @@ def list_buffers( raise_if_stderr(proc, "list-buffers") - return proc.stdout + return list(proc.stdout) def if_shell( self, @@ -2391,7 +2392,7 @@ def list_clients(self) -> list[str]: raise_if_stderr(proc, "list-clients") - return proc.stdout + return list(proc.stdout) def switch_client(self, target_session: str) -> None: """Switch tmux client. diff --git a/src/libtmux/session.py b/src/libtmux/session.py index 4277052a37..148e1ec382 100644 --- a/src/libtmux/session.py +++ b/src/libtmux/session.py @@ -14,8 +14,9 @@ import warnings from libtmux._internal.query_list import QueryList -from libtmux.common import has_gte_version, raise_if_stderr, tmux_cmd +from libtmux.common import has_gte_version, raise_if_stderr from libtmux.constants import WINDOW_DIRECTION_FLAG_MAP, OptionScope, WindowDirection +from libtmux.engines.base import CommandResult from libtmux.formats import FORMAT_SEPARATOR from libtmux.hooks import HooksMixin from libtmux.neo import Obj, fetch_obj, fetch_objs @@ -35,7 +36,6 @@ import types from libtmux._internal.types import StrPath - from libtmux.common import tmux_cmd if sys.version_info >= (3, 11): from typing import Self @@ -418,7 +418,7 @@ def cmd( cmd: str, *args: t.Any, target: str | int | None = None, - ) -> tmux_cmd: + ) -> CommandResult: """Execute tmux subcommand within session context. Automatically binds target by adding ``-t`` for object's session ID to the diff --git a/src/libtmux/window.py b/src/libtmux/window.py index b57db99692..8159642040 100644 --- a/src/libtmux/window.py +++ b/src/libtmux/window.py @@ -15,7 +15,7 @@ import warnings from libtmux._internal.query_list import QueryList -from libtmux.common import has_gte_version, raise_if_stderr, tmux_cmd +from libtmux.common import has_gte_version, raise_if_stderr from libtmux.constants import ( RESIZE_ADJUSTMENT_DIRECTION_FLAG_MAP, OptionScope, @@ -23,6 +23,7 @@ ResizeAdjustmentDirection, WindowDirection, ) +from libtmux.engines.base import CommandResult from libtmux.hooks import HooksMixin from libtmux.neo import Obj, fetch_obj, fetch_objs from libtmux.pane import Pane @@ -463,7 +464,7 @@ def cmd( cmd: str, *args: t.Any, target: str | int | None = None, - ) -> tmux_cmd: + ) -> CommandResult: """Execute tmux subcommand within window context. Automatically binds target by adding ``-t`` for object's window ID to the @@ -1353,7 +1354,7 @@ def display_message( ) if get_text: - return proc.stdout + return list(proc.stdout) return None diff --git a/tests/test_engines.py b/tests/test_engines.py index 8076748f47..4234423462 100644 --- a/tests/test_engines.py +++ b/tests/test_engines.py @@ -5,7 +5,6 @@ import json import subprocess import typing as t -import warnings import pytest @@ -97,25 +96,21 @@ def test_injected_engine_receives_target_flag() -> None: assert engine.requests[0].args == ("kill-window", "-t", "@3") -def test_process_raises_on_engine_without_subprocess() -> None: - """``.process`` is unavailable when no OS process was forked.""" - server = Server(socket_name="canned_process", engine=CannedEngine()) - proc = server.cmd("list-sessions") +def test_process_is_none_on_engine_without_subprocess() -> None: + """``.process`` is ``None`` when no OS process was forked. - with warnings.catch_warnings(record=True) as caught: - warnings.simplefilter("always") - with pytest.raises(exc.LibTmuxException): - _ = proc.process + ``tmux_cmd`` raised here, because it could only ever wrap a subprocess. + A :class:`~libtmux.engines.base.CommandResult` describes any engine, so it + reports the absence rather than treating it as an error. + """ + server = Server(socket_name="canned_process", engine=CannedEngine()) - assert any(issubclass(entry.category, DeprecationWarning) for entry in caught) + assert server.cmd("list-sessions").process is None def test_process_is_popen_under_default_engine(session: Session) -> None: - """``.process`` still resolves to the Popen, with a DeprecationWarning.""" - proc = session.server.cmd("display-message", "-p", "hi") - - with pytest.deprecated_call(): - process = proc.process + """``.process`` carries the Popen the default engine forked.""" + process = session.server.cmd("display-message", "-p", "hi").process assert isinstance(process, subprocess.Popen) assert process.returncode == 0 diff --git a/tests/test_hooks.py b/tests/test_hooks.py index 4e03f0a26b..ca626cc726 100644 --- a/tests/test_hooks.py +++ b/tests/test_hooks.py @@ -260,7 +260,7 @@ def test_hooks_dataclass( == "set-option -g status-left-style bg=blue" ) - hooks = Hooks.from_stdout(global_out + session_out + window_out + pane_out) + hooks = Hooks.from_stdout([*global_out, *session_out, *window_out, *pane_out]) assert hooks.session_renamed.as_list() == [ "set-option -g status-left-style bg=red", diff --git a/tests/test_session.py b/tests/test_session.py index f7d95e4cca..942b5de346 100644 --- a/tests/test_session.py +++ b/tests/test_session.py @@ -543,7 +543,8 @@ def test_session_attach_does_not_fail_if_session_killed_during_attach( 2. Session state can change arbitrarily while the user is attached 3. Refreshing after such a command makes no semantic sense """ - from libtmux.common import tmux_cmd + from libtmux.common import tmux_cmd # noqa: F401 + from libtmux.engines.base import CommandResult # Create a new session specifically for this test test_session = server.new_session(detach=True) @@ -558,7 +559,11 @@ def __init__(self) -> None: self.stderr: list[str] = [] self.cmd: list[str] = ["tmux", "attach-session"] - def patched_cmd(cmd_name: str, *args: t.Any, **kwargs: t.Any) -> tmux_cmd: + def patched_cmd( + cmd_name: str, + *args: t.Any, + **kwargs: t.Any, + ) -> CommandResult: """Patched cmd that kills session after attach-session.""" if cmd_name == "attach-session": # Simulate: attach-session succeeded, user worked, then killed session From 8e3312f173dc843f0ee1102a8e5ac21efb5f94c3 Mon Sep 17 00:00:00 2001 From: Tony Narlock Date: Wed, 12 Aug 2026 17:52:07 -0500 Subject: [PATCH 10/32] Engines(refactor[dispatch]): Take tmux_cmd off the dispatch path why: Returning CommandResult from cmd() left every command building a deprecated tmux_cmd just to read a result off it, so the back-compat class sat on the hot path of the thing replacing it. what: - Add common.dispatch(): the single path from an engine to a result, owning the debug logging and tmux's has-session stdout quirk - Route Server.cmd() and neo.fetch_objs() straight through it - Rebuild tmux_cmd on top of it, so it is a leaf nothing depends on - Drop the tmux_cmd.result property, which existed only to bridge the two and is now unreferenced --- src/libtmux/common.py | 140 +++++++++++++++++++++++++---------------- src/libtmux/neo.py | 6 +- src/libtmux/server.py | 9 ++- tests/test_dispatch.py | 75 ++++++++++++++++++++++ 4 files changed, 170 insertions(+), 60 deletions(-) create mode 100644 tests/test_dispatch.py diff --git a/src/libtmux/common.py b/src/libtmux/common.py index afc7eefdde..461a3b8206 100644 --- a/src/libtmux/common.py +++ b/src/libtmux/common.py @@ -7,6 +7,7 @@ from __future__ import annotations +import dataclasses import functools import logging import re @@ -289,6 +290,88 @@ def raise_if_stderr(proc: CommandResult, subcommand: str) -> None: ) +def dispatch( + engine: TmuxEngine, + *args: t.Any, + tmux_bin: str | None = None, +) -> CommandResult: + """Run one tmux command through *engine* and adapt its result. + + The single dispatch path every wrapper uses. Two things happen here rather + than in an engine, so that every engine stays a plain executor: the debug + logging that names the command line before and after it runs, and tmux's + ``has-session`` quirk. + + tmux answers ``has-session`` on stderr, while libtmux has always reported it + on stdout. Adapting it here keeps that promise for whichever engine ran the + command. + + Parameters + ---------- + engine : TmuxEngine + The executor. + *args : typing.Any + The tmux subcommand and its arguments, stringified. + tmux_bin : str, optional + Override the tmux binary for this one command. + + Returns + ------- + CommandResult + The adapted result. + + Examples + -------- + >>> from libtmux.engines import SubprocessEngine + >>> engine = SubprocessEngine.for_server(server) + >>> dispatch(engine, "display-message", "-p", "hi").stdout + ['hi'] + + ``has-session`` reports on stdout, as it always has: + + >>> dispatch(engine, "has-session", "-t", "nope").stdout # doctest: +ELLIPSIS + ["can't find session: nope"] + """ + request = CommandRequest.from_args(*args, tmux_bin=tmux_bin) + + if logger.isEnabledFor(logging.DEBUG): + logger.debug( + "tmux command dispatched", + extra={ + "tmux_cmd": shlex.join( + engine.command_line(request) + if isinstance(engine, SupportsCommandLine) + else request.args, + ), + "tmux_subcommand": request.subcommand, + }, + ) + + result = engine.run(request) + + cmd = list(result.cmd) + stderr = list(result.stderr) + stdout = list(result.stdout) + if "has-session" in cmd and stderr and not stdout: + stdout = [stderr[0]] + result = dataclasses.replace(result, stdout=stdout) + + if logger.isEnabledFor(logging.DEBUG): + logger.debug( + "tmux command completed", + extra={ + "tmux_cmd": shlex.join(cmd), + "tmux_subcommand": request.subcommand, + "tmux_exit_code": result.returncode, + "tmux_stdout": stdout[:100], + "tmux_stderr": stderr[:100], + "tmux_stdout_len": len(stdout), + "tmux_stderr_len": len(stderr), + }, + ) + return result + + class tmux_cmd: """Run any :term:`tmux(1)` command, returning list-shaped output. @@ -355,67 +438,14 @@ def __init__( runner: TmuxEngine = ( engine if engine is not None else SubprocessEngine.of(tmux_bin) ) - request = CommandRequest.from_args(*args) - - if logger.isEnabledFor(logging.DEBUG): - logger.debug( - "tmux command dispatched", - extra={ - "tmux_cmd": shlex.join( - runner.command_line(request) - if isinstance(runner, SupportsCommandLine) - else request.args, - ), - "tmux_subcommand": request.subcommand, - }, - ) - - result = runner.run(request) + result = dispatch(runner, *args) self.cmd = list(result.cmd) self.returncode = result.returncode + self.stdout = list(result.stdout) self.stderr = list(result.stderr) self._process = result.process - # tmux writes ``has-session``'s answer to stderr; the wrappers have - # always read it off stdout. Adapted here, not in an engine, so every - # engine stays a plain executor. - stdout = list(result.stdout) - self.stdout = ( - [self.stderr[0]] - if "has-session" in self.cmd and self.stderr and not stdout - else stdout - ) - - if logger.isEnabledFor(logging.DEBUG): - logger.debug( - "tmux command completed", - extra={ - "tmux_cmd": shlex.join(self.cmd), - "tmux_subcommand": request.subcommand, - "tmux_exit_code": self.returncode, - "tmux_stdout": self.stdout[:100], - "tmux_stderr": self.stderr[:100], - "tmux_stdout_len": len(self.stdout), - "tmux_stderr_len": len(self.stderr), - }, - ) - - @property - def result(self) -> CommandResult: - """Return this command's outcome as a :class:`CommandResult`. - - Carries the ``has-session`` stdout adaptation already applied, so the - engine-native result and this wrapper never disagree. - """ - return CommandResult( - cmd=self.cmd, - stdout=self.stdout, - stderr=self.stderr, - returncode=self.returncode, - process=self._process, - ) - @property def ok(self) -> bool: """Whether tmux accepted the command. diff --git a/src/libtmux/neo.py b/src/libtmux/neo.py index 32d087d34a..e69fe7035c 100644 --- a/src/libtmux/neo.py +++ b/src/libtmux/neo.py @@ -11,7 +11,7 @@ from libtmux import exc from libtmux._compat import LooseVersion -from libtmux.common import get_version, raise_if_stderr, tmux_cmd +from libtmux.common import dispatch, get_version, raise_if_stderr from libtmux.engines.base import SupportsTmuxVersion from libtmux.formats import FORMAT_SEPARATOR @@ -1151,9 +1151,9 @@ def fetch_objs( }, ) - proc = tmux_cmd(*tmux_cmds, engine=server.engine) + proc = dispatch(server.engine, *tmux_cmds) - raise_if_stderr(proc.result, list_cmd) + raise_if_stderr(proc, list_cmd) outputs = [parse_output(line, list_cmd, tmux_version) for line in proc.stdout] diff --git a/src/libtmux/server.py b/src/libtmux/server.py index 6082e8b561..8327c7fc31 100644 --- a/src/libtmux/server.py +++ b/src/libtmux/server.py @@ -20,7 +20,12 @@ from libtmux._internal.env import socket_path_from_env from libtmux._internal.query_list import QueryList from libtmux.client import Client -from libtmux.common import get_version, has_gte_version, raise_if_stderr, tmux_cmd +from libtmux.common import ( + dispatch, + get_version, + has_gte_version, + raise_if_stderr, +) from libtmux.constants import OptionScope from libtmux.engines.base import ( CommandRequest, @@ -640,7 +645,7 @@ def cmd( ["-t", str(target), *args] if target is not None else [*args] ) - return tmux_cmd(cmd, *cmd_args, engine=self.engine).result + return dispatch(self.engine, cmd, *cmd_args) @property def attached_sessions(self) -> list[Session]: diff --git a/tests/test_dispatch.py b/tests/test_dispatch.py new file mode 100644 index 0000000000..21abc6e6d5 --- /dev/null +++ b/tests/test_dispatch.py @@ -0,0 +1,75 @@ +"""The deprecated wrapper must not sit on the dispatch path.""" + +from __future__ import annotations + +import typing as t + +from libtmux.common import dispatch, tmux_cmd +from libtmux.engines import CommandResult +from libtmux.server import Server + +if t.TYPE_CHECKING: + from collections.abc import Sequence + + from libtmux.engines import CommandRequest + from libtmux.session import Session + + +class CountingEngine: + """Count dispatches so a test can prove how many objects were built.""" + + def __init__(self, stdout: Sequence[str] = ()) -> None: + self.calls = 0 + self._stdout = tuple(stdout) + + def run(self, request: CommandRequest) -> CommandResult: + self.calls += 1 + return CommandResult(cmd=("tmux", *request.args), stdout=self._stdout) + + def run_batch(self, requests: Sequence[CommandRequest]) -> list[CommandResult]: + return [self.run(r) for r in requests] + + +def test_server_cmd_does_not_build_a_tmux_cmd(monkeypatch) -> None: # type: ignore[no-untyped-def] + """Dispatch goes engine-direct; the back-compat class is never constructed.""" + built = 0 + original = tmux_cmd.__init__ + + def counting_init(self: tmux_cmd, *args: t.Any, **kwargs: t.Any) -> None: + nonlocal built + built += 1 + original(self, *args, **kwargs) + + monkeypatch.setattr(tmux_cmd, "__init__", counting_init) + + engine = CountingEngine(stdout=("ok",)) + result = Server(socket_name="hotpath", engine=engine).cmd("list-sessions") + + assert result.stdout == ["ok"] + assert engine.calls == 1 + assert built == 0 + + +def test_dispatch_applies_the_has_session_adaptation() -> None: + """Tmux answers has-session on stderr; libtmux has always read it on stdout.""" + engine = CountingEngine() + + def run(request: CommandRequest) -> CommandResult: + return CommandResult( + cmd=("tmux", *request.args), + stderr=("can't find session: nope",), + returncode=1, + ) + + engine.run = run # type: ignore[method-assign] + result = dispatch(engine, "has-session", "-t", "nope") + + assert result.stdout == ["can't find session: nope"] + + +def test_tmux_cmd_still_works_standalone(session: Session) -> None: + """The back-compat class keeps its own behavior, built on the same helper.""" + proc = tmux_cmd(f"-L{session.server.socket_name}", "display-message", "-p", "hi") + + assert proc.stdout == ["hi"] + assert proc.returncode == 0 From 32c1c2f91d71d4628adc63fc30c5c96c2ec4324e Mon Sep 17 00:00:00 2001 From: Tony Narlock Date: Wed, 12 Aug 2026 17:56:49 -0500 Subject: [PATCH 11/32] Engines(feat): Ship the control-mode argv codecs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit why: The seam exists so a transport other than fork-per-command can be plugged in, but nothing proved one could. A persistent `tmux -C` engine needed one helper core did not expose, so the claim was untested. what: - Add render_control_line() and unescape_control_output(): encode a command for tmux's line-oriented control parser, and decode a %output payload back to the bytes a pane wrote - Add a worked example holding one long-lived tmux -C connection, and assert the object API traverses through it — listing queries included --- CHANGES | 10 ++ src/libtmux/engines/__init__.py | 4 + src/libtmux/engines/base.py | 73 +++++++++ .../engines/test_control_mode_engine.py | 142 ++++++++++++++++++ 4 files changed, 229 insertions(+) create mode 100644 tests/examples/engines/test_control_mode_engine.py diff --git a/CHANGES b/CHANGES index ea99398a64..51ac5bbe20 100644 --- a/CHANGES +++ b/CHANGES @@ -126,6 +126,16 @@ lookup instead of re-walking `$PATH` for every command. See {ref}`engines` for the guide and {ref}`engines-api` for the reference. +#### A persistent-connection engine is buildable on the public API + +{func}`~libtmux.engines.base.render_control_line` and +{func}`~libtmux.engines.base.unescape_control_output` encode a command for +tmux's line-oriented control parser and decode a ``%output`` payload back to the +bytes a pane wrote. With them, an engine that holds one long-lived ``tmux -C`` +connection can be written entirely against the public engine API — including the +format-heavy listing queries the object API depends on. See +`tests/examples/engines/test_control_mode_engine.py` for a worked example. + #### Testing without a tmux server {class}`~libtmux.engines.record.RecordingEngine` wraps a real engine and keeps diff --git a/src/libtmux/engines/__init__.py b/src/libtmux/engines/__init__.py index 7eeda2c8ea..ba2fae8ff2 100644 --- a/src/libtmux/engines/__init__.py +++ b/src/libtmux/engines/__init__.py @@ -50,7 +50,9 @@ TmuxEngine, encode_direct_argv, is_command_separator, + render_control_line, split_direct_argv, + unescape_control_output, ) from libtmux.engines.connection import ServerConnection from libtmux.engines.record import ( @@ -90,6 +92,8 @@ "encode_direct_argv", "is_command_separator", "register_engine", + "render_control_line", "split_direct_argv", + "unescape_control_output", "unregister_engine", ) diff --git a/src/libtmux/engines/base.py b/src/libtmux/engines/base.py index cc3e9d06bd..032ff3aa10 100644 --- a/src/libtmux/engines/base.py +++ b/src/libtmux/engines/base.py @@ -14,6 +14,8 @@ from __future__ import annotations +import re +import shlex import typing as t from collections.abc import Sequence from dataclasses import dataclass, field @@ -33,6 +35,10 @@ ) +#: tmux escapes a byte in ``%output`` as a backslash plus three octal digits. +_CONTROL_OCTAL = re.compile(rb"\\([0-7]{3})") + + class CommandSeparator(str): """A caller-authored command boundary, distinct from a literal ``";"``. @@ -229,6 +235,73 @@ def encode_direct_argv(argv: Sequence[str]) -> tuple[str, ...]: return (*direct.global_args, *_encode_command_argv(direct.command_argv)) +def _quote_control_token(token: str) -> str: + r"""Quote one literal token for tmux's line-oriented control parser.""" + if "\0" in token: + msg = "tmux command arguments cannot contain NUL" + raise ValueError(msg) + if "\n" in token or "\r" in token: + return "".join(f"\\{byte:03o}" for byte in token.encode()) + return shlex.quote(token) + + +def render_control_line(argv: Sequence[str]) -> str: + r"""Render a tmux argv as a control-mode (``tmux -C``) command line. + + Literal tokens are quoted for the control parser. Tokens containing a line + delimiter are UTF-8 octal encoded so one request remains one physical line. + Only a :class:`CommandSeparator` is left bare. + + Examples + -------- + >>> render_control_line(("rename-window", "-t", "@1", "a b")) + "rename-window -t @1 'a b'" + >>> render_control_line( + ... ("rename-window", "a", CommandSeparator(";"), "kill-window", "@2") + ... ) + 'rename-window a ; kill-window @2' + >>> "\n" not in render_control_line(("display-message", "first\nsecond")) + True + """ + return " ".join( + str(token) if is_command_separator(token) else _quote_control_token(token) + for token in argv + ) + + +def unescape_control_output(payload: str) -> bytes: + r"""Decode a control-mode ``%output`` payload back to the bytes the pane wrote. + + tmux does not forward pane output verbatim: in a ``%output`` notification it + writes every non-printable byte -- and the backslash itself -- as a backslash + followed by three octal digits. A reader that scans for raw bytes must undo + this first, or it can never match: an ``ESC`` (``0x1b``) arrives on the wire + as the four *characters* ``\``, ``0``, ``3``, ``3``. + + Bytes tmux left alone pass through untouched, so feeding this an already-raw + payload is harmless. + + Examples + -------- + Printable output is returned as-is: + + >>> unescape_control_output("hello world") + b'hello world' + + An escape sequence tmux octal-escaped comes back as real bytes: + + >>> unescape_control_output(r"\033]3008;state=idle\033\134") + b'\x1b]3008;state=idle\x1b\\' + + Multi-byte UTF-8 survives the round trip: + + >>> unescape_control_output(r"caf\303\251").decode() + 'café' + """ + raw = payload.encode("utf-8", "surrogateescape") + return _CONTROL_OCTAL.sub(lambda m: bytes((int(m.group(1), 8),)), raw) + + @dataclass(frozen=True) class CommandRequest: """A tmux command, ready for an engine to execute. diff --git a/tests/examples/engines/test_control_mode_engine.py b/tests/examples/engines/test_control_mode_engine.py new file mode 100644 index 0000000000..58f3b81458 --- /dev/null +++ b/tests/examples/engines/test_control_mode_engine.py @@ -0,0 +1,142 @@ +"""Drive libtmux over a persistent ``tmux -C`` connection. + +The engine seam exists so a transport other than "fork the tmux binary once per +command" can be plugged in. This is the proof: a control-mode engine holds one +long-lived ``tmux -C`` process and writes command lines to it, and the whole +object API -- including the format-heavy listing queries -- works through it +unchanged. + +It is deliberately minimal. A production control-mode engine also handles +notifications, reconnection, and pipelining a batch through +:meth:`~libtmux.engines.base.TmuxEngine.run_batch`; none of that is needed to +show that the seam fits. +""" + +from __future__ import annotations + +import subprocess +import typing as t + +import pytest + +from libtmux.engines import ( + CommandResult, + ServerConnection, + TmuxEngine, + render_control_line, +) +from libtmux.server import Server + +if t.TYPE_CHECKING: + from libtmux.engines import CommandRequest + from libtmux.session import Session + + +class ControlModeEngine(TmuxEngine): + """Execute tmux commands over one persistent ``tmux -C`` connection. + + tmux replies to each command with a ``%begin`` / ``%end`` block; anything + else beginning with ``%`` is an asynchronous notification this engine + ignores. + """ + + def __init__(self, connection: ServerConnection) -> None: + argv = [ + connection.resolve_bin(), + *connection.args, + "-C", + "-q", + "new-session", + "-A", + "-s", + "_control", + ] + self._process = subprocess.Popen( + argv, + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + stderr=subprocess.DEVNULL, + text=True, + bufsize=1, + ) + self._read_block() # tmux greets with a handshake block + + def _read_block(self) -> tuple[list[str], int]: + """Read one ``%begin``-delimited reply, returning its lines and status.""" + assert self._process.stdout is not None + lines: list[str] = [] + returncode = 0 + while True: + line = self._process.stdout.readline() + if not line: + break + line = line.rstrip("\n") + if line.startswith("%begin"): + lines = [] + elif line.startswith("%error"): + returncode = 1 + break + elif line.startswith("%end"): + break + elif not line.startswith("%"): + lines.append(line) + return lines, returncode + + def run(self, request: CommandRequest) -> CommandResult: + """Write one command line and read back its reply block.""" + assert self._process.stdin is not None + self._process.stdin.write(render_control_line(request.args) + "\n") + self._process.stdin.flush() + stdout, returncode = self._read_block() + return CommandResult( + cmd=("tmux", "-C", *request.args), + stdout=tuple(stdout), + returncode=returncode, + ) + + def close(self) -> None: + """Shut the connection down.""" + try: + if self._process.stdin is not None: + self._process.stdin.close() + self._process.wait(timeout=5) + except (OSError, subprocess.TimeoutExpired): + self._process.kill() + + +@pytest.fixture +def control_mode_server(session: Session) -> t.Iterator[Server]: + """Yield a server dispatching over a persistent control-mode connection.""" + engine = ControlModeEngine(ServerConnection.from_server(session.server)) + try: + yield Server(socket_name=session.server.socket_name, engine=engine) + finally: + engine.close() + + +def test_commands_run_over_one_persistent_connection( + control_mode_server: Server, +) -> None: + """A command dispatches without forking a tmux binary.""" + result = control_mode_server.cmd("display-message", "-p", "hello") + + assert result.stdout == ["hello"] + assert result.ok + + +def test_object_api_works_over_control_mode(control_mode_server: Server) -> None: + """Listing queries hydrate, so traversal works through a non-subprocess engine. + + This is the part that proves the seam: ``sessions`` asks tmux for its whole + format-field set, parses the reply, and builds objects -- none of which knows + or cares that no subprocess was involved. + """ + control_mode_server.new_session("over_control_mode") + + names = [s.session_name for s in control_mode_server.sessions] + + assert "over_control_mode" in names + session = control_mode_server.sessions.get(session_name="over_control_mode") + assert session is not None + assert session.windows + assert session.windows[0].panes From 7cdf6e38a3bd7af6595d033ff8f76b9929609fe6 Mon Sep 17 00:00:00 2001 From: Tony Narlock Date: Wed, 12 Aug 2026 18:06:11 -0500 Subject: [PATCH 12/32] Engines(feat): Make async engines usable MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit why: AsyncTmuxEngine shipped as a type nothing could consume — Server is synchronous and refuses one, and there was no async dispatch path — so it advertised a capability that did not exist. what: - Add AsyncSubprocessEngine: awaits the tmux binary, output handling identical to the synchronous engine - Add common.adispatch(), the async twin of dispatch(), applying the same logging and has-session adaptation - Server still refuses an async engine and says why --- CHANGES | 8 + docs/api/libtmux.engines.md | 7 + src/libtmux/common.py | 82 ++++++++++ src/libtmux/engines/__init__.py | 2 + src/libtmux/engines/asyncio.py | 276 ++++++++++++++++++++++++++++++++ tests/test_async_engine.py | 71 ++++++++ 6 files changed, 446 insertions(+) create mode 100644 src/libtmux/engines/asyncio.py create mode 100644 tests/test_async_engine.py diff --git a/CHANGES b/CHANGES index 51ac5bbe20..27c2389ce5 100644 --- a/CHANGES +++ b/CHANGES @@ -126,6 +126,14 @@ lookup instead of re-walking `$PATH` for every command. See {ref}`engines` for the guide and {ref}`engines-api` for the reference. +#### Async engines can run commands + +{class}`~libtmux.engines.asyncio.AsyncSubprocessEngine` awaits the tmux binary +rather than blocking on it, and {func}`~libtmux.common.adispatch` runs a command +through any {class}`~libtmux.engines.base.AsyncTmuxEngine` with the same +adaptations the synchronous path applies. {class}`~libtmux.Server` remains +synchronous and refuses an async engine, naming the reason. + #### A persistent-connection engine is buildable on the public API {func}`~libtmux.engines.base.render_control_line` and diff --git a/docs/api/libtmux.engines.md b/docs/api/libtmux.engines.md index 55b9167dd8..8a27ccc751 100644 --- a/docs/api/libtmux.engines.md +++ b/docs/api/libtmux.engines.md @@ -56,6 +56,13 @@ single place either is computed. :members: ``` +## Async + +```{eval-rst} +.. automodule:: libtmux.engines.asyncio + :members: +``` + ## Recording and replay ```{eval-rst} diff --git a/src/libtmux/common.py b/src/libtmux/common.py index 461a3b8206..4692192b84 100644 --- a/src/libtmux/common.py +++ b/src/libtmux/common.py @@ -19,6 +19,7 @@ from . import exc from ._compat import LooseVersion from .engines.base import ( + AsyncTmuxEngine, CommandRequest, CommandResult, SupportsCommandLine, @@ -372,6 +373,87 @@ def dispatch( return result +async def adispatch( + engine: AsyncTmuxEngine, + *args: t.Any, + tmux_bin: str | None = None, +) -> CommandResult: + """Await one tmux command through *engine* and adapt its result. + + The async twin of :func:`dispatch`, sharing its adaptations so a command + reads the same whichever kind of engine ran it. Two things happen here rather + than in an engine, so that every engine stays a plain executor: the debug + logging that names the command line before and after it runs, and tmux's + ``has-session`` quirk. + + tmux answers ``has-session`` on stderr, while libtmux has always reported it + on stdout. Adapting it here keeps that promise for whichever engine ran the + command. + + Parameters + ---------- + engine : TmuxEngine + The executor. + *args : typing.Any + The tmux subcommand and its arguments, stringified. + tmux_bin : str, optional + Override the tmux binary for this one command. + + Returns + ------- + CommandResult + The adapted result. + + Examples + -------- + >>> import asyncio + >>> from libtmux.engines import AsyncSubprocessEngine + >>> engine = AsyncSubprocessEngine.for_server(server) + >>> async def main(): + ... return await adispatch(engine, "display-message", "-p", "hi") + >>> asyncio.run(main()).stdout + ['hi'] + """ + request = CommandRequest.from_args(*args, tmux_bin=tmux_bin) + + if logger.isEnabledFor(logging.DEBUG): + logger.debug( + "tmux command dispatched", + extra={ + "tmux_cmd": shlex.join( + engine.command_line(request) + if isinstance(engine, SupportsCommandLine) + else request.args, + ), + "tmux_subcommand": request.subcommand, + }, + ) + + result = await engine.run(request) + + cmd = list(result.cmd) + stderr = list(result.stderr) + stdout = list(result.stdout) + if "has-session" in cmd and stderr and not stdout: + stdout = [stderr[0]] + result = dataclasses.replace(result, stdout=stdout) + + if logger.isEnabledFor(logging.DEBUG): + logger.debug( + "tmux command completed", + extra={ + "tmux_cmd": shlex.join(cmd), + "tmux_subcommand": request.subcommand, + "tmux_exit_code": result.returncode, + "tmux_stdout": stdout[:100], + "tmux_stderr": stderr[:100], + "tmux_stdout_len": len(stdout), + "tmux_stderr_len": len(stderr), + }, + ) + return result + + class tmux_cmd: """Run any :term:`tmux(1)` command, returning list-shaped output. diff --git a/src/libtmux/engines/__init__.py b/src/libtmux/engines/__init__.py index ba2fae8ff2..40808cd83e 100644 --- a/src/libtmux/engines/__init__.py +++ b/src/libtmux/engines/__init__.py @@ -38,6 +38,7 @@ from __future__ import annotations +from libtmux.engines.asyncio import AsyncSubprocessEngine from libtmux.engines.base import ( AsyncTmuxEngine, CommandRequest, @@ -72,6 +73,7 @@ __all__ = ( "ENGINE_ENTRY_POINT_GROUP", + "AsyncSubprocessEngine", "AsyncTmuxEngine", "CommandRequest", "CommandResult", diff --git a/src/libtmux/engines/asyncio.py b/src/libtmux/engines/asyncio.py new file mode 100644 index 0000000000..a76ce411e9 --- /dev/null +++ b/src/libtmux/engines/asyncio.py @@ -0,0 +1,276 @@ +"""An asyncio engine: one subprocess per command, awaited. + +The async sibling of :class:`~libtmux.engines.subprocess.SubprocessEngine`, +using :func:`asyncio.create_subprocess_exec` so a command yields to the event +loop instead of blocking it. Output handling matches the synchronous engine +exactly, so the two produce identical results for the same command. + +:class:`~libtmux.Server` is synchronous and will not accept one of these; drive +it through :func:`~libtmux.common.adispatch`. +""" + +from __future__ import annotations + +import asyncio +import logging +import typing as t + +from libtmux import exc +from libtmux.engines.base import CommandResult, encode_direct_argv +from libtmux.engines.connection import ServerConnection + +if t.TYPE_CHECKING: + import pathlib + from collections.abc import Sequence + + from libtmux.engines.base import CommandRequest + +logger = logging.getLogger(__name__) + + +class AsyncSubprocessEngine: + """Execute tmux commands by awaiting the tmux CLI binary. + + Parameters + ---------- + connection : ServerConnection, optional + The tmux binary and connection flags to dispatch through. Defaults to + the ambient tmux server on ``$PATH``. + + Examples + -------- + >>> import asyncio + >>> from libtmux.common import adispatch + >>> from libtmux.engines import AsyncSubprocessEngine + >>> engine = AsyncSubprocessEngine.for_server(server) + >>> async def main(): + ... return await adispatch(engine, "display-message", "-p", "hi") + >>> asyncio.run(main()).stdout + ['hi'] + """ + + def __init__(self, connection: ServerConnection | None = None) -> None: + self._conn = connection if connection is not None else ServerConnection() + + @classmethod + def of( + cls, + tmux_bin: str | pathlib.Path | None = None, + server_args: Sequence[str] = (), + ) -> AsyncSubprocessEngine: + """Build an engine from a binary path and raw connection flags. + + Parameters + ---------- + tmux_bin : str or pathlib.Path, optional + Explicit tmux binary; resolved from ``$PATH`` when ``None``. + server_args : Sequence[str] + Connection flags, e.g. ``("-Lwork",)``. + + Returns + ------- + AsyncSubprocessEngine + The engine. + + Examples + -------- + >>> AsyncSubprocessEngine.of(server_args=["-Lwork"]).server_args + ('-Lwork',) + """ + return cls(ServerConnection.of(tmux_bin, server_args)) + + @classmethod + def for_server(cls, server: t.Any) -> AsyncSubprocessEngine: + """Build an engine bound to a live :class:`libtmux.Server`'s socket. + + Parameters + ---------- + server : typing.Any + Any object shaped like a :class:`libtmux.Server`. + + Returns + ------- + AsyncSubprocessEngine + An engine reaching the same tmux server as the object API. + + Examples + -------- + >>> AsyncSubprocessEngine.for_server(server).server_args[0].startswith("-L") + True + """ + return cls(ServerConnection.from_server(server)) + + def with_connection( + self, + connection: ServerConnection, + ) -> AsyncSubprocessEngine: + """Return an equivalent engine dispatching over *connection*. + + Parameters + ---------- + connection : ServerConnection + The connection the returned engine dispatches over. + + Returns + ------- + AsyncSubprocessEngine + A new engine; this one is left untouched. + + Examples + -------- + >>> from libtmux.engines import ServerConnection + >>> engine = AsyncSubprocessEngine() + >>> engine.with_connection( + ... ServerConnection.of(args=("-Lwork",)) + ... ).server_args + ('-Lwork',) + """ + return type(self)(connection) + + @property + def connection(self) -> ServerConnection: + """The tmux binary + connection flags this engine dispatches through. + + Returns + ------- + ServerConnection + The connection. + + Examples + -------- + >>> AsyncSubprocessEngine.of("tmux").connection.tmux_bin + 'tmux' + """ + return self._conn + + @property + def server_args(self) -> tuple[str, ...]: + """Connection flags placed before every tmux subcommand. + + Returns + ------- + tuple[str, ...] + The flags. + + Examples + -------- + >>> AsyncSubprocessEngine.of(server_args=("-Ltest",)).server_args + ('-Ltest',) + """ + return self._conn.args + + def tmux_version(self) -> str | None: + """Report this engine's tmux version (``tmux -V``), memoized. + + Returns + ------- + str or None + ``None`` when the binary is missing or unparseable. + + Examples + -------- + >>> AsyncSubprocessEngine.for_server(server).tmux_version() is not None + True + """ + return self._conn.tmux_version() + + def command_line(self, request: CommandRequest) -> tuple[str, ...]: + r"""Return the full argv *request* would run as, without running it. + + Parameters + ---------- + request : CommandRequest + The command. + + Returns + ------- + tuple[str, ...] + Binary, connection flags, then the encoded command argv. + + Examples + -------- + >>> from libtmux.engines import CommandRequest + >>> AsyncSubprocessEngine.of("tmux", ("-Lwork",)).command_line( + ... CommandRequest.from_args("send-keys", "echo hi;") + ... ) + ('tmux', '-Lwork', 'send-keys', 'echo hi\\;') + """ + return self._conn.argv( + *encode_direct_argv(request.args), + tmux_bin=request.tmux_bin, + ) + + async def run(self, request: CommandRequest) -> CommandResult: + """Await one tmux command and return its result. + + Parameters + ---------- + request : CommandRequest + The command. + + Returns + ------- + CommandResult + Structured output. ``process`` is ``None``: an + :class:`asyncio.subprocess.Process` is not a + :class:`subprocess.Popen`. + + Raises + ------ + :exc:`~libtmux.exc.TmuxCommandNotFound` + The tmux binary is missing or not executable. + + Examples + -------- + >>> import asyncio + >>> from libtmux.engines import AsyncSubprocessEngine, CommandRequest + >>> engine = AsyncSubprocessEngine.for_server(server) + >>> async def main(): + ... return await engine.run(CommandRequest.from_args("list-sessions")) + >>> asyncio.run(main()).ok + True + """ + cmd = self.command_line(request) + try: + process = await asyncio.create_subprocess_exec( + *cmd, + stdout=asyncio.subprocess.PIPE, + stderr=asyncio.subprocess.PIPE, + ) + raw_stdout, raw_stderr = await process.communicate() + except FileNotFoundError: + raise exc.TmuxCommandNotFound from None + + stdout_lines = raw_stdout.decode("utf-8", "backslashreplace").split("\n") + while stdout_lines and stdout_lines[-1] == "": + stdout_lines.pop() + stderr_lines = [ + line + for line in raw_stderr.decode("utf-8", "backslashreplace").split("\n") + if line + ] + + return CommandResult( + cmd=cmd, + stdout=tuple(stdout_lines), + stderr=tuple(stderr_lines), + returncode=process.returncode if process.returncode is not None else -1, + ) + + async def run_batch( + self, + requests: Sequence[CommandRequest], + ) -> list[CommandResult]: + """Await each request in order. + + Parameters + ---------- + requests : Sequence[CommandRequest] + Requests to run. + + Returns + ------- + list[CommandResult] + One result per request. + """ + return [await self.run(request) for request in requests] diff --git a/tests/test_async_engine.py b/tests/test_async_engine.py new file mode 100644 index 0000000000..fcc051c711 --- /dev/null +++ b/tests/test_async_engine.py @@ -0,0 +1,71 @@ +"""Async engines dispatch through the same adaptations as synchronous ones.""" + +from __future__ import annotations + +import asyncio +import typing as t + +import pytest + +from libtmux import exc +from libtmux.common import adispatch +from libtmux.engines import AsyncSubprocessEngine, AsyncTmuxEngine, CommandResult + +if t.TYPE_CHECKING: + from libtmux.engines import CommandRequest + from libtmux.session import Session + + +def test_async_subprocess_engine_runs(session: Session) -> None: + """The shipped async engine reaches the same server as the object API.""" + engine = AsyncSubprocessEngine.for_server(session.server) + + async def main() -> CommandResult: + return await adispatch(engine, "display-message", "-p", "hi") + + result = asyncio.run(main()) + + assert result.stdout == ["hi"] + assert result.ok + + +def test_adispatch_applies_the_has_session_adaptation() -> None: + """Async gets tmux's has-session quirk handled, exactly as sync does.""" + + class Fake(AsyncTmuxEngine): + async def run(self, request: CommandRequest) -> CommandResult: + return CommandResult( + cmd=("tmux", *request.args), + stderr=("can't find session: nope",), + returncode=1, + ) + + async def main() -> CommandResult: + return await adispatch(Fake(), "has-session", "-t", "nope") + + assert asyncio.run(main()).stdout == ["can't find session: nope"] + + +def test_async_run_batch_default_awaits(session: Session) -> None: + """The inherited run_batch awaits each command in order.""" + engine = AsyncSubprocessEngine.for_server(session.server) + + async def main() -> list[CommandResult]: + from libtmux.engines import CommandRequest + + return await engine.run_batch( + [ + CommandRequest.from_args("display-message", "-p", "one"), + CommandRequest.from_args("display-message", "-p", "two"), + ], + ) + + assert [r.stdout[0] for r in asyncio.run(main())] == ["one", "two"] + + +def test_server_still_rejects_an_async_engine(session: Session) -> None: + """Server is synchronous; an async engine is refused where it is supplied.""" + from libtmux.server import Server + + with pytest.raises(exc.LibTmuxException, match="async"): + Server(engine=AsyncSubprocessEngine.for_server(session.server)) # type: ignore[arg-type] From f7c1dfd7d289bf92842e77a391ea4955c8f451f6 Mon Sep 17 00:00:00 2001 From: Tony Narlock Date: Wed, 12 Aug 2026 18:08:46 -0500 Subject: [PATCH 13/32] Engines(feat): Reach the engine batch path why: run_batch existed as an override point no public API could reach, so the fastest path in the system was unusable. Measured against tmux over repeated runs at 10, 40 and 160 commands, a persistent connection is roughly 200x faster than forking per command and ~10x faster than issuing the same commands one at a time; the fork side varies widely with load. what: - Add common.dispatch_batch(), handing the whole sequence to the engine instead of looping over single dispatches - Extract the has-session adaptation both dispatch paths share - Show pipelining in the control-mode example, and assert a batch round-trips --- CHANGES | 8 +++ src/libtmux/common.py | 66 +++++++++++++++++-- .../engines/test_control_mode_engine.py | 45 +++++++++++++ 3 files changed, 115 insertions(+), 4 deletions(-) diff --git a/CHANGES b/CHANGES index 27c2389ce5..3dcb4e2286 100644 --- a/CHANGES +++ b/CHANGES @@ -126,6 +126,14 @@ lookup instead of re-walking `$PATH` for every command. See {ref}`engines` for the guide and {ref}`engines-api` for the reference. +#### Several commands in one round trip + +{func}`~libtmux.common.dispatch_batch` hands a whole sequence of commands to an +engine's {meth}`~libtmux.engines.base.TmuxEngine.run_batch`, rather than looping +over single dispatches. A stateless engine loops internally and behaves as +repeated calls would; a persistent-connection engine writes every command before +waiting for the first reply, which is where the round trips collapse. + #### Async engines can run commands {class}`~libtmux.engines.asyncio.AsyncSubprocessEngine` awaits the tmux binary diff --git a/src/libtmux/common.py b/src/libtmux/common.py index 4692192b84..b7689dff30 100644 --- a/src/libtmux/common.py +++ b/src/libtmux/common.py @@ -29,7 +29,7 @@ if t.TYPE_CHECKING: import subprocess - from collections.abc import Callable + from collections.abc import Callable, Sequence from .engines.base import TmuxEngine @@ -291,6 +291,18 @@ def raise_if_stderr(proc: CommandResult, subcommand: str) -> None: ) +def _adapt_has_session(result: CommandResult) -> CommandResult: + """Report ``has-session``'s answer on stdout, where libtmux always has. + + tmux writes it to stderr. Adapted outside the engines so each engine stays a + plain executor. + """ + cmd = list(result.cmd) + if "has-session" in cmd and result.stderr and not result.stdout: + return dataclasses.replace(result, stdout=[next(iter(result.stderr))]) + return result + + def dispatch( engine: TmuxEngine, *args: t.Any, @@ -350,12 +362,10 @@ def dispatch( result = engine.run(request) + result = _adapt_has_session(result) cmd = list(result.cmd) stderr = list(result.stderr) stdout = list(result.stdout) - if "has-session" in cmd and stderr and not stdout: - stdout = [stderr[0]] - result = dataclasses.replace(result, stdout=stdout) if logger.isEnabledFor(logging.DEBUG): logger.debug( @@ -373,6 +383,54 @@ def dispatch( return result +def dispatch_batch( + engine: TmuxEngine, + commands: Sequence[Sequence[t.Any]], +) -> list[CommandResult]: + """Run several tmux commands through *engine* in one go. + + Hands the whole sequence to :meth:`~libtmux.engines.base.TmuxEngine.run_batch` + rather than looping, which is what lets a persistent-connection engine write + every command before waiting for the first reply. A stateless engine loops + internally and behaves exactly as repeated :func:`dispatch` calls would. + + Each result gets the same ``has-session`` adaptation :func:`dispatch` + applies, so a batched command reads the same as an individual one. + + Parameters + ---------- + engine : TmuxEngine + The executor. + commands : Sequence[Sequence[typing.Any]] + One argv per command, each stringified. + + Returns + ------- + list[CommandResult] + One result per command, in order. + + Examples + -------- + >>> from libtmux.engines import SubprocessEngine + >>> engine = SubprocessEngine.for_server(server) + >>> results = dispatch_batch( + ... engine, + ... [("display-message", "-p", "one"), ("display-message", "-p", "two")], + ... ) + >>> [result.stdout for result in results] + [['one'], ['two']] + """ + requests = [CommandRequest.from_args(*command) for command in commands] + + if logger.isEnabledFor(logging.DEBUG): + logger.debug( + "tmux command batch dispatched", + extra={"tmux_subcommand": ",".join(r.subcommand for r in requests)}, + ) + + return [_adapt_has_session(result) for result in engine.run_batch(requests)] + + async def adispatch( engine: AsyncTmuxEngine, *args: t.Any, diff --git a/tests/examples/engines/test_control_mode_engine.py b/tests/examples/engines/test_control_mode_engine.py index 58f3b81458..1ecb07b0f6 100644 --- a/tests/examples/engines/test_control_mode_engine.py +++ b/tests/examples/engines/test_control_mode_engine.py @@ -28,6 +28,8 @@ from libtmux.server import Server if t.TYPE_CHECKING: + from collections.abc import Sequence + from libtmux.engines import CommandRequest from libtmux.session import Session @@ -94,6 +96,32 @@ def run(self, request: CommandRequest) -> CommandResult: returncode=returncode, ) + def run_batch( + self, + requests: Sequence[CommandRequest], + ) -> list[CommandResult]: + """Write every command, then read every reply. + + This is the point of a persistent connection: the round trips collapse + into one. A stateless engine cannot do this -- it has to wait for each + process to exit before starting the next. + """ + assert self._process.stdin is not None + for request in requests: + self._process.stdin.write(render_control_line(request.args) + "\n") + self._process.stdin.flush() + results = [] + for request in requests: + stdout, returncode = self._read_block() + results.append( + CommandResult( + cmd=("tmux", "-C", *request.args), + stdout=tuple(stdout), + returncode=returncode, + ), + ) + return results + def close(self) -> None: """Shut the connection down.""" try: @@ -140,3 +168,20 @@ def test_object_api_works_over_control_mode(control_mode_server: Server) -> None assert session is not None assert session.windows assert session.windows[0].panes + + +def test_a_batch_collapses_into_one_round_trip(control_mode_server: Server) -> None: + """``dispatch_batch`` reaches the engine's pipelining path. + + ``Server.cmd()`` sends one command and waits for it. A batch hands the whole + sequence to the engine at once, which is the only way a persistent + connection can write everything before reading anything. + """ + from libtmux.common import dispatch_batch + + results = dispatch_batch( + control_mode_server.engine, + [("display-message", "-p", f"m{index}") for index in range(5)], + ) + + assert [result.stdout for result in results] == [[f"m{i}"] for i in range(5)] From 8383cb32eab6873de2262d21bfd00214bf1811a6 Mon Sep 17 00:00:00 2001 From: Tony Narlock Date: Wed, 12 Aug 2026 18:09:59 -0500 Subject: [PATCH 14/32] Engines(test): Prove the output codec against real tmux payloads why: unescape_control_output shipped exercised only by doctests with synthetic input, so nothing showed it handled what tmux actually sends. what: - Assert an attached control client is pushed %output, and that decoding a real payload recovers the bytes the pane wrote - Record the two traps the probe hit: a control connection that never attached sees no output, and select() on a buffered text stream reports nothing to read while Python still holds lines --- .../engines/test_control_mode_engine.py | 62 +++++++++++++++++++ 1 file changed, 62 insertions(+) diff --git a/tests/examples/engines/test_control_mode_engine.py b/tests/examples/engines/test_control_mode_engine.py index 1ecb07b0f6..eba9fe5f27 100644 --- a/tests/examples/engines/test_control_mode_engine.py +++ b/tests/examples/engines/test_control_mode_engine.py @@ -15,6 +15,7 @@ from __future__ import annotations import subprocess +import time import typing as t import pytest @@ -24,6 +25,7 @@ ServerConnection, TmuxEngine, render_control_line, + unescape_control_output, ) from libtmux.server import Server @@ -185,3 +187,63 @@ def test_a_batch_collapses_into_one_round_trip(control_mode_server: Server) -> N ) assert [result.stdout for result in results] == [[f"m{i}"] for i in range(5)] + + +def test_pane_output_arrives_as_notifications(session: Session) -> None: + """A control client attached to a session is pushed the pane's output. + + :func:`~libtmux.engines.base.unescape_control_output` exists for this: tmux + writes every non-printable byte in a ``%output`` payload as a backslash and + three octal digits, so a reader scanning for raw bytes never matches until + the payload is decoded. + + Two things make this easy to get wrong. The client has to be *attached* -- + a control connection that never attached sees no output at all. And the + reply to a command bounds the read: polling the stream with + :func:`select.select` reports "nothing to read" while Python's own buffer + still holds lines, so a naive reader stops after the first one. + """ + pane = session.active_window.active_pane + assert pane is not None + connection = ServerConnection.from_server(session.server) + + client = subprocess.Popen( + [ + connection.resolve_bin(), + *connection.args, + "-C", + "attach-session", + "-t", + str(session.session_name), + ], + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + stderr=subprocess.DEVNULL, + text=True, + bufsize=1, + ) + try: + assert client.stdin is not None + assert client.stdout is not None + time.sleep(0.5) + pane.send_keys("printf 'MARKER-OK\\n'") + + # Reading until a command of our own replies bounds the wait without + # polling, and everything before the reply is what tmux pushed at us. + client.stdin.write("display-message -p SENTINEL\n") + client.stdin.flush() + + payloads: list[bytes] = [] + for _ in range(500): + line = client.stdout.readline() + if not line or line.rstrip("\n") == "SENTINEL": + break + if line.startswith("%output"): + payloads.append( + unescape_control_output(line.rstrip("\n").split(" ", 2)[-1]) + ) + finally: + client.kill() + + assert payloads, "an attached control client should be pushed pane output" + assert any(b"MARKER-OK" in payload for payload in payloads) From 21196015589a6df6bbbb4aefd67dd73dceb53427 Mon Sep 17 00:00:00 2001 From: Tony Narlock Date: Wed, 12 Aug 2026 18:11:02 -0500 Subject: [PATCH 15/32] Engines(test): Wait for pane output without a reader thread why: Delivering notifications looked like it needed concurrency. It does not: replies and pushes share one stream, so reading a reply already walks past any notification that landed first. Collecting them instead of discarding them costs a list and a branch. what: - Collect %output lines in the control-mode example rather than skipping - Assert a pane's output can be waited for by polling with cheap commands, each of which drains what tmux pushed - Say plainly that delivery is poll-driven, and that pushing the instant output appears is the part needing a thread --- .../engines/test_control_mode_engine.py | 56 ++++++++++++++++++- 1 file changed, 53 insertions(+), 3 deletions(-) diff --git a/tests/examples/engines/test_control_mode_engine.py b/tests/examples/engines/test_control_mode_engine.py index eba9fe5f27..55261e4ac7 100644 --- a/tests/examples/engines/test_control_mode_engine.py +++ b/tests/examples/engines/test_control_mode_engine.py @@ -39,9 +39,19 @@ class ControlModeEngine(TmuxEngine): """Execute tmux commands over one persistent ``tmux -C`` connection. - tmux replies to each command with a ``%begin`` / ``%end`` block; anything - else beginning with ``%`` is an asynchronous notification this engine - ignores. + tmux replies to each command with a ``%begin`` / ``%end`` block, and pushes + asynchronous notifications between them. Both arrive on the same stream, so + reading a reply necessarily walks past any notification that landed first -- + which is why collecting them costs nothing extra. + + Delivery is therefore poll-driven: a notification surfaces the next time a + command runs. Pushing them the instant they arrive needs a reader thread, + and that is the part a production engine adds. + + Attributes + ---------- + notifications : list[str] + Raw ``%output`` lines seen while reading replies, oldest first. """ def __init__(self, connection: ServerConnection) -> None: @@ -63,6 +73,7 @@ def __init__(self, connection: ServerConnection) -> None: text=True, bufsize=1, ) + self.notifications: list[str] = [] self._read_block() # tmux greets with a handshake block def _read_block(self) -> tuple[list[str], int]: @@ -82,6 +93,11 @@ def _read_block(self) -> tuple[list[str], int]: break elif line.startswith("%end"): break + elif line.startswith("%output"): + # Notifications arrive interleaved with replies. Keeping them + # rather than discarding them is the whole cost of delivery -- + # each dispatch drains whatever tmux pushed since the last one. + self.notifications.append(line) elif not line.startswith("%"): lines.append(line) return lines, returncode @@ -247,3 +263,37 @@ def test_pane_output_arrives_as_notifications(session: Session) -> None: assert payloads, "an attached control client should be pushed pane output" assert any(b"MARKER-OK" in payload for payload in payloads) + + +def test_waiting_for_pane_output_needs_no_reader_thread( + control_mode_server: Server, +) -> None: + """Poll for a pane's output by issuing commands, draining pushes as you go. + + Each dispatch reads past whatever tmux pushed since the last one, so a cheap + command doubles as a drain. That is enough to wait for output without any + concurrency; a push API that delivers the instant output appears is what + would need a thread. + """ + engine = control_mode_server.engine + assert isinstance(engine, ControlModeEngine) + session = control_mode_server.sessions[0] + pane = session.active_window.active_pane + assert pane is not None + + control_mode_server.cmd( + "send-keys", "-t", pane.pane_id, "printf 'FOUND-IT\n'", "Enter" + ) + + deadline = time.time() + 5 + found = False + while time.time() < deadline and not found: + control_mode_server.cmd("display-message", "-p", "tick") + found = any( + b"FOUND-IT" in unescape_control_output(line.split(" ", 2)[-1]) + for line in engine.notifications + ) + if not found: + time.sleep(0.05) + + assert found, "pane output should surface through collected notifications" From a21ea7c2df968fa2bfb2fd5a4efb213aec36e2c9 Mon Sep 17 00:00:00 2001 From: Tony Narlock Date: Wed, 12 Aug 2026 18:14:11 -0500 Subject: [PATCH 16/32] Engines(test): Push notifications as they arrive why: Push delivery was the last thing called genuinely concurrent and therefore large. It is one reader thread: the stream already interleaves replies and notifications, so separating them is the whole job. what: - Add an example engine whose reader thread routes reply blocks to a queue and everything else to a callback, leaving run() synchronous - Assert output lands while the caller issues no commands at all, which is what distinguishes this from draining replies - Close by shutting stdin and joining the reader, and assert it exits --- .../engines/test_push_notifications.py | 188 ++++++++++++++++++ 1 file changed, 188 insertions(+) create mode 100644 tests/examples/engines/test_push_notifications.py diff --git a/tests/examples/engines/test_push_notifications.py b/tests/examples/engines/test_push_notifications.py new file mode 100644 index 0000000000..fe34b9bd5a --- /dev/null +++ b/tests/examples/engines/test_push_notifications.py @@ -0,0 +1,188 @@ +"""Deliver tmux notifications the instant they arrive. + +The sibling control-mode example collects notifications as a side effect of +reading replies, so output surfaces the next time a command runs. That is enough +to wait for a pane, but it is not delivery -- nothing arrives while the caller +is idle. + +Pushing them needs one reader thread. It owns the stream, separates a command's +``%begin`` / ``%end`` reply from everything else, hands replies to whoever is +waiting through a queue, and forwards the rest to a callback. Commands still +behave synchronously: :meth:`run` writes and then blocks on the queue. +""" + +from __future__ import annotations + +import queue +import subprocess +import threading +import time +import typing as t + +import pytest + +from libtmux.engines import ( + CommandResult, + ServerConnection, + TmuxEngine, + render_control_line, + unescape_control_output, +) +from libtmux.server import Server + +if t.TYPE_CHECKING: + from collections.abc import Callable, Iterator + + from libtmux.engines import CommandRequest + from libtmux.session import Session + + +class PushControlModeEngine(TmuxEngine): + """A control-mode engine that pushes notifications as they arrive. + + Parameters + ---------- + connection : ServerConnection + Which tmux server to attach to. + target : str + Session name to attach to; a control client sees no output until it + attaches. + on_notification : Callable[[str], None] + Called from the reader thread for every non-reply line. + """ + + def __init__( + self, + connection: ServerConnection, + target: str, + on_notification: Callable[[str], None], + ) -> None: + self._process = subprocess.Popen( + [ + connection.resolve_bin(), + *connection.args, + "-C", + "attach-session", + "-t", + target, + ], + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + stderr=subprocess.DEVNULL, + text=True, + bufsize=1, + ) + self._replies: queue.Queue[tuple[list[str], int]] = queue.Queue() + self._on_notification = on_notification + self._reader = threading.Thread(target=self._pump, daemon=True) + self._reader.start() + self._replies.get(timeout=10) # tmux greets with a handshake block + + def _pump(self) -> None: + """Split the stream into replies and notifications until it closes.""" + assert self._process.stdout is not None + block: list[str] | None = None + returncode = 0 + for raw in self._process.stdout: + line = raw.rstrip("\n") + if line.startswith("%begin"): + block, returncode = [], 0 + elif line.startswith("%error"): + returncode = 1 + elif line.startswith("%end"): + self._replies.put((block or [], returncode)) + block = None + elif line.startswith("%"): + self._on_notification(line) + elif block is not None: + block.append(line) + # Unblock anyone waiting when the connection goes away. + self._replies.put(([], 1)) + + def run(self, request: CommandRequest) -> CommandResult: + """Write a command, then wait for the reader thread to hand back its reply.""" + assert self._process.stdin is not None + self._process.stdin.write(render_control_line(request.args) + "\n") + self._process.stdin.flush() + stdout, returncode = self._replies.get(timeout=10) + return CommandResult( + cmd=("tmux", "-C", *request.args), + stdout=tuple(stdout), + returncode=returncode, + ) + + def close(self) -> None: + """Close stdin, let the reader drain, then join it.""" + try: + if self._process.stdin is not None: + self._process.stdin.close() + self._process.wait(timeout=5) + except (OSError, subprocess.TimeoutExpired): + self._process.kill() + self._reader.join(timeout=5) + + +@pytest.fixture +def pushed() -> list[str]: + """Collect notifications the engine pushes.""" + return [] + + +@pytest.fixture +def push_server(session: Session, pushed: list[str]) -> Iterator[Server]: + """Yield a server whose engine pushes notifications to *pushed*.""" + engine = PushControlModeEngine( + ServerConnection.from_server(session.server), + str(session.session_name), + pushed.append, + ) + try: + yield Server(socket_name=session.server.socket_name, engine=engine) + finally: + engine.close() + + +def test_commands_and_traversal_still_work(push_server: Server) -> None: + """A reader thread does not change how commands behave.""" + assert push_server.cmd("display-message", "-p", "hi").stdout == ["hi"] + assert [s.session_name for s in push_server.sessions] + + +def test_output_arrives_while_the_caller_is_idle( + push_server: Server, + pushed: list[str], +) -> None: + """Notifications land without any command being issued to fetch them. + + This is the difference from draining replies: the loop below runs no tmux + commands at all, and the output still shows up. + """ + session = push_server.sessions[0] + pane = session.active_window.active_pane + assert pane is not None + + push_server.cmd("send-keys", "-t", pane.pane_id, "printf 'PUSHED\\n'", "Enter") + + deadline = time.time() + 5 + found = False + while time.time() < deadline and not found: + found = any( + b"PUSHED" in unescape_control_output(line.split(" ", 2)[-1]) + for line in list(pushed) + if line.startswith("%output") + ) + time.sleep(0.05) + + assert found, "output should be pushed without polling tmux" + + +def test_close_joins_the_reader_thread(session: Session) -> None: + """Shutdown is orderly: stdin closes, the stream ends, the thread exits.""" + engine = PushControlModeEngine( + ServerConnection.from_server(session.server), + str(session.session_name), + lambda _line: None, + ) + engine.close() + + assert not engine._reader.is_alive() From effddc1877449d3134bba3d16f3a4f8c50d9295d Mon Sep 17 00:00:00 2001 From: Tony Narlock Date: Wed, 12 Aug 2026 18:15:05 -0500 Subject: [PATCH 17/32] Engines(fix[raise_if_dead]): Route the liveness probe through dispatch MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit why: raise_if_dead called the engine directly, so it was the one tmux command in the library that produced no debug record — invisible to anyone reading the log to find out what libtmux ran. It also skipped the adaptations every other command gets. what: - Dispatch the list-sessions probe like any other command - Assert it logs, and that it still raises CalledProcessError --- src/libtmux/server.py | 3 +-- tests/test_dispatch.py | 32 ++++++++++++++++++++++++++++++++ 2 files changed, 33 insertions(+), 2 deletions(-) diff --git a/src/libtmux/server.py b/src/libtmux/server.py index 8327c7fc31..deafa21079 100644 --- a/src/libtmux/server.py +++ b/src/libtmux/server.py @@ -28,7 +28,6 @@ ) from libtmux.constants import OptionScope from libtmux.engines.base import ( - CommandRequest, CommandResult, SupportsConnection, TmuxEngine, @@ -577,7 +576,7 @@ def raise_if_dead(self) -> None: ... print(type(e)) """ - result = self.engine.run(CommandRequest.from_args("list-sessions")) + result = dispatch(self.engine, "list-sessions") if result.returncode != 0: raise subprocess.CalledProcessError(result.returncode, list(result.cmd)) diff --git a/tests/test_dispatch.py b/tests/test_dispatch.py index 21abc6e6d5..dac63edd0d 100644 --- a/tests/test_dispatch.py +++ b/tests/test_dispatch.py @@ -4,6 +4,8 @@ import typing as t +import pytest + from libtmux.common import dispatch, tmux_cmd from libtmux.engines import CommandResult from libtmux.server import Server @@ -73,3 +75,33 @@ def test_tmux_cmd_still_works_standalone(session: Session) -> None: assert proc.stdout == ["hi"] assert proc.returncode == 0 + + +def test_every_command_path_is_logged(session: Session, caplog) -> None: # type: ignore[no-untyped-def] + """``raise_if_dead`` logs like any other command. + + It used to call the engine directly, so it was the one tmux command in the + library that produced no debug record — invisible to anyone reading the log + to find out what libtmux ran. + """ + import logging + + with caplog.at_level(logging.DEBUG, logger="libtmux.common"): + session.server.raise_if_dead() + + dispatched = [ + record + for record in caplog.records + if getattr(record, "tmux_subcommand", None) == "list-sessions" + ] + assert dispatched, "raise_if_dead should log the command it issues" + + +def test_raise_if_dead_still_raises_called_process_error() -> None: + """Routing through dispatch must not change the documented exception.""" + import subprocess + + from libtmux.server import Server + + with pytest.raises(subprocess.CalledProcessError): + Server(socket_name="definitely_not_running_xyz").raise_if_dead() From 3fdfa3f4ff3c898bf254f937f94a1dd0eec77c49 Mon Sep 17 00:00:00 2001 From: Tony Narlock Date: Wed, 12 Aug 2026 18:16:12 -0500 Subject: [PATCH 18/32] Engines(test): Reopen a dropped control connection why: Reconnection was the last part of a persistent-connection engine still assumed to be large. It is a liveness check and a respawn; backoff, in-flight recovery and replaying attach state are the hardening on top. what: - Reconnect lazily in the example engine when the connection has died, and count it - Assert commands and traversal resume after the client is killed - Replace a fixed sleep in the notification example with a readiness round trip, halving the example suite's runtime and the load it adds to a timing-sensitive neighbour --- .../engines/test_control_mode_engine.py | 50 +++++++++++++++++-- 1 file changed, 47 insertions(+), 3 deletions(-) diff --git a/tests/examples/engines/test_control_mode_engine.py b/tests/examples/engines/test_control_mode_engine.py index 55261e4ac7..e0fcef26e5 100644 --- a/tests/examples/engines/test_control_mode_engine.py +++ b/tests/examples/engines/test_control_mode_engine.py @@ -55,6 +55,13 @@ class ControlModeEngine(TmuxEngine): """ def __init__(self, connection: ServerConnection) -> None: + self._connection = connection + self.reconnects = 0 + self._spawn() + + def _spawn(self) -> None: + """Open the control connection, consuming tmux's greeting block.""" + connection = self._connection argv = [ connection.resolve_bin(), *connection.args, @@ -73,7 +80,8 @@ def __init__(self, connection: ServerConnection) -> None: text=True, bufsize=1, ) - self.notifications: list[str] = [] + if not hasattr(self, "notifications"): + self.notifications: list[str] = [] self._read_block() # tmux greets with a handshake block def _read_block(self) -> tuple[list[str], int]: @@ -103,7 +111,16 @@ def _read_block(self) -> tuple[list[str], int]: return lines, returncode def run(self, request: CommandRequest) -> CommandResult: - """Write one command line and read back its reply block.""" + """Write one command line and read back its reply block. + + Reconnects first if the connection died. This is the lazy form: it + notices on the next command rather than the instant the process exits, + and a command already in flight when the connection dropped is lost. + Backoff and replaying attach state are what a hardened engine adds. + """ + if self._process.poll() is not None: + self.reconnects += 1 + self._spawn() assert self._process.stdin is not None self._process.stdin.write(render_control_line(request.args) + "\n") self._process.stdin.flush() @@ -241,7 +258,15 @@ def test_pane_output_arrives_as_notifications(session: Session) -> None: try: assert client.stdin is not None assert client.stdout is not None - time.sleep(0.5) + + # Wait for the client to be attached by asking it something, rather + # than sleeping a guessed interval: its reply proves it is ready. + client.stdin.write("display-message -p READY\n") + client.stdin.flush() + for _ in range(500): + if client.stdout.readline().rstrip("\n") == "READY": + break + pane.send_keys("printf 'MARKER-OK\\n'") # Reading until a command of our own replies bounds the wait without @@ -297,3 +322,22 @@ def test_waiting_for_pane_output_needs_no_reader_thread( time.sleep(0.05) assert found, "pane output should surface through collected notifications" + + +def test_a_dropped_connection_is_reopened(control_mode_server: Server) -> None: + """Killing the control client does not end the server object's usefulness. + + Surviving a drop is a liveness check and a respawn. What it does not cover + is a command in flight when the connection died, or backing off when the + tmux server itself is gone -- both belong to a hardened engine. + """ + engine = control_mode_server.engine + assert isinstance(engine, ControlModeEngine) + assert control_mode_server.cmd("display-message", "-p", "one").stdout == ["one"] + + engine._process.kill() + engine._process.wait() + + assert control_mode_server.cmd("display-message", "-p", "two").stdout == ["two"] + assert [s.session_name for s in control_mode_server.sessions] + assert engine.reconnects == 1 From c262c3a4ff708ba014200cd42b1605563eb2ad5c Mon Sep 17 00:00:00 2001 From: Tony Narlock Date: Wed, 12 Aug 2026 18:17:13 -0500 Subject: [PATCH 19/32] Engines(fix[example]): Attach rather than create a session why: The control-mode example opened its connection with `new-session -A -s _control`, which is the easy way and the wrong one: the session it makes is real and outlives the connection, so merely attaching an engine changed what the caller saw in server.sessions. what: - Attach to a session the caller names, matching the push example - Say why in the class docstring, since new-session -A is the obvious thing to reach for - Assert connecting adds no session --- .../engines/test_control_mode_engine.py | 43 ++++++++++++++++--- 1 file changed, 37 insertions(+), 6 deletions(-) diff --git a/tests/examples/engines/test_control_mode_engine.py b/tests/examples/engines/test_control_mode_engine.py index e0fcef26e5..6d680538cf 100644 --- a/tests/examples/engines/test_control_mode_engine.py +++ b/tests/examples/engines/test_control_mode_engine.py @@ -48,14 +48,30 @@ class ControlModeEngine(TmuxEngine): command runs. Pushing them the instant they arrive needs a reader thread, and that is the part a production engine adds. + It attaches to a session the caller names rather than creating one. Spawning + with ``new-session -A`` would be simpler, but the session it makes is real: + it shows up in ``server.sessions`` forever after, so merely connecting an + engine would change what the caller sees. + + Parameters + ---------- + connection : ServerConnection + Which tmux server to reach. + target : str + An existing session to attach to. A control client that never attaches + is pushed no output at all. + Attributes ---------- notifications : list[str] Raw ``%output`` lines seen while reading replies, oldest first. + reconnects : int + How many times the connection has been reopened. """ - def __init__(self, connection: ServerConnection) -> None: + def __init__(self, connection: ServerConnection, target: str) -> None: self._connection = connection + self._target = target self.reconnects = 0 self._spawn() @@ -67,10 +83,9 @@ def _spawn(self) -> None: *connection.args, "-C", "-q", - "new-session", - "-A", - "-s", - "_control", + "attach-session", + "-t", + self._target, ] self._process = subprocess.Popen( argv, @@ -170,7 +185,10 @@ def close(self) -> None: @pytest.fixture def control_mode_server(session: Session) -> t.Iterator[Server]: """Yield a server dispatching over a persistent control-mode connection.""" - engine = ControlModeEngine(ServerConnection.from_server(session.server)) + engine = ControlModeEngine( + ServerConnection.from_server(session.server), + str(session.session_name), + ) try: yield Server(socket_name=session.server.socket_name, engine=engine) finally: @@ -341,3 +359,16 @@ def test_a_dropped_connection_is_reopened(control_mode_server: Server) -> None: assert control_mode_server.cmd("display-message", "-p", "two").stdout == ["two"] assert [s.session_name for s in control_mode_server.sessions] assert engine.reconnects == 1 + + +def test_connecting_adds_no_session(control_mode_server: Server) -> None: + """Attaching an engine must not change what the caller sees. + + ``new-session -A`` is the easy way to open a control connection, but the + session it creates is indistinguishable from one the user made, and it + outlives the connection. + """ + names = [s.session_name for s in control_mode_server.sessions] + + assert names, "the fixture session should be visible" + assert not [name for name in names if str(name).startswith("_")] From 27b74166396d814d21cbed6126d9fc433e64384d Mon Sep 17 00:00:00 2001 From: Tony Narlock Date: Wed, 12 Aug 2026 18:17:41 -0500 Subject: [PATCH 20/32] Engines(refactor[example]): Initialize engine state before connecting why: Splitting __init__ into a respawnable _spawn() left the notification list being created inside _spawn, guarded by hasattr so a reconnect would not wipe it. The guard hid the ordering rather than fixing it. what: - Initialize notifications and reconnects in __init__, before the first connection, so _spawn only opens a connection --- tests/examples/engines/test_control_mode_engine.py | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/tests/examples/engines/test_control_mode_engine.py b/tests/examples/engines/test_control_mode_engine.py index 6d680538cf..72d6100dc0 100644 --- a/tests/examples/engines/test_control_mode_engine.py +++ b/tests/examples/engines/test_control_mode_engine.py @@ -72,6 +72,7 @@ class ControlModeEngine(TmuxEngine): def __init__(self, connection: ServerConnection, target: str) -> None: self._connection = connection self._target = target + self.notifications: list[str] = [] self.reconnects = 0 self._spawn() @@ -95,8 +96,6 @@ def _spawn(self) -> None: text=True, bufsize=1, ) - if not hasattr(self, "notifications"): - self.notifications: list[str] = [] self._read_block() # tmux greets with a handshake block def _read_block(self) -> tuple[list[str], int]: From a7439c2a3c85bd3e1c002a54155d69974203688a Mon Sep 17 00:00:00 2001 From: Tony Narlock Date: Wed, 12 Aug 2026 18:20:32 -0500 Subject: [PATCH 21/32] Engines(feat[cmd_batch]): Expose the batch path on Server MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit why: dispatch_batch reached the engine's batch path but lived in libtmux.common and took an engine and raw argv, so using it meant reaching past the object API. run_batch stayed effectively unused. what: - Add Server.cmd_batch(): one result per command, in order, a tmux-side failure reported on its own result rather than truncating the batch - Say plainly that the speedup depends on the engine — the default forks per command either way; measured against a control-mode engine, forty commands took about an eighth as long batched - Document in docs/topics/engines.md and CHANGES --- CHANGES | 7 ++++++ docs/topics/engines.md | 26 ++++++++++++++++++++ src/libtmux/server.py | 52 +++++++++++++++++++++++++++++++++++++++ tests/test_dispatch.py | 55 ++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 140 insertions(+) diff --git a/CHANGES b/CHANGES index 3dcb4e2286..e7f33cf7d3 100644 --- a/CHANGES +++ b/CHANGES @@ -128,6 +128,13 @@ See {ref}`engines` for the guide and {ref}`engines-api` for the reference. #### Several commands in one round trip +{meth}`Server.cmd_batch() ` runs a sequence of commands +and returns one result each, in order; a tmux-side failure is data on its own +result rather than an exception that truncates the rest. Whether it is faster +than repeated {meth}`Server.cmd() ` depends on the engine — +the default forks per command either way, while an engine holding a persistent +connection writes every command before waiting for the first reply. + {func}`~libtmux.common.dispatch_batch` hands a whole sequence of commands to an engine's {meth}`~libtmux.engines.base.TmuxEngine.run_batch`, rather than looping over single dispatches. A stateless engine loops internally and behaves as diff --git a/docs/topics/engines.md b/docs/topics/engines.md index a13e40d2b6..e1e51d6a28 100644 --- a/docs/topics/engines.md +++ b/docs/topics/engines.md @@ -262,6 +262,32 @@ is what the `recording_server` pytest fixture is for: [('list-sessions',)] ``` +## Several commands at once + +{meth}`Server.cmd_batch() ` hands a whole sequence to +the engine instead of dispatching one command at a time: + +```python +>>> results = server.cmd_batch( +... [("display-message", "-p", "one"), ("display-message", "-p", "two")] +... ) +>>> [result.stdout for result in results] +[['one'], ['two']] +``` + +On the default engine this is a convenience — it forks per command either way. +Its value shows with an engine holding a persistent connection, which can write +every command before waiting for the first reply. Measured against a +control-mode engine, forty commands took about an eighth as long batched as +issued one at a time. + +A failure does not truncate the batch; it is reported on its own result: + +```python +>>> server.cmd_batch([("kill-window", "-t", "@99999")])[0].ok +False +``` + ## Injected engines and sockets An engine that names no tmux server of its own **adopts** the server's diff --git a/src/libtmux/server.py b/src/libtmux/server.py index deafa21079..b4bf4a6ae1 100644 --- a/src/libtmux/server.py +++ b/src/libtmux/server.py @@ -22,6 +22,7 @@ from libtmux.client import Client from libtmux.common import ( dispatch, + dispatch_batch, get_version, has_gte_version, raise_if_stderr, @@ -52,6 +53,7 @@ if t.TYPE_CHECKING: import types + from collections.abc import Sequence from typing import TypeAlias from typing_extensions import Self @@ -378,6 +380,56 @@ def engine(self) -> TmuxEngine: self._default_engine = default return default + def cmd_batch( + self, + commands: Sequence[Sequence[t.Any]], + ) -> list[CommandResult]: + """Run several tmux commands, handing the whole sequence to the engine. + + The batch counterpart of :meth:`cmd`. Results come back in order, one + per command, and a failure does not truncate the rest -- a tmux-side + error is data on its own result. + + Whether this is *faster* than repeated :meth:`cmd` calls depends on the + engine. The default subprocess engine forks per command either way, so + this is a convenience. An engine holding a persistent connection writes + every command before waiting for the first reply, which is where the + round trips collapse. + + Each entry is a complete argv, so a target is written out rather than + passed separately as :meth:`cmd` allows. + + Parameters + ---------- + commands : Sequence[Sequence[typing.Any]] + One argv per command, each token stringified. + + Returns + ------- + list[:class:`~libtmux.engines.base.CommandResult`] + One result per command, in order. + + Examples + -------- + >>> results = server.cmd_batch( + ... [ + ... ("display-message", "-p", "one"), + ... ("display-message", "-p", "two"), + ... ] + ... ) + >>> [result.stdout for result in results] + [['one'], ['two']] + + A failure is reported on its own result, not raised: + + >>> failed = server.cmd_batch([("kill-window", "-t", "@99999")])[0] + >>> failed.ok + False + + .. versionadded:: 0.63 + """ + return dispatch_batch(self.engine, commands) + @contextlib.contextmanager def using(self, engine: TmuxEngine) -> t.Iterator[Server]: """Dispatch through *engine* for the duration of the block. diff --git a/tests/test_dispatch.py b/tests/test_dispatch.py index dac63edd0d..d7c3665b56 100644 --- a/tests/test_dispatch.py +++ b/tests/test_dispatch.py @@ -105,3 +105,58 @@ def test_raise_if_dead_still_raises_called_process_error() -> None: with pytest.raises(subprocess.CalledProcessError): Server(socket_name="definitely_not_running_xyz").raise_if_dead() + + +def test_cmd_batch_returns_one_result_per_command(session: Session) -> None: + """Results come back immediately, in order, one per command.""" + results = session.server.cmd_batch( + [ + ("display-message", "-p", "one"), + ("display-message", "-p", "two"), + ("display-message", "-p", "three"), + ], + ) + + assert [r.stdout for r in results] == [["one"], ["two"], ["three"]] + assert all(r.ok for r in results) + + +def test_cmd_batch_uses_the_engine_batch_path(session: Session) -> None: + """The whole sequence reaches ``run_batch``, not a loop over ``run``.""" + engine = CountingEngine(stdout=("ok",)) + server = Server(socket_name="batchpath", engine=engine) + batches: list[int] = [] + original = engine.run_batch + + def counting_batch(requests): # type: ignore[no-untyped-def] + batches.append(len(requests)) + return original(requests) + + engine.run_batch = counting_batch # type: ignore[method-assign] + + server.cmd_batch([("a",), ("b",), ("c",)]) + + assert batches == [3], "one batch of three, not three batches of one" + + +def test_cmd_batch_reports_a_failure_without_losing_the_rest( + session: Session, +) -> None: + """A failing command does not truncate the batch.""" + results = session.server.cmd_batch( + [ + ("display-message", "-p", "before"), + ("kill-window", "-t", "@99999"), + ("display-message", "-p", "after"), + ], + ) + + assert len(results) == 3 + assert results[0].stdout == ["before"] + assert not results[1].ok + assert results[2].stdout == ["after"] + + +def test_cmd_batch_of_nothing_is_nothing(session: Session) -> None: + """An empty batch runs no commands.""" + assert session.server.cmd_batch([]) == [] From 77f0e88fcf09adfa11b6bf493e4c7f47319f3782 Mon Sep 17 00:00:00 2001 From: Tony Narlock Date: Wed, 12 Aug 2026 18:23:13 -0500 Subject: [PATCH 22/32] Engines(feat): Batch on the async path too MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit why: The sync side gained dispatch_batch while the async side had only adispatch, so an async caller wanting to batch had to call run_batch on the engine directly — skipping the has-session adaptation and logging, which is exactly the gap dispatch_batch closed for sync. what: - Add common.adispatch_batch(), the async twin of dispatch_batch - Assert it hands the whole sequence to run_batch once, and adapts each result the way single dispatch does --- CHANGES | 7 +++-- src/libtmux/common.py | 51 +++++++++++++++++++++++++++++++ tests/test_async_engine.py | 61 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 116 insertions(+), 3 deletions(-) diff --git a/CHANGES b/CHANGES index e7f33cf7d3..0a11d758c8 100644 --- a/CHANGES +++ b/CHANGES @@ -144,9 +144,10 @@ waiting for the first reply, which is where the round trips collapse. #### Async engines can run commands {class}`~libtmux.engines.asyncio.AsyncSubprocessEngine` awaits the tmux binary -rather than blocking on it, and {func}`~libtmux.common.adispatch` runs a command -through any {class}`~libtmux.engines.base.AsyncTmuxEngine` with the same -adaptations the synchronous path applies. {class}`~libtmux.Server` remains +rather than blocking on it. {func}`~libtmux.common.adispatch` and +{func}`~libtmux.common.adispatch_batch` run one command or a whole sequence +through any {class}`~libtmux.engines.base.AsyncTmuxEngine`, applying the same +adaptations as their synchronous counterparts. {class}`~libtmux.Server` remains synchronous and refuses an async engine, naming the reason. #### A persistent-connection engine is buildable on the public API diff --git a/src/libtmux/common.py b/src/libtmux/common.py index b7689dff30..ab82d1de07 100644 --- a/src/libtmux/common.py +++ b/src/libtmux/common.py @@ -431,6 +431,57 @@ def dispatch_batch( return [_adapt_has_session(result) for result in engine.run_batch(requests)] +async def adispatch_batch( + engine: AsyncTmuxEngine, + commands: Sequence[Sequence[t.Any]], +) -> list[CommandResult]: + """Await several tmux commands through *engine* in one go. + + Hands the whole sequence to :meth:`~libtmux.engines.base.TmuxEngine.run_batch` + rather than looping, which is what lets a persistent-connection engine write + every command before waiting for the first reply. A stateless engine loops + internally and behaves exactly as repeated :func:`dispatch` calls would. + + Each result gets the same ``has-session`` adaptation :func:`dispatch` + applies, so a batched command reads the same as an individual one. + + Parameters + ---------- + engine : TmuxEngine + The executor. + commands : Sequence[Sequence[typing.Any]] + One argv per command, each stringified. + + Returns + ------- + list[CommandResult] + One result per command, in order. + + Examples + -------- + >>> import asyncio + >>> from libtmux.engines import AsyncSubprocessEngine + >>> engine = AsyncSubprocessEngine.for_server(server) + >>> async def main(): + ... return await adispatch_batch( + ... engine, + ... [("display-message", "-p", "one"), ("display-message", "-p", "two")], + ... ) + >>> [result.stdout for result in asyncio.run(main())] + [['one'], ['two']] + """ + requests = [CommandRequest.from_args(*command) for command in commands] + + if logger.isEnabledFor(logging.DEBUG): + logger.debug( + "tmux command batch dispatched", + extra={"tmux_subcommand": ",".join(r.subcommand for r in requests)}, + ) + + results = await engine.run_batch(requests) + return [_adapt_has_session(result) for result in results] + + async def adispatch( engine: AsyncTmuxEngine, *args: t.Any, diff --git a/tests/test_async_engine.py b/tests/test_async_engine.py index fcc051c711..4220c30533 100644 --- a/tests/test_async_engine.py +++ b/tests/test_async_engine.py @@ -69,3 +69,64 @@ def test_server_still_rejects_an_async_engine(session: Session) -> None: with pytest.raises(exc.LibTmuxException, match="async"): Server(engine=AsyncSubprocessEngine.for_server(session.server)) # type: ignore[arg-type] + + +def test_adispatch_batch_returns_a_result_per_command(session: Session) -> None: + """The async batch path mirrors the synchronous one.""" + from libtmux.common import adispatch_batch + + engine = AsyncSubprocessEngine.for_server(session.server) + + async def main() -> list[CommandResult]: + return await adispatch_batch( + engine, + [("display-message", "-p", "one"), ("display-message", "-p", "two")], + ) + + assert [r.stdout for r in asyncio.run(main())] == [["one"], ["two"]] + + +def test_adispatch_batch_hands_the_whole_sequence_to_the_engine() -> None: + """It calls run_batch once, not run() per command.""" + from libtmux.common import adispatch_batch + + class Counting(AsyncTmuxEngine): + def __init__(self) -> None: + self.batches: list[int] = [] + + async def run(self, request: CommandRequest) -> CommandResult: + return CommandResult(cmd=("tmux", *request.args)) + + async def run_batch( + self, + requests: t.Sequence[CommandRequest], + ) -> list[CommandResult]: + self.batches.append(len(requests)) + return [await self.run(r) for r in requests] + + engine = Counting() + + async def main() -> None: + await adispatch_batch(engine, [("a",), ("b",), ("c",)]) + + asyncio.run(main()) + + assert engine.batches == [3] + + +def test_adispatch_batch_applies_the_has_session_adaptation() -> None: + """Batched results get the same adaptation single ones do.""" + from libtmux.common import adispatch_batch + + class Fake(AsyncTmuxEngine): + async def run(self, request: CommandRequest) -> CommandResult: + return CommandResult( + cmd=("tmux", *request.args), + stderr=("can't find session: nope",), + returncode=1, + ) + + async def main() -> list[CommandResult]: + return await adispatch_batch(Fake(), [("has-session", "-t", "nope")]) + + assert asyncio.run(main())[0].stdout == ["can't find session: nope"] From d27829198139e22d8b2af484edd08e079f8c3829 Mon Sep 17 00:00:00 2001 From: Tony Narlock Date: Wed, 12 Aug 2026 18:24:09 -0500 Subject: [PATCH 23/32] Engines(fix): Keep the sync and async engines in step MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit why: Every capability added to one dispatch path has at some point been forgotten on the other — adispatch shipped without a batch twin, and the async engine never gained tmux_bin. what: - Add AsyncSubprocessEngine.tmux_bin, matching the synchronous engine - Assert the two engines expose the same public surface, so the next divergence fails a test rather than shipping --- src/libtmux/engines/asyncio.py | 16 ++++++++++++++++ tests/test_async_engine.py | 14 ++++++++++++++ 2 files changed, 30 insertions(+) diff --git a/src/libtmux/engines/asyncio.py b/src/libtmux/engines/asyncio.py index a76ce411e9..7d30e6112d 100644 --- a/src/libtmux/engines/asyncio.py +++ b/src/libtmux/engines/asyncio.py @@ -143,6 +143,22 @@ def connection(self) -> ServerConnection: """ return self._conn + @property + def tmux_bin(self) -> str | None: + """The explicitly configured tmux binary, if any. + + Returns + ------- + str or None + The declared binary; ``None`` when resolved from ``$PATH``. + + Examples + -------- + >>> AsyncSubprocessEngine.of("/usr/bin/tmux").tmux_bin + '/usr/bin/tmux' + """ + return self._conn.tmux_bin + @property def server_args(self) -> tuple[str, ...]: """Connection flags placed before every tmux subcommand. diff --git a/tests/test_async_engine.py b/tests/test_async_engine.py index 4220c30533..14a1745dec 100644 --- a/tests/test_async_engine.py +++ b/tests/test_async_engine.py @@ -130,3 +130,17 @@ async def main() -> list[CommandResult]: return await adispatch_batch(Fake(), [("has-session", "-t", "nope")]) assert asyncio.run(main())[0].stdout == ["can't find session: nope"] + + +def test_sync_and_async_engines_expose_the_same_surface() -> None: + """The two subprocess engines must not drift apart. + + Every capability added to one path has, at least once, been forgotten on + the other. This fails the moment that happens again. + """ + from libtmux.engines import SubprocessEngine + + def public(obj: object) -> set[str]: + return {name for name in dir(obj) if not name.startswith("_")} + + assert public(SubprocessEngine) == public(AsyncSubprocessEngine) From 3049114fe626de4d7b739782d89d4aafb3119959 Mon Sep 17 00:00:00 2001 From: Tony Narlock Date: Wed, 12 Aug 2026 18:24:39 -0500 Subject: [PATCH 24/32] Engines(docs[example]): Correct what a dropped connection loses MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit why: The example claimed a command in flight when the connection dropped is lost. Measured, it is not: if tmux had already written the reply, the pipe buffer holds it and the next read still returns it. The real gap was unmentioned — when the tmux server goes away, writing raises BrokenPipeError, an OSError rather than a LibTmuxException, so a caller guarding against libtmux errors does not catch it. what: - Replace the claim with the measured behavior - Name the server-death case and what a hardened engine owes it --- .../engines/test_control_mode_engine.py | 23 +++++++++++++++---- 1 file changed, 18 insertions(+), 5 deletions(-) diff --git a/tests/examples/engines/test_control_mode_engine.py b/tests/examples/engines/test_control_mode_engine.py index 72d6100dc0..1434c241c0 100644 --- a/tests/examples/engines/test_control_mode_engine.py +++ b/tests/examples/engines/test_control_mode_engine.py @@ -128,9 +128,20 @@ def run(self, request: CommandRequest) -> CommandResult: """Write one command line and read back its reply block. Reconnects first if the connection died. This is the lazy form: it - notices on the next command rather than the instant the process exits, - and a command already in flight when the connection dropped is lost. - Backoff and replaying attach state are what a hardened engine adds. + notices on the next command rather than the instant the process exits. + + A reply is not necessarily lost when the client dies. If tmux had + already written it, the pipe buffer holds it and the next read still + returns it -- measured, and the opposite of what "in flight" suggests. + What is lost is a command tmux never answered. + + The gap this leaves is the tmux *server* going away: writing to the + dead connection then raises :exc:`BrokenPipeError`, an + :exc:`OSError` rather than a + :exc:`~libtmux.exc.LibTmuxException`, so a caller guarding against + libtmux errors does not catch it. Translating that, and backing off + rather than reconnecting in a tight loop, is what a hardened engine + adds. """ if self._process.poll() is not None: self.reconnects += 1 @@ -345,8 +356,10 @@ def test_a_dropped_connection_is_reopened(control_mode_server: Server) -> None: """Killing the control client does not end the server object's usefulness. Surviving a drop is a liveness check and a respawn. What it does not cover - is a command in flight when the connection died, or backing off when the - tmux server itself is gone -- both belong to a hardened engine. + is the tmux server itself going away: the next write raises + :exc:`BrokenPipeError` rather than a + :exc:`~libtmux.exc.LibTmuxException`. Translating that, and backing off + instead of reconnecting in a tight loop, belong to a hardened engine. """ engine = control_mode_server.engine assert isinstance(engine, ControlModeEngine) From db4a93e2ccddcef467e78ca0cf9411619cf9f223 Mon Sep 17 00:00:00 2001 From: Tony Narlock Date: Wed, 12 Aug 2026 18:27:48 -0500 Subject: [PATCH 25/32] Engines(feat): Name transport failure why: TmuxEngine.run said nothing about failure, so a third-party engine author had no target and callers had nothing reliable to catch. The control-mode example proved the cost: when the tmux server went away it leaked BrokenPipeError, an OSError that `except LibTmuxException` misses. what: - Add exc.EngineError for a command that never reached tmux, keeping a tmux-side rejection as data on the result - Reparent TmuxCommandNotFound under it, which widens the hierarchy and leaves existing handlers working - State the contract on TmuxEngine.run - Translate the dead-connection write in the control-mode example --- CHANGES | 8 +++ src/libtmux/engines/base.py | 13 ++++- src/libtmux/exc.py | 26 ++++++++- .../engines/test_control_mode_engine.py | 23 ++++---- tests/test_engine_error.py | 53 +++++++++++++++++++ 5 files changed, 112 insertions(+), 11 deletions(-) create mode 100644 tests/test_engine_error.py diff --git a/CHANGES b/CHANGES index 0a11d758c8..88f49ca53e 100644 --- a/CHANGES +++ b/CHANGES @@ -207,6 +207,14 @@ that was never taught to answer is a gap in a fixture, not an unreachable tmux. `returncode` comparisons, and {class}`~libtmux.common.tmux_cmd` carries the same two, so results read the same whichever layer produced them. +#### Engines report transport failure by name + +{exc}`~libtmux.exc.EngineError` names the case where a command never reached +tmux at all — a missing binary, a closed connection, a desynchronized protocol +— as distinct from tmux rejecting a command, which stays data on the result. +{exc}`~libtmux.exc.TmuxCommandNotFound` now subclasses it, so existing handlers +are unaffected and a caller can additionally catch the broader case. + #### Engines are validated where they are supplied {class}`~libtmux.Server` now rejects a non-engine at construction, naming the diff --git a/src/libtmux/engines/base.py b/src/libtmux/engines/base.py index 032ff3aa10..92ff1a0af0 100644 --- a/src/libtmux/engines/base.py +++ b/src/libtmux/engines/base.py @@ -603,7 +603,18 @@ class TmuxEngine(t.Protocol): """ def run(self, request: CommandRequest) -> CommandResult: - """Execute one tmux command and return its structured result.""" + """Execute one tmux command and return its structured result. + + A tmux-side failure is *data*: set ``returncode`` and ``stderr`` on the + result rather than raising, so a caller can inspect a rejected command. + + Raise :exc:`~libtmux.exc.EngineError` when the command never reached + tmux at all -- a missing binary, a closed connection, a desynchronized + protocol. That distinction is the whole reason a caller can tell "tmux + said no" from "tmux was never asked", so an engine that lets a raw + :exc:`OSError` escape instead leaves callers guarding + :exc:`~libtmux.exc.LibTmuxException` with nothing to catch. + """ ... def run_batch(self, requests: Sequence[CommandRequest]) -> list[CommandResult]: diff --git a/src/libtmux/exc.py b/src/libtmux/exc.py index b851b71222..433efa3ffb 100644 --- a/src/libtmux/exc.py +++ b/src/libtmux/exc.py @@ -95,7 +95,31 @@ class TmuxSessionExists(LibTmuxException): """Session does not exist in the server.""" -class TmuxCommandNotFound(LibTmuxException): +class EngineError(LibTmuxException): + """An engine could not carry a command to tmux. + + The failure of the *transport*, not of the command. A tmux-side failure -- + a bad target, an unknown option -- is data on a + :class:`~libtmux.engines.base.CommandResult` instead, carried in + ``returncode`` and ``stderr``. + + Engines raise this so a caller can tell "tmux said no" from "tmux was never + reached": a missing binary, a closed connection, a protocol desync. The + shipped engines raise :exc:`TmuxCommandNotFound`, which is one of these. + + Examples + -------- + >>> from libtmux import exc + >>> issubclass(exc.TmuxCommandNotFound, exc.EngineError) + True + >>> raise exc.EngineError("control connection closed") + Traceback (most recent call last): + ... + libtmux.exc.EngineError: control connection closed + """ + + +class TmuxCommandNotFound(EngineError): """Application binary for tmux not found.""" diff --git a/tests/examples/engines/test_control_mode_engine.py b/tests/examples/engines/test_control_mode_engine.py index 1434c241c0..a23bab9182 100644 --- a/tests/examples/engines/test_control_mode_engine.py +++ b/tests/examples/engines/test_control_mode_engine.py @@ -20,6 +20,7 @@ import pytest +from libtmux import exc from libtmux.engines import ( CommandResult, ServerConnection, @@ -135,20 +136,24 @@ def run(self, request: CommandRequest) -> CommandResult: returns it -- measured, and the opposite of what "in flight" suggests. What is lost is a command tmux never answered. - The gap this leaves is the tmux *server* going away: writing to the - dead connection then raises :exc:`BrokenPipeError`, an - :exc:`OSError` rather than a - :exc:`~libtmux.exc.LibTmuxException`, so a caller guarding against - libtmux errors does not catch it. Translating that, and backing off - rather than reconnecting in a tight loop, is what a hardened engine - adds. + When the tmux server itself goes away the write fails, and that is + translated to :exc:`~libtmux.exc.EngineError` so a caller guarding + against libtmux errors catches it rather than a bare + :exc:`BrokenPipeError`. Backing off instead of reconnecting in a tight + loop is what a hardened engine still adds. """ if self._process.poll() is not None: self.reconnects += 1 self._spawn() assert self._process.stdin is not None - self._process.stdin.write(render_control_line(request.args) + "\n") - self._process.stdin.flush() + try: + self._process.stdin.write(render_control_line(request.args) + "\n") + self._process.stdin.flush() + except OSError as error: + # The tmux server went away. Translate, so a caller guarding + # LibTmuxException catches it instead of a bare BrokenPipeError. + msg = "control connection closed" + raise exc.EngineError(msg) from error stdout, returncode = self._read_block() return CommandResult( cmd=("tmux", "-C", *request.args), diff --git a/tests/test_engine_error.py b/tests/test_engine_error.py new file mode 100644 index 0000000000..bd24d326e7 --- /dev/null +++ b/tests/test_engine_error.py @@ -0,0 +1,53 @@ +"""Engines report transport failure as a libtmux exception.""" + +from __future__ import annotations + +import typing as t + +import pytest + +from libtmux import exc +from libtmux.engines import CommandResult, SubprocessEngine +from libtmux.server import Server + +if t.TYPE_CHECKING: + from libtmux.engines import CommandRequest + + +def test_engine_error_is_a_libtmux_exception() -> None: + """Catching LibTmuxException still catches an engine failure.""" + assert issubclass(exc.EngineError, exc.LibTmuxException) + + +def test_missing_binary_is_an_engine_error() -> None: + """A missing tmux is a transport failure, so it answers to both names.""" + assert issubclass(exc.TmuxCommandNotFound, exc.EngineError) + + engine = SubprocessEngine.of("/nonexistent/tmux") + with pytest.raises(exc.EngineError): + Server(engine=engine).cmd("list-sessions") + + +def test_existing_handlers_keep_working() -> None: + """Widening the hierarchy must not break code catching the old name.""" + engine = SubprocessEngine.of("/nonexistent/tmux") + with pytest.raises(exc.TmuxCommandNotFound): + Server(engine=engine).cmd("list-sessions") + + +def test_an_engine_may_raise_engine_error_directly() -> None: + """A third-party engine has a name to raise when its transport dies.""" + + class DeadTransport: + def run(self, request: CommandRequest) -> CommandResult: + msg = "connection lost" + raise exc.EngineError(msg) + + def run_batch( + self, + requests: t.Sequence[CommandRequest], + ) -> list[CommandResult]: + return [self.run(r) for r in requests] + + with pytest.raises(exc.EngineError, match="connection lost"): + Server(engine=DeadTransport()).cmd("list-sessions") From d0849932308ae4a4e2faac5c0a2a771047981d45 Mon Sep 17 00:00:00 2001 From: Tony Narlock Date: Wed, 12 Aug 2026 18:28:38 -0500 Subject: [PATCH 26/32] Engines(fix[example]): Raise when a connection never establishes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit why: A failed attach was reported as an ordinary result. tmux answers `attach-session -t missing` with a %begin/%error block and exits, so the engine read returncode 1 and handed back something indistinguishable from tmux rejecting a command — while the connection was, in fact, dead. Reconnecting against a gone server also spawned a client per attempt, 11 for 10 commands, ~11ms each, every one reported as a failed command. what: - Check the handshake block's status and raise EngineError naming the session and what tmux said - Document _read_block's failure, and that the stream closing mid-block is a transport failure rather than an empty success --- .../engines/test_control_mode_engine.py | 40 +++++++++++++++++-- 1 file changed, 37 insertions(+), 3 deletions(-) diff --git a/tests/examples/engines/test_control_mode_engine.py b/tests/examples/engines/test_control_mode_engine.py index a23bab9182..3093cae17e 100644 --- a/tests/examples/engines/test_control_mode_engine.py +++ b/tests/examples/engines/test_control_mode_engine.py @@ -97,17 +97,35 @@ def _spawn(self) -> None: text=True, bufsize=1, ) - self._read_block() # tmux greets with a handshake block + # tmux greets with a handshake block either way; a failed attach + # terminates it with %error rather than %end and then exits. Reported + # as an ordinary result that is indistinguishable from tmux rejecting + # a command, so it is raised instead. + lines, returncode = self._read_block() + if returncode != 0: + detail = " ".join(lines) or "no reason given" + msg = f"could not attach to session {self._target!r}: {detail}" + raise exc.EngineError(msg) def _read_block(self) -> tuple[list[str], int]: - """Read one ``%begin``-delimited reply, returning its lines and status.""" + """Read one ``%begin``-delimited reply, returning its lines and status. + + Raises + ------ + :exc:`~libtmux.exc.EngineError` + The stream closed before tmux terminated the block. + """ assert self._process.stdout is not None lines: list[str] = [] returncode = 0 while True: line = self._process.stdout.readline() if not line: - break + # The stream ended before tmux terminated the block. Returning + # here would report the dead connection as a *successful* + # empty result, which is the worst of the three options. + msg = "control connection closed" + raise exc.EngineError(msg) line = line.rstrip("\n") if line.startswith("%begin"): lines = [] @@ -389,3 +407,19 @@ def test_connecting_adds_no_session(control_mode_server: Server) -> None: assert names, "the fixture session should be visible" assert not [name for name in names if str(name).startswith("_")] + + +def test_a_failed_reconnect_raises_rather_than_looking_like_an_error( + session: Session, +) -> None: + """A connection that never established is a transport failure, not a result. + + ``attach-session`` against a dead server makes tmux start a fresh one, which + has no such session, so the client exits quietly. Reported as a result it + would carry ``returncode`` 1 and be indistinguishable from tmux rejecting a + command. + """ + connection = ServerConnection.from_server(session.server) + + with pytest.raises(exc.EngineError, match="could not attach"): + ControlModeEngine(connection, "no_such_session_exists") From ab5ad9713b7d0b6bf59e5fdabdf1f1234a9b2705 Mon Sep 17 00:00:00 2001 From: Tony Narlock Date: Wed, 12 Aug 2026 18:29:37 -0500 Subject: [PATCH 27/32] Engines(fix[example]): Check the reply id tmux sends MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit why: Pipelining pairs a reply with its command by position alone. That is correct — measured, tmux answers in the order it was asked, including when one command fails — but the engine discarded the id tmux tags each reply with, so a desynchronized stream would have attributed every later reply to the wrong command, silently. what: - Keep the id from %begin and reject a block whose %end or %error carries a different one, as EngineError - Assert a batch's results pair with their commands, failure included --- .../engines/test_control_mode_engine.py | 47 +++++++++++++++++-- 1 file changed, 43 insertions(+), 4 deletions(-) diff --git a/tests/examples/engines/test_control_mode_engine.py b/tests/examples/engines/test_control_mode_engine.py index 3093cae17e..e0c0b180be 100644 --- a/tests/examples/engines/test_control_mode_engine.py +++ b/tests/examples/engines/test_control_mode_engine.py @@ -118,6 +118,7 @@ def _read_block(self) -> tuple[list[str], int]: assert self._process.stdout is not None lines: list[str] = [] returncode = 0 + block_id: str | None = None while True: line = self._process.stdout.readline() if not line: @@ -129,10 +130,21 @@ def _read_block(self) -> tuple[list[str], int]: line = line.rstrip("\n") if line.startswith("%begin"): lines = [] - elif line.startswith("%error"): - returncode = 1 - break - elif line.startswith("%end"): + block_id = line.split()[2] if len(line.split()) > 2 else None + elif line.startswith(("%end", "%error")): + # tmux tags each reply with the id it gave the command. Order + # is FIFO, so matching them is not required to pair a reply + # with its command -- but a mismatch means the stream has + # desynchronized, and every later reply would be attributed to + # the wrong command. + terminator_id = line.split()[2] if len(line.split()) > 2 else None + if block_id is not None and terminator_id != block_id: + msg = ( + f"control stream desynchronized: reply {terminator_id} " + f"terminates block {block_id}" + ) + raise exc.EngineError(msg) + returncode = 1 if line.startswith("%error") else 0 break elif line.startswith("%output"): # Notifications arrive interleaved with replies. Keeping them @@ -423,3 +435,30 @@ def test_a_failed_reconnect_raises_rather_than_looking_like_an_error( with pytest.raises(exc.EngineError, match="could not attach"): ControlModeEngine(connection, "no_such_session_exists") + + +def test_pipelined_replies_pair_with_their_commands( + control_mode_server: Server, +) -> None: + """A batch's results line up with the commands that produced them. + + Batching depends on tmux answering in the order it was asked, including + when one command fails. Verified against tmux rather than assumed: it tags + every reply with the command's id, and the engine rejects a block whose + terminator does not match its opener. + """ + from libtmux.common import dispatch_batch + + results = dispatch_batch( + control_mode_server.engine, + [ + ("display-message", "-p", "c0"), + ("display-message", "-p", "c1"), + ("kill-window", "-t", "@99999"), + ("display-message", "-p", "c2"), + ], + ) + + assert [r.stdout for r in results][:2] == [["c0"], ["c1"]] + assert not results[2].ok, "the failure keeps its own position" + assert results[3].stdout == ["c2"], "a failure does not shift later replies" From 3b14ece39021fd4ea8a91b09d695c34530016cd4 Mon Sep 17 00:00:00 2001 From: Tony Narlock Date: Wed, 12 Aug 2026 18:30:37 -0500 Subject: [PATCH 28/32] Engines(fix[example]): Satisfy the optional capability protocols MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit why: The example is what someone copies to write an engine, and it implemented only TmuxEngine. An engine built from it forgoes connection adoption — so Server(socket_name=..., engine=it) can reach the ambient tmux server — and has its tmux version resolved by running the binary rather than being asked. what: - Add connection, with_connection and tmux_version, so the template produces a well-behaved engine - Say in each docstring what omitting it costs, since all three fail quietly - Assert the example satisfies SupportsConnection and SupportsTmuxVersion --- .../engines/test_control_mode_engine.py | 45 +++++++++++++++++++ 1 file changed, 45 insertions(+) diff --git a/tests/examples/engines/test_control_mode_engine.py b/tests/examples/engines/test_control_mode_engine.py index e0c0b180be..e3a08e0472 100644 --- a/tests/examples/engines/test_control_mode_engine.py +++ b/tests/examples/engines/test_control_mode_engine.py @@ -217,6 +217,33 @@ def run_batch( ) return results + @property + def connection(self) -> ServerConnection: + """The tmux server this engine is attached to. + + With :meth:`with_connection` this satisfies + :class:`~libtmux.engines.base.SupportsConnection`, without which + :class:`~libtmux.Server` cannot bind an engine to its own socket -- so + an engine that omits it can silently reach the ambient tmux server. + """ + return self._connection + + def with_connection(self, connection: ServerConnection) -> ControlModeEngine: + """Return an engine attached to *connection* instead. + + A new engine rather than a rebind: a control connection is a live + process, and moving it would mean tearing one down mid-flight. + """ + return type(self)(connection, self._target) + + def tmux_version(self) -> str | None: + """Report the tmux version, satisfying ``SupportsTmuxVersion``. + + Without it the version-gated listing format is resolved by running + ``tmux -V``, which works only because a binary happens to be present. + """ + return self._connection.tmux_version() + def close(self) -> None: """Shut the connection down.""" try: @@ -462,3 +489,21 @@ def test_pipelined_replies_pair_with_their_commands( assert [r.stdout for r in results][:2] == [["c0"], ["c1"]] assert not results[2].ok, "the failure keeps its own position" assert results[3].stdout == ["c2"], "a failure does not shift later replies" + + +def test_the_example_satisfies_the_optional_protocols( + control_mode_server: Server, +) -> None: + """An engine copied from here should be a well-behaved one. + + The optional capabilities are easy to omit, and omitting them fails + quietly: no connection adoption, and the tmux version resolved by running + the binary rather than asking the engine. + """ + from libtmux.engines import SupportsConnection, SupportsTmuxVersion + + engine = control_mode_server.engine + + assert isinstance(engine, SupportsConnection) + assert isinstance(engine, SupportsTmuxVersion) + assert engine.tmux_version() is not None From bd7afbe5c3f1688e7afcdb065a5202349a758814 Mon Sep 17 00:00:00 2001 From: Tony Narlock Date: Wed, 12 Aug 2026 18:31:12 -0500 Subject: [PATCH 29/32] Docs(engines): Point readers at the worked engines MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit why: The guide taught the protocol and then left readers to invent the rest. Two complete engines live in the test suite encoding the traps that cost the most to rediscover, and nothing outside a changelog line mentioned them — so a reader wrote a toy from the sketch and met the traps one at a time. what: - Name both examples and what each demonstrates, next to the section that teaches writing one - List the traps they encode, so the guide is useful even to someone who never opens them --- docs/topics/engines.md | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/docs/topics/engines.md b/docs/topics/engines.md index e1e51d6a28..bc0c73ff4a 100644 --- a/docs/topics/engines.md +++ b/docs/topics/engines.md @@ -133,6 +133,28 @@ out: [('list-sessions',)] ``` +### Worked examples + +The engine above runs nothing. Two complete ones live in the test suite, written +to be read and copied: + +`tests/examples/engines/test_control_mode_engine.py` holds one long-lived +`tmux -C` connection instead of forking per command. It is where the traps are +recorded — a control client must *attach* before tmux pushes it anything; +opening it with `new-session -A` works but leaves a real session behind in +`server.sessions`; tmux tags every reply with the command's id, and a mismatch +means the stream has desynchronized; and a connection that never established +answers with an error block rather than closing, so it must be raised rather +than returned as an ordinary failed result. + +`tests/examples/engines/test_push_notifications.py` adds a reader thread, so a +pane's output arrives while the caller is idle rather than the next time a +command runs. + +Both implement the optional capabilities below. An engine that omits them still +works, but quietly gives up connection adoption and version reporting, so start +from these rather than from the minimal sketch above. + ## Testing without tmux Writing a fake that *simulates* tmux is a trap. A listing query asks tmux for From 6a085f89d37c6e124a0ff3a6b265d178dfe74571 Mon Sep 17 00:00:00 2001 From: Tony Narlock Date: Wed, 12 Aug 2026 18:49:16 -0500 Subject: [PATCH 30/32] Engines(test): Docstring the engine test cases why: The suite's docstring rule failed on the new engine tests, which turns the branch's own lint gate red. what: - Say what each registry, dispatch, and result-compat case proves - Describe what test_result_compat covers rather than when it was written --- tests/test_dispatch.py | 2 ++ tests/test_engine_registry.py | 6 ++++++ tests/test_result_compat.py | 5 ++++- 3 files changed, 12 insertions(+), 1 deletion(-) diff --git a/tests/test_dispatch.py b/tests/test_dispatch.py index d7c3665b56..76c3a65151 100644 --- a/tests/test_dispatch.py +++ b/tests/test_dispatch.py @@ -25,10 +25,12 @@ def __init__(self, stdout: Sequence[str] = ()) -> None: self._stdout = tuple(stdout) def run(self, request: CommandRequest) -> CommandResult: + """Count the call and answer with the canned stdout.""" self.calls += 1 return CommandResult(cmd=("tmux", *request.args), stdout=self._stdout) def run_batch(self, requests: Sequence[CommandRequest]) -> list[CommandResult]: + """Answer each request through :meth:`run`, counting every one.""" return [self.run(r) for r in requests] diff --git a/tests/test_engine_registry.py b/tests/test_engine_registry.py index 22c8831acf..14acf31f2b 100644 --- a/tests/test_engine_registry.py +++ b/tests/test_engine_registry.py @@ -20,16 +20,19 @@ def test_builtin_engines_are_registered() -> None: + """The built-in subprocess engine resolves by name.""" assert "subprocess" in available_engines() assert isinstance(create_engine("subprocess"), SubprocessEngine) def test_available_engines_is_sorted() -> None: + """Names come back sorted, so a CLI can list them as given.""" names = available_engines() assert list(names) == sorted(names) def test_unknown_engine_fails_closed_and_lists_options() -> None: + """An unknown name raises, naming both it and the registered alternatives.""" with pytest.raises(exc.LibTmuxException) as excinfo: create_engine("does-not-exist") message = str(excinfo.value) @@ -38,11 +41,14 @@ def test_unknown_engine_fails_closed_and_lists_options() -> None: def test_factory_receives_kwargs() -> None: + """Keyword arguments reach the factory rather than being dropped.""" engine = create_engine("subprocess", server_args=("-Lfromregistry",)) assert engine.server_args == ("-Lfromregistry",) # type: ignore[attr-defined] def test_third_party_can_register() -> None: + """A registered engine resolves by name, and unregistering removes it.""" + class Custom: def run(self, request: CommandRequest) -> CommandResult: return CommandResult(cmd=("tmux", *request.args)) diff --git a/tests/test_result_compat.py b/tests/test_result_compat.py index 06def5d208..6c5deba462 100644 --- a/tests/test_result_compat.py +++ b/tests/test_result_compat.py @@ -1,4 +1,4 @@ -"""Tests written before the implementation exists.""" +"""Tests for ``CommandResult``'s compatibility with the old list-shaped output.""" from __future__ import annotations @@ -16,6 +16,7 @@ def test_output_is_a_list_so_error_paths_keep_raising() -> None: def test_output_compares_equal_to_both_list_and_tuple() -> None: + """Output equals a list of the same items, so old assertions keep passing.""" r = CommandResult(cmd=("tmux",), stdout=("a", "b")) assert r.stdout == ["a", "b"] assert r.stdout == ("a", "b") # type: ignore[comparison-overlap] @@ -24,12 +25,14 @@ def test_output_compares_equal_to_both_list_and_tuple() -> None: def test_output_is_read_only() -> None: + """Mutating output raises rather than corrupting a shared result.""" r = CommandResult(cmd=("tmux",), stdout=("a",)) with pytest.raises(TypeError): r.stdout.append("b") # type: ignore[attr-defined] def test_result_is_hashable_and_comparable() -> None: + """Two results built from the same fields are equal and hash alike.""" a = CommandResult(cmd=("tmux",), stdout=("a",)) b = CommandResult(cmd=("tmux",), stdout=("a",)) assert a == b From 74d0639c256971b5ab67a8b21c97843762b5bf29 Mon Sep 17 00:00:00 2001 From: Tony Narlock Date: Wed, 12 Aug 2026 18:49:38 -0500 Subject: [PATCH 31/32] Engines(fix[registry]): Report a skipped engine why: A distribution whose engine will not import was skipped in silence, so a name that should resolve simply was not there, with nothing to explain why. Reporting the failure is also what makes catching a third party's arbitrary exception legitimate. what: - Warn with the traceback and the entry-point name, then carry on - Cover the skip-and-report path with a deliberately broken entry point --- src/libtmux/engines/registry.py | 13 +++++++++-- tests/test_engine_registry.py | 39 +++++++++++++++++++++++++++++++++ 2 files changed, 50 insertions(+), 2 deletions(-) diff --git a/src/libtmux/engines/registry.py b/src/libtmux/engines/registry.py index 5af1fb7f17..fca7c2c6b8 100644 --- a/src/libtmux/engines/registry.py +++ b/src/libtmux/engines/registry.py @@ -13,6 +13,7 @@ from __future__ import annotations +import logging import typing as t from importlib import metadata @@ -23,6 +24,8 @@ if t.TYPE_CHECKING: from libtmux.engines.base import TmuxEngine +logger = logging.getLogger(__name__) + ENGINE_ENTRY_POINT_GROUP = "libtmux.engines" """Entry-point group a packaged engine registers under.""" @@ -110,8 +113,14 @@ def _load_entry_points() -> None: continue try: _registry[entry_point.name] = entry_point.load() - except Exception: # noqa: BLE001 - a third party's import is not ours to trust - continue + except Exception: + # A third party's import is not ours to trust, so catch anything it + # raises -- but say so, because a silently missing engine is worse. + logger.warning( + "engine entry point failed to load", + exc_info=True, + extra={"tmux_engine_name": entry_point.name}, + ) def available_engines() -> tuple[str, ...]: diff --git a/tests/test_engine_registry.py b/tests/test_engine_registry.py index 14acf31f2b..82fbc6e622 100644 --- a/tests/test_engine_registry.py +++ b/tests/test_engine_registry.py @@ -2,6 +2,8 @@ from __future__ import annotations +import logging +import types import typing as t import pytest @@ -13,6 +15,7 @@ available_engines, create_engine, register_engine, + registry, ) if t.TYPE_CHECKING: @@ -46,6 +49,42 @@ def test_factory_receives_kwargs() -> None: assert engine.server_args == ("-Lfromregistry",) # type: ignore[attr-defined] +def test_broken_entry_point_is_skipped_and_reported( + monkeypatch: pytest.MonkeyPatch, + caplog: pytest.LogCaptureFixture, +) -> None: + """A distribution whose engine will not import is skipped, but says so.""" + + class BrokenEntryPoint: + name = "broken-for-test" + + def load(self) -> t.NoReturn: + msg = "this engine's import is broken" + raise ImportError(msg) + + monkeypatch.setattr(registry, "_entry_points_loaded", False) + monkeypatch.setattr( + registry, + "metadata", + types.SimpleNamespace(entry_points=lambda group: [BrokenEntryPoint()]), + ) + + with caplog.at_level(logging.WARNING, logger="libtmux.engines.registry"): + names = available_engines() + + assert "broken-for-test" not in names + assert "subprocess" in names, "one bad engine must not hide the others" + + reported = [ + record + for record in caplog.records + if getattr(record, "tmux_engine_name", None) == "broken-for-test" + ] + assert len(reported) == 1 + assert reported[0].levelno == logging.WARNING + assert reported[0].exc_info is not None, "the traceback is what makes it useful" + + def test_third_party_can_register() -> None: """A registered engine resolves by name, and unregistering removes it.""" From 1fa02d434125815f9880c59b1fe4b5278be6d3e4 Mon Sep 17 00:00:00 2001 From: Tony Narlock Date: Wed, 12 Aug 2026 19:12:00 -0500 Subject: [PATCH 32/32] CHANGES(docs): Reference the engines PR why: The changelog convention carries the pull request ref in each deliverable heading, and the number only existed once the pull request was opened. what: - Add the ref to the fifteen headings this branch introduced --- CHANGES | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/CHANGES b/CHANGES index 88f49ca53e..6210b27ab7 100644 --- a/CHANGES +++ b/CHANGES @@ -47,7 +47,7 @@ _Notes on the upcoming release will go here._ ### Breaking changes -#### A bare `";"` argument is now literal data +#### A bare `";"` argument is now literal data (#738) tmux reads a trailing `;` on a command argument as a command boundary, so a `;` passed to {meth}`Server.cmd() ` as a *separator* worked by @@ -71,7 +71,7 @@ server.cmd( See {ref}`migration-0-63-command-separator`. -#### Commands return `CommandResult` +#### Commands return `CommandResult` (#738) {meth}`Server.cmd() ` and its `Session`, `Window`, and `Pane` counterparts return {class}`~libtmux.engines.base.CommandResult` rather than @@ -83,14 +83,14 @@ now read-only, so a caller that mutated a result in place gets `tmux_cmd` is unchanged and still constructible directly. See {ref}`migration-0-63-command-result`. -#### `tmux_cmd.process` deprecated +#### `tmux_cmd.process` deprecated (#738) {attr}`libtmux.common.tmux_cmd.process` is now a deprecated property. Reading it warns, and it raises {exc}`~libtmux.exc.LibTmuxException` under an engine that forks no process. Use `returncode`, `stdout`, and `stderr` on the result, which are unchanged. -#### `raise_if_dead()` no longer echoes tmux's error +#### `raise_if_dead()` no longer echoes tmux's error (#738) {meth}`Server.raise_if_dead() ` previously let tmux write its message straight to the terminal. It now captures that text onto @@ -99,7 +99,7 @@ unchanged. ### What's new -#### Pluggable command engines +#### Pluggable command engines (#738) Every tmux command libtmux runs now goes through an *engine* — an object that takes a rendered argv and returns a structured result. The default, @@ -126,7 +126,7 @@ lookup instead of re-walking `$PATH` for every command. See {ref}`engines` for the guide and {ref}`engines-api` for the reference. -#### Several commands in one round trip +#### Several commands in one round trip (#738) {meth}`Server.cmd_batch() ` runs a sequence of commands and returns one result each, in order; a tmux-side failure is data on its own @@ -141,7 +141,7 @@ over single dispatches. A stateless engine loops internally and behaves as repeated calls would; a persistent-connection engine writes every command before waiting for the first reply, which is where the round trips collapse. -#### Async engines can run commands +#### Async engines can run commands (#738) {class}`~libtmux.engines.asyncio.AsyncSubprocessEngine` awaits the tmux binary rather than blocking on it. {func}`~libtmux.common.adispatch` and @@ -150,7 +150,7 @@ through any {class}`~libtmux.engines.base.AsyncTmuxEngine`, applying the same adaptations as their synchronous counterparts. {class}`~libtmux.Server` remains synchronous and refuses an async engine, naming the reason. -#### A persistent-connection engine is buildable on the public API +#### A persistent-connection engine is buildable on the public API (#738) {func}`~libtmux.engines.base.render_control_line` and {func}`~libtmux.engines.base.unescape_control_output` encode a command for @@ -160,7 +160,7 @@ connection can be written entirely against the public engine API — including t format-heavy listing queries the object API depends on. See `tests/examples/engines/test_control_mode_engine.py` for a worked example. -#### Testing without a tmux server +#### Testing without a tmux server (#738) {class}`~libtmux.engines.record.RecordingEngine` wraps a real engine and keeps what tmux answered; {class}`~libtmux.engines.record.ReplayEngine` serves those @@ -199,7 +199,7 @@ the version-gated `-F` template is otherwise resolved by running `tmux -V`. reached, but no longer swallow {exc}`~libtmux.exc.UnscriptedCommand` — an engine that was never taught to answer is a gap in a fixture, not an unreachable tmux. -#### Result objects report success directly +#### Result objects report success directly (#738) {attr}`CommandResult.ok ` and {meth}`CommandResult.raise_for_status() @@ -207,7 +207,7 @@ that was never taught to answer is a gap in a fixture, not an unreachable tmux. `returncode` comparisons, and {class}`~libtmux.common.tmux_cmd` carries the same two, so results read the same whichever layer produced them. -#### Engines report transport failure by name +#### Engines report transport failure by name (#738) {exc}`~libtmux.exc.EngineError` names the case where a command never reached tmux at all — a missing binary, a closed connection, a desynchronized protocol @@ -215,7 +215,7 @@ tmux at all — a missing binary, a closed connection, a desynchronized protocol {exc}`~libtmux.exc.TmuxCommandNotFound` now subclasses it, so existing handlers are unaffected and a caller can additionally catch the broader case. -#### Engines are validated where they are supplied +#### Engines are validated where they are supplied (#738) {class}`~libtmux.Server` now rejects a non-engine at construction, naming the missing method, rather than failing with an {exc}`AttributeError` inside the @@ -230,13 +230,13 @@ hold a persistent connection. ### Fixes -#### A trailing `;` in a command argument is no longer swallowed +#### A trailing `;` in a command argument is no longer swallowed (#738) `pane.cmd("send-keys", "echo hello;")` sent `echo hello` — tmux consumed the final `;` as a command boundary and the character never reached the pane. Arguments are now escaped for tmux's parser, so the `;` arrives as typed. -#### Listing queries honor `config_file` and `colors` +#### Listing queries honor `config_file` and `colors` (#738) {meth}`Server.raise_if_dead() ` and the listing queries behind {attr}`~libtmux.Server.sessions` built their own connection flags @@ -247,7 +247,7 @@ share one connection. A `colors=` value other than `256` or `88` raises ### Documentation -#### Engines guide and API reference +#### Engines guide and API reference (#738) {ref}`engines` covers what an engine is, writing one, the optional capability protocols, and explicit command separators. {ref}`engines-api` documents the