Repository navigation
Conversation
jollm
marked this pull request as ready for review
November 1, 2026 04:37
jollm
force-pushed
the
feature-ghostel-eshell-mode
branch
from
November 1, 2026 14:14
13a10dd to
521e078
Compare
added 6 commits
November 1, 2026 06:26
An embedded terminal has to draw into part of a host buffer while the text above and below it stays intact, but the renderer assumed it owned the entire buffer: it erased the buffer on init/clear and resolved every position from point-min. Give the renderer an optional, marker-backed region, set through a new module entry `ghostel--set-region TERM START END` (nil keeps the current whole-buffer behaviour): - Positions are mapped through a RowMap (terminal row base offset by the region's buffer base). Saved window/mouse markers and scrollback eviction (evictScrollback) now anchor at the region start rather than buffer point 1. - `clear` deletes the region instead of calling `erase-buffer`, so a redraw cannot wipe text outside the terminal. - `init` now only applies a pending resize. Creating a terminal inside an existing buffer must not touch the host's text; clearing is deferred to the first real render. - `atRenderEnd` compares against the region's end marker instead of `eobp`. For a whole-buffer terminal the region is nil and every offset collapses to the previous values, so behaviour there is unchanged. This change only exercises ghostel--set-region through testing. The Elisp side to drive it will follow.
Adds the Elisp side of in-buffer terminals (`ghostel-embedded`), built on the renderer's region support. One live terminal is allowed per host buffer; a `ghostel-embedded` struct holds the term, process, region markers, size, redraw timer, and saved host state. Lifecycle: `start` opens a region at point-max on a fresh line, `feed` writes PTY bytes to the VT and schedules one coalesced redraw, `finish` freezes the result into ordinary text (discard any alternate screen, draw a final frame, remove the live-region `read-only` property with `inhibit-read-only` bound explicitly, trim trailing blank rows, ensure a trailing newline), then restores the host keymap and drops the handle. Behaviour worth calling out: - Following output: a full-screen TUI anchors the window at the region start (`set-window-start`); normal scrolling output recentres with `recenter -1` so the cursor row stays at the bottom. An earlier formula counted the terminal's still-blank rows below the cursor, which made short output jump to the top of the window the moment a progress bar completed. - Input modes: semi-char, char, and Emacs. The terminal maps live in `emulation-mode-map-alists`, because minor-mode maps (e.g. Eshell's history mode binding the arrow keys) outrank a buffer's local map. - Mouse bindings live in those maps too. They are deliberately not a `keymap` text property on the region: that would replace the active input-mode map for every key press there, so terminal keys fell through to the global map. - Hosts can react to rendered frames via `ghostel-embedded-redraw-hook` (Eshell can use it to size a TUI to the whole window). This allows embedding terminals (at most one per buffer). The Eshell specific functionality will follow.
Adds `ghostel-eshell-mode`, a global opt-in mode that runs interactive external Eshell commands (including visual ones such as vim, htop, less) in a terminal region inside the Eshell buffer. Output interleaves with normal prompts and the final screen is frozen into ordinary text on exit. While enabled it supersedes `ghostel-eshell-visual-command-mode`; piped, redirected and subjob commands, and remote buffers, keep Eshell's normal output path. A pipeline element is left to Eshell's visual handling (its stdin is a pipe) and gets a one-line hint, since Eshell then drops the piped text. Spawning: `eshell-gather-process-output` is wrapped so the command is prefixed with a small `stty` shim that sizes the PTY before the program starts (Eshell does not), and TERM/TERMINFO are bound for that process only. The process is attached on `eshell-exec-hook`; the original filter and sentinel are saved, replaced, then restored and chained on exit. `ghostel--eshell-attach-process` also sets the process coding to `binary`. Eshell otherwise decodes the PTY stream, so a multibyte sequence cut in half by an interrupt (a box-drawing glyph truncated by C-c C-c) arrives as a multibyte string containing a raw byte. The module's string extraction rejects that as non-Unicode, which surfaced as `ExtractStringFailed` and dropped output. Reading raw bytes matches the standalone terminal and avoids the decode entirely. Sizing: the terminal fills the space below the prompt, recomputed on window size changes; when a program switches to the alternate screen it is resized to the whole window and restored on exit. `eshell-delete-output` (C-o) reads `eshell-last-input-end` as the start of the last output. After a signal, Eshell inserts its message and moves the markers so that C-o would try to delete the read-only prompt. The sentinel now re-points `eshell-last-input-end` at the frozen region's start and `eshell-last-output-start` at its end, once per command, so C-o stays confined to the terminal's rows. The mode-line lighter shows the active input mode; `lisp/ghostel.el` skips `ghostel--anchor-window` while an embedded session owns the buffer. It includes mode-line mouse actions allowing mouse-1 and mouse-3 to change the input mode.
Document `ghostel-eshell-mode`: how to enable it, how it relates to `ghostel-eshell-visual-command-mode`, when it defers to Eshell's normal output path, and the `stty` fallback option. Add a CHANGELOG entry for the mode and the embedded-terminal layer. README.md links to the relevant README.org section.
This the one from-scratch, non-test file for the in-buffer eshell feature.
This gives the user more control of the mode-line lighter text.
jollm
force-pushed
the
feature-ghostel-eshell-mode
branch
from
November 1, 2026 14:27
521e078 to
10ff683
Compare
|
Could only test this on Windows today - which does not go far due to the stated limitation: Embedded terminals use Emacs PTY, not the native PTY. Is that a fundamental issue or would it be possible to use native PTY same as the normal ghostel buffers? |
`ghostel--events-filter' always invalidated the whole buffer, which is right for a standalone terminal but wrong for one embedded in a host buffer because the host has no `ghostel--term', so the reader's update events would never schedule a redraw of the embedded region. This lets the pipe carry a `ghostel--invalidate-function' property and call it instead when present. The default remains `ghostel--invalidate'.
The embedded Eshell terminal needs the same pipe-plus-native-child setup as a standalone buffer, but with its own event filter and sentinel and without the standalone exit cleanup. Move the shared part (create the pipe, spawn the child, record its pid) into `ghostel--spawn-native-pipe', parameterised by filter and sentinel. `ghostel--spawn-via-native' keeps the standalone behaviour: the whole-buffer event filter, the "finished"/"exited abnormally" sentinel, `ghostel--pid' and the kill-buffer hook. `ghostel--spawn-native-pipe' also gains an optional NAME so the embedded spawner can identify its pipes in `list-processes'.
Prepare the embedded layer for a native child. Add a `native-p' field to the session and branch where the two PTY kinds differ: - resizing: `set-process-window-size' only applies to an Emacs PTY. the renderer resizes a native one on the next redraw. - input: a native PTY is written through `ghostel--write-pty', an Emacs one through `process-send-string'. - cleanup: a native child is stopped with `ghostel--kill-native-process' rather than by deleting its process object.
Emacs cannot give an external process a real terminal on MS-Windows, so in-buffer Eshell terminals did not work there: Eshell's process is pipe-backed and there is no `stty' to size a PTY. Run the command in ghostel's native PTY (ConPTY on Windows) instead, as a standalone ghostel buffer does. Add `ghostel-eshell-use-native-pty' (`auto' means native on MS-Windows, Emacs's PTY elsewhere) and `ghostel--eshell-native-p'. For a native command, `eshell-gather-process-output' returns the event pipe from the `make-process' interception instead of letting Emacs spawn. The embedded session is created first so the reader always has a region to draw into. The native path skips the `stty' shim and the `binary' coding (the native pipe uses UTF-8 coding), closes a redirected stderr pipe before the Eshell sentinel waits on it, and restores the child's exit code in the buffer-local `eshell-last-command-status', which the pipe itself reports as 0. Remote, piped, redirected and Emacs 28 commands keep the existing Eshell-PTY path. Adds tests for gating, output and exit status, input, and cleanup.
Update README.org, README.md and the CHANGELOG to document `ghostel-eshell-use-native-pty' options native if on Windows (auto), native (t), or Emacs (nil).
Author
Thanks for pointing that out about Windows! I've added a native pty code path for embedded terminals. The additional changes for native support only involve elisp, nothing in zig code. I've tested it on Linux, but at the moment I have no way to test on Windows. Let me know if you do get a chance. |
|
Tested briefly, this now works on Windows, nice! I ran emacs -nw -Q from eshell and it will show the first 6 or so lines from terminal emacs. That feels a bit weird as it doesn't show its full frame/window. If you manage to get to its minibuffer through eg. ESC-x it will actually show the full terminal emacs window. There are some rough edges like that. Also eg. doing pacman -Syu (MSYS2) has some jumping up & down as the progress bars update and new lines are appended. It's difficult to say what it should actually do differently, but it feel like having some more space below the visual program in general would help. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I frequently use eat-eshell-mode, and so I was inspired by ghostel issue 426 to attempt to implement a similar feature for ghostel.
This add a global, opt-in minor mode,
ghostel-eshell-mode.I used an AI assistant (deepseek-flash) for the implementation and tests. I reviewed the changes, edited where necessary, performed manual testing, and the full suite and lint pass.
The most significant change to existing code is that the renderer is made aware of a RegionPos render region holding start and end marker positions. For the non-embedded case this will be nil (null for Zig) and behavior unchanged. For the embedded case the RegionPos is used to determine a row to line mapping. The start of RegionPos is fixed and the end grows with output.
One Emacs module function is added,
ghostel--set-region, which stores the region's start and end markers in the buffer-localghostel--render-region, marking the terminal as region-aware. On the Elisp side, ghostel-embedded.el contains logic for embedded terminals. There is one change to ghostel.el to avoidghostel--anchor-windowfor embedded sessions. The Eshell in-buffer support logic is added to ghostel-eshell.el. Elisp tests and native tests are added. Readmes are updated.Enable with
(ghostel-eshell-mode 1)Change input modes with:
Or click in the mode-line (see below).
Notable limitations:
Detailed description of user facing changes
Adds
ghostel-eshell-mode, a global opt-in mode that runs interactive external Eshell commands (including visual ones such as vim, htop, less) in a terminal region inside the Eshell buffer. Output interleaves with normal prompts and the final screen is frozen into ordinary text on exit. While enabled it supersedesghostel-eshell-visual-command-mode; piped, redirected and subjob commands, and remote buffers, keep Eshell's normal output path. A pipeline element is left to Eshell's visual handling (its stdin is a pipe) and gets a one-line hint, since Eshell then drops the piped text.Spawning:
eshell-gather-process-outputis wrapped so the command is prefixed with a smallsttyshim that sizes the PTY before the program starts (Eshell does not), and TERM/TERMINFO are bound for that process only. The process is attached oneshell-exec-hook; the original filter and sentinel are saved, replaced, then restored and chained on exit.ghostel--eshell-attach-processalso sets the process coding tobinary. Eshell otherwise decodes the PTY stream, so a multibyte sequence cut in half by an interrupt (a box-drawing glyph truncated by C-c C-c) arrives as a multibyte string containing a raw byte. The module's string extraction rejects that as non-Unicode, which surfaced asExtractStringFailedand dropped output. Reading raw bytes matches the standalone terminal and avoids the decode entirely.Sizing: the terminal fills the space below the prompt, recomputed on window size changes; when a program switches to the alternate screen it is resized to the whole window and restored on exit.
eshell-delete-output(C-o) readseshell-last-input-endas the start of the last output. After a signal, Eshell inserts its message and moves the markers so that C-o would try to delete the read-only prompt. The sentinel now re-pointseshell-last-input-endat the frozen region's start andeshell-last-output-startat its end, once per command, so C-o stays confined to the terminal's rows.The mode-line lighter shows the active input mode;
lisp/ghostel.elskipsghostel--anchor-windowwhile an embedded session owns the buffer. It includes mode-line mouse actions allowing mouse-1 and mouse-3 to change the input mode.