Fix how neat handles late output, errors, stdin and sessions

CHANGELOG.md

@@ -6,6 +6,16 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
 ## [Unreleased]
 
+### Changed
+
+- [#2](https://github.com/nrepl/neat/pull/2): Handle the less common parts of the protocol properly:
+  - Late and id-less output shows up above the REPL prompt, and so do the results of source-buffer evals.
+  - Error statuses that come without an `err` (`namespace-not-found`, `unknown-session`, `unknown-op`) are reported, and an `unknown-session` reply offers to clone a new session.
+  - The stdin prompt can send end of input with `C-c C-d`.
+  - Completion, eldoc and interrupt check `describe` first, and completion and eldoc back off on servers that answer one request at a time.
+  - Quitting the REPL closes its session on the server.
+  - Requests still pending when a connection drops get a `connection-closed` status instead of `interrupted`, and `neat-bencode-encode` signals on nil.
+
 ## [0.1.0] - 2026-05-20
 
 ### Added

README.md

@@ -33,25 +33,24 @@ edges - this is the start of the road, not the end.
 ecosystem rather than a Clojure-only protocol. That effort has three
 strands:
 
-1. **An official nREPL specification.** Today the [nREPL][nrepl] project
+1. An official nREPL specification. Today the [nREPL][nrepl] project
    is the de facto spec; a formal version is being drafted at
    [nrepl/spec.nrepl.org][spec]. `neat` aims to keep pressure on the spec
    to stay genuinely language-agnostic by being a client that refuses to
    silently assume Clojure.
-2. **Reference clients.** A spec without independent client
+2. Reference clients. A spec without independent client
    implementations is wishful thinking. `neat` is one such reference
    client, intentionally built on Emacs builtins and free of
    Clojure-specific helpers, so it can act as a baseline for what
    "compliant" should mean on the client side.
-3. **A compatibility test suite.** The parameterised integration suite
-   under [`test/neat-integration-test.el`](test/neat-integration-test.el)
-   already runs the same assertions against multiple servers (Clojure,
-   Babashka, Basilisp), and divergences between them get surfaced as
-   real findings rather than mysterious bugs. The long-term goal is
-   to grow this into a portable suite any nREPL server can self-check
-   against.
+3. A compatibility test suite. That one lives in its own project,
+   [proof][proof]. It talks to a server the way clients do, runs a set
+   of checks and tells you what's broken and which clients it breaks.
+   If you work on an nREPL server, that's the tool to reach for.
+   neat's own integration suite only tests the client.
 
 [spec]: https://github.com/nrepl/spec.nrepl.org
+[proof]: https://github.com/nrepl/proof
 
 ## Installation
 
@@ -205,23 +204,28 @@ the source (Clojure's `(ns foo.bar)`, etc.), swap in a parser via
 ### Eldoc, completion, or `M-.` quietly do nothing
 
 The CAPF, eldoc backend, and xref backend all rely on the standard
-`completions` and `lookup` nREPL ops. If your server doesn't implement
-them, our sync helpers return nil and we defer silently - that's by
-design, a language-agnostic client can't assume anything.
+`completions` and `lookup` nREPL ops. neat checks the ops the server
+lists in its `describe` reply, and when one of them is missing these
+features stay out of the way - that's by design, a language-agnostic
+client can't assume anything.
 
 Diagnosis: turn on the message log (next-to-last entry below) and
-look for `unknown-op` in the status.
+look at the `ops` in the `describe` reply, or for `unknown-op` in a
+status.
 
 ### Evaluation just sits there
 
 Three usual suspects:
 
 - The eval is reading from stdin. Look at the minibuffer for a
-  `stdin:` prompt and answer it. `C-g` interrupts the read.
+  `stdin` prompt and answer it, REPL buffer or not. `C-c C-d` there
+  sends end-of-file, and `C-g` interrupts the eval (or sends
+  end-of-file, if the server can't interrupt).
 - The connection died. The mode-line shows `[closed]` and a
   `;; connection closed` line appears in the REPL buffer.
 - The eval is actually running, just slowly. `C-c C-c` in the REPL
-  (or `C-c C-k` in a source buffer) sends an `interrupt` op.
+  (or `C-c C-k` in a source buffer) sends an `interrupt` op. Not every
+  server has one, and neat tells you when yours doesn't.
 
 ### Evaluation lands in the wrong namespace
 
@@ -234,6 +238,10 @@ its current namespace is. To pin a source buffer:
 - Or swap `neat-buffer-ns-function` for one that derives the ns from
   the buffer (parsing a `(ns ...)` form, reading file metadata, etc.).
 
+A namespace the server doesn't know (a typo in `neat-ns`, say) shows
+up as a `;; namespace not found: ...` line in the REPL, or in the echo
+area when there's no REPL buffer.
+
 ### `M-x neat` doesn't autofill the port
 
 The port comes from the nearest `.nrepl-port` file walking up from
@@ -309,31 +317,19 @@ eldev lint        # lint
 eldev test        # run Buttercup suites
 ```
 
-The default suite is the fast one. There's also an integration suite that
-boots real nREPL servers as subprocesses and exercises the full client.
-It's gated behind an env var since starting a server adds a few seconds:
+The default suite is the fast one. There's also an integration suite
+that starts an nREPL server (`nrepl/nrepl`, through the Clojure CLI)
+and drives the client and the REPL buffer end to end: rendering,
+stdin, interrupts, error statuses and late output. It's gated behind
+an env var since starting the JVM adds a few seconds, and it's
+skipped when `clojure` isn't on your PATH:
 
 ```
 NEAT_INTEGRATION=1 eldev test
 ```
 
-The integration suite walks `neat-it--server-impls` in
-`test/neat-integration-test.el` and registers a block per implementation
-that's installed on PATH. Currently:
-
-| Implementation | Executable | How to install |
-|----------------|------------|----------------|
-| Clojure (`nrepl/nrepl`) | `clojure` | [clojure.org/guides/install_clojure](https://clojure.org/guides/install_clojure) |
-| [Babashka](https://babashka.org) | `bb` | `brew install borkdude/brew/babashka` |
-| [Basilisp](https://basilisp.readthedocs.io) (Python) | `basilisp` | `pipx install basilisp` |
-| [let-go](https://github.com/nooga/let-go) (Go) | `let-go` | `go install github.com/nooga/let-go@latest` |
-
-Add more entries to the list to teach the suite about your favorite
-nREPL implementation. Most servers just need an executable that prints
-a port banner on stdout; for ones that can't or won't announce an
-OS-assigned port (let-go, currently), the entry can supply a
-`:port-fn` that pre-allocates a free port for the framework to pass
-in.
+The suite tests neat, not the server. To find out how well a server
+speaks nREPL, run [proof][proof] against it.
 
 ## License
 

doc/design.md

@@ -94,12 +94,22 @@ The single-callback model is what every responding nREPL client we
 surveyed (monroe, CIDER, vim-fireplace) uses. We're not breaking
 that pattern.
 
+Some messages don't belong to any pending request. Servers send `out`
+with no `id` at all, or keep sending it for a request that's already
+`done` (think of a `future` that prints after the eval has returned).
+Those go to `neat-unhandled-message-functions`, and the REPL buffer
+prints their output above the prompt.
+
 ### Op discovery via `describe`, no hardcoded Clojurisms
 
 nREPL is a protocol; servers advertise the ops they implement via the
-`describe` op. neat sends `describe` on connect and stashes the response
-on the connection. UI-level features like CAPF and eldoc consult that
-capability map and silently no-op when the op they need is missing.
+`describe` op. neat sends `describe` on connect and keeps the response
+on the connection, and `neat-op-supported-p` answers from it.
+Completion, eldoc and xref quietly step aside when the op they need is
+missing. The doc lookup and interrupt commands tell you instead, as
+you asked for them directly. A server that lists no ops at all gets
+the benefit of the doubt, so we just send the request and see what
+comes back.
 There are no hardcoded `clojure.repl/doc` forms, no assumption the
 server runs on a JVM, no Leiningen or Clojure CLI defaults baked into
 the connect command.
@@ -168,19 +178,16 @@ Supporting Emacs 27 would mean fallbacks for each of those and worse
 UX where the modern API does the right thing. Not worth it for a
 young project.
 
-## Integration testing as a compatibility tool
-
-The parameterised suite at `test/neat-integration-test.el` runs the
-same assertions against every nREPL implementation whose executable
-is on PATH. Today: Clojure (`nrepl/nrepl`), Babashka, Basilisp.
+## Integration tests
 
-This isn't just nice-to-have; it's a first step toward a real nREPL
-compatibility test suite. The contract under test is what we believe
-any conformant server should support, and concrete divergences (for
-example, Basilisp chunking `(println "hi")` into two `out` messages
-where Clojure batches them into `"hi\n"`) are exactly the findings
-that should feed back into the nREPL specification work.
+The suite at `test/neat-integration-test.el` runs neat end to end
+against the reference nREPL server. It types into a real REPL buffer
+and checks what comes back, which covers the parts unit tests can
+only fake: the stdin round trip, interrupts, error statuses and
+output that arrives after `done`.
 
-Adding a new implementation is a single plist entry in
-`neat-it--server-impls`: name, executable, command builder, and a
-regex for the port banner.
+It used to run the same checks against several servers, with the
+idea of growing into a compatibility suite for nREPL servers. That
+job has moved to [proof](https://github.com/nrepl/proof), which is
+built for it, so this suite sticks to one server and tests the
+client.

neat-bencode.el

@@ -38,21 +38,29 @@
 OBJ may be:
   - an integer
   - a string (encoded as a bytestring; multibyte strings are UTF-8 encoded)
-  - a list of encodable values
+  - a list or vector of encodable values
   - an alist whose keys are strings or symbols (encoded as a bencode dict
     with keys sorted lexicographically by their byte representation)
 
+A list whose every element is a cons headed by a string or symbol
+looks like an alist, so it goes out as a dict; use a vector to send
+a list of lists.  nil is rejected rather than guessed at, since it
+could stand for an empty list, an empty dict or a missing value.
+Send [] for an empty list.
+
 Signals `neat-bencode-error' for values that do not match any of the
 above."
   (cond
+   ((null obj)
+    (signal 'neat-bencode-error (list "cannot encode nil")))
    ((integerp obj)
     (neat-bencode--ascii (format "i%de" obj)))
    ((stringp obj)
     (let ((bytes (neat-bencode--to-bytes obj)))
       (concat (neat-bencode--ascii (number-to-string (length bytes)))
               (neat-bencode--ascii ":")
               bytes)))
-   ((neat-bencode--alistp obj)
+   ((neat-bencode-dict-p obj)
     (let ((pairs (cl-sort (mapcar (lambda (cell)
                                     (cons (neat-bencode--key-string (car cell))
                                           (cdr cell)))
@@ -65,7 +73,7 @@ above."
                                    (neat-bencode-encode (cdr cell))))
                          pairs)
                ,(neat-bencode--ascii "e")))))
-   ((listp obj)
+   ((or (listp obj) (vectorp obj))
     (apply #'concat
            `(,(neat-bencode--ascii "l")
              ,@(mapcar #'neat-bencode-encode obj)
@@ -91,10 +99,11 @@ above."
    ((symbolp k) (symbol-name k))
    (t (signal 'neat-bencode-error (list "non-string dict key" k)))))
 
-(defun neat-bencode--alistp (obj)
-  "Return non-nil if OBJ has the shape of a dict-style alist.
-That is, every cell of OBJ is a cons whose car is a string or symbol.
-An empty list is not an alist (it encodes as a list)."
+(defun neat-bencode-dict-p (obj)
+  "Return non-nil if OBJ has the shape of a bencode dict.
+That is, every cell of OBJ is a cons whose car is a string or symbol:
+what `neat-bencode-encode' sends as a dict, and what a decoded dict
+looks like.  An empty list is not a dict (it encodes as a list)."
   (and (consp obj)
        (consp (car obj))
        (cl-every (lambda (cell)