Add nbb and let-go to the compatibility matrix

.github/workflows/compat.yml

@@ -21,7 +21,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        profile: [clojure, babashka, clojure-clr, basilisp, jank, dialtone, repartee]
+        profile: [clojure, babashka, nbb, clojure-clr, basilisp, jank, let-go, dialtone, repartee]
     steps:
       # The reports say which version of proof made them. Go 1.24 and newer
       # work it out from the tags and the commit, like for go install.
@@ -48,6 +48,13 @@ jobs:
         with:
           bb: latest
 
+      - if: matrix.profile == 'nbb'
+        uses: actions/setup-node@v7
+        with:
+          node-version: lts/*
+      - if: matrix.profile == 'nbb'
+        run: npm install -g nbb
+
       - if: matrix.profile == 'clojure-clr'
         uses: actions/setup-dotnet@v6
         with:
@@ -73,6 +80,15 @@ jobs:
           sudo apt-get update
           sudo apt-get install -y jank
 
+      - if: matrix.profile == 'let-go'
+        env:
+          GH_TOKEN: ${{ github.token }}
+        run: |
+          # The binary from the latest release, which is called lg.
+          gh release download --repo nooga/let-go --pattern '*_linux_amd64.tar.gz' --pattern checksums.txt --dir let-go
+          (cd let-go && sha256sum --check --ignore-missing checksums.txt && tar xzf let-go_*_linux_amd64.tar.gz lg)
+          echo "$PWD/let-go" >> "$GITHUB_PATH"
+
       - if: matrix.profile == 'dialtone' || matrix.profile == 'repartee'
         uses: erlef/setup-beam@v1.24.1
         with:
@@ -127,7 +143,7 @@ jobs:
         run: |
           # The reference implementation goes first.
           reports=()
-          for p in clojure babashka clojure-clr basilisp jank dialtone repartee; do
+          for p in clojure babashka nbb clojure-clr basilisp jank let-go dialtone repartee; do
             [ -f "reports/report-$p.json" ] && reports+=("reports/report-$p.json")
           done
           bin/proof matrix "${reports[@]}" > matrix.md

CHANGELOG.md

@@ -6,6 +6,7 @@
 
 - [#18](https://github.com/nrepl/proof/pull/18): The `version` snippet, so reports and the compatibility matrix show which version of a server proof checked.
 - [#18](https://github.com/nrepl/proof/pull/18): `proof matrix -html`, which builds the compatibility matrix as a web page, and the daily matrix of the bundled profiles on [GitHub Pages](https://nrepl.github.io/proof/).
+- [#19](https://github.com/nrepl/proof/pull/19): Profiles for nbb and let-go, and scenarios for `proof serve` that act like them.
 
 ## 0.1.0 (2026-10-26)
 

PLAN.md

@@ -153,7 +153,12 @@ clients.
   [GitHub Pages](https://nrepl.github.io/proof/), with the version of
   each server, which comes from the new `version` snippet. `proof matrix
   -html` builds the same page out of any reports.
-- nbb and let-go, which are easy to install in CI.
+- Done: nbb and let-go, and scenarios for what they do differently in
+  `proof serve`. Neither says `eval-error` when the code throws or sends
+  what it prints to stderr, nbb stops taking connections after a frame
+  that isn't a dict, and let-go shares the current namespace between
+  sessions and doesn't list `interrupt` in `describe` (even though it
+  has it).
 
 ### More Clients
 

README.md

@@ -47,10 +47,10 @@ value = "3"
 ```
 
 The [profiles](profiles) folder has complete profiles for nREPL itself,
-Babashka, Basilisp, ClojureCLR, jank, dialtone (Erlang) and repartee
-(Elixir), which you can use as a starting point. They come with proof as
-well, so `proof run babashka` will check Babashka (provided you have it
-installed). The [compatibility matrix](https://nrepl.github.io/proof/)
+Babashka, nbb, Basilisp, ClojureCLR, jank, let-go, dialtone (Erlang) and
+repartee (Elixir), which you can use as a starting point. They come with
+proof as well, so `proof run babashka` will check Babashka (provided you
+have it installed). The [compatibility matrix](https://nrepl.github.io/proof/)
 shows how all of them do, and it's updated every day.
 
 Now you can run the checks:

doc/usage.md

@@ -34,11 +34,11 @@ $ go build -o bin/proof ./cmd/proof
 proof needs a profile for your server. That's a small TOML file that
 explains how to start the server and provides a few snippets of code in
 its language. The [profiles](../profiles) folder has profiles for nREPL
-itself, Babashka, Basilisp, ClojureCLR, jank, dialtone (Erlang) and
-repartee (Elixir), and it's a good idea to start from the one that's
-closest to your server. If you're working on a Clojure dialect you can
-probably use `clojure.toml` almost as is, otherwise `dialtone.toml` is a
-better starting point.
+itself, Babashka, nbb, Basilisp, ClojureCLR, jank, let-go, dialtone
+(Erlang) and repartee (Elixir), and it's a good idea to start from the
+one that's closest to your server. If you're working on a Clojure
+dialect you can probably use `clojure.toml` almost as is, otherwise
+`dialtone.toml` is a better starting point.
 
 The profiles in this folder come with proof, so you can also use them by
 name instead of passing a file (e.g. `proof run babashka`). If you've
@@ -354,20 +354,24 @@ the scenarios, and every one of them is something a real server does
 |---|---|---|
 | `split-output` | Output comes one character per message | nREPL splits long output, Basilisp sends `println`'s newline on its own |
 | `empty-messages` | Replies to `eval` include messages with nothing but `id` and `session` | jank |
-| `last-value` | Only the value of the last form is sent | Basilisp, jank, dialtone |
-| `no-err` | What the code prints to stderr never reaches the client | Basilisp, dialtone, repartee |
+| `last-value` | Only the value of the last form is sent | Basilisp, jank, let-go, dialtone |
+| `no-err` | What the code prints to stderr never reaches the client | Basilisp, nbb, let-go, dialtone, repartee |
 | `error-with-done` | `eval-error` comes in the same message as `done` | jank |
-| `no-op-echo` | Replies to unknown ops don't say which op it was | ClojureCLR, Basilisp, jank, dialtone, repartee |
+| `no-eval-error` | An error comes with `err` and `ex`, but without the `eval-error` status | nbb, let-go |
+| `ansi-errors` | Error messages are colored with ANSI escape codes | let-go |
+| `no-op-echo` | Replies to unknown ops don't say which op it was | ClojureCLR, Basilisp, jank, nbb, let-go, dialtone, repartee |
 | `no-close-op` | `describe` doesn't list `close`, even though `close` works | jank |
-| `no-interrupt` | There's no `interrupt` op | ClojureCLR, Basilisp, jank |
+| `no-interrupt` | There's no `interrupt` op | ClojureCLR, Basilisp, jank, nbb, and let-go doesn't list it, nor stop an eval with it |
 | `no-stdin` | There's no `stdin` op, and reading input gets `nil` right away | ClojureCLR and Basilisp started in the background, as they read their own stdin, where Basilisp gets an empty string |
-| `read-line-throws` | There's no `stdin` op, and reading input throws | jank |
+| `read-line-throws` | There's no `stdin` op, and reading input throws | jank, and nbb, which has no `read-line` |
+| `read-line-waits` | There's no `stdin` op, and reading input waits until the eval is interrupted or its session closed | let-go, which reads the terminal it runs in, where its own REPL takes the lines |
 | `string-versions` | `versions.proof` is a plain string rather than a dict | Babashka for `versions.babashka`, ClojureCLR for `versions.clojure.tools.nrepl` |
+| `no-version-string` | `versions` has `proof` and `nrepl` with the numbers of their versions as strings, but no `version-string` | let-go for `versions.let-go` and `versions.nrepl` |
 | `no-session-closed` | `close` replies with `done` alone | Basilisp, jank, dialtone, repartee |
-| `shared-state` | Sessions on the same connection share `*1`, `*e` and the current namespace | ClojureCLR, Basilisp, jank |
+| `shared-state` | Sessions on the same connection share `*1`, `*e` and the current namespace | ClojureCLR, Basilisp, jank, nbb, and let-go for the current namespace |
 | `socket-sessions` | A session only exists on the connection that cloned it | ClojureCLR, Basilisp, jank |
-| `any-session` | Requests for sessions that don't exist run in a new session | ClojureCLR, Basilisp, jank |
-| `ns-fallback` | An eval in a namespace that doesn't exist runs in the current one | jank |
+| `any-session` | Requests for sessions that don't exist run in a new session | ClojureCLR, Basilisp, jank, nbb, let-go |
+| `ns-fallback` | An eval in a namespace that doesn't exist runs in the current one | jank, nbb, let-go |
 | `ns-error` | An eval in a namespace that doesn't exist fails without `namespace-not-found` | Basilisp |
 | `eof-error` | Reading past the end of input fails instead of returning `nil` | nREPL 1.7.0 |
 | `late-exception` | An interrupted eval's exception comes after its `done`, and the rest of its code still runs | nREPL 1.8.0 and Babashka |

internal/serve/matrix_test.go

@@ -72,6 +72,10 @@ var noInterrupts = func() map[string]check.Verdict {
 	return v
 }()
 
+// noStdin is what servers without stdin get from the stdin checks.
+var noStdin = map[string]check.Verdict{"stdin.need-input": check.Skipped, "stdin.roundtrip": check.Skipped,
+	"stdin.eof": check.Skipped}
+
 // proof serve has nREPL's completions and lookup, but not the ops of
 // cider-nrepl that clients prefer.
 var noCiderOps = map[string]check.Verdict{"completion.complete": check.Skipped, "completion.complete-symbol": check.Skipped,
@@ -85,7 +89,6 @@ func TestPassesEveryServerCheck(t *testing.T) {
 // verdicts they get in the compatibility matrix.
 func TestScenariosGetTheVerdictsOfTheirServers(t *testing.T) {
 	F, W, S := check.Failed, check.Warned, check.Skipped
-	noStdin := map[string]check.Verdict{"stdin.need-input": S, "stdin.roundtrip": S, "stdin.eof": S}
 	cases := []struct {
 		scenarios []string
 		want      map[string]check.Verdict
@@ -97,12 +100,16 @@ func TestScenariosGetTheVerdictsOfTheirServers(t *testing.T) {
 		{[]string{"last-value"}, map[string]check.Verdict{"eval.multiple-forms": F, "fireplace.connect": F}},
 		{[]string{"no-err"}, map[string]check.Verdict{"eval.stderr": F}},
 		{[]string{"error-with-done"}, nil},
+		{[]string{"no-eval-error"}, map[string]check.Verdict{"eval.error-status": F, "load-file.error": F}},
+		{[]string{"ansi-errors"}, map[string]check.Verdict{"eval.error-report": W}},
 		{[]string{"no-op-echo"}, map[string]check.Verdict{"op.unknown-echo": W}},
 		{[]string{"no-close-op"}, map[string]check.Verdict{"describe.required-ops": F}},
 		{[]string{"no-interrupt"}, noInterrupts},
 		{[]string{"no-stdin"}, noStdin},
 		{[]string{"read-line-throws"}, noStdin},
+		{[]string{"read-line-waits"}, noStdin},
 		{[]string{"string-versions"}, nil},
+		{[]string{"no-version-string"}, map[string]check.Verdict{"describe.versions": W}},
 		{[]string{"no-session-closed"}, map[string]check.Verdict{"session.close": F}},
 		{[]string{"shared-state"}, map[string]check.Verdict{"session.isolated": F}},
 		{[]string{"socket-sessions"}, map[string]check.Verdict{"session.across-connections": W,
@@ -144,16 +151,21 @@ func TestPresetsGetTheColumnsOfTheirServers(t *testing.T) {
 	columns := map[string]map[string]check.Verdict{
 		"clojure":  nreplInterrupts,
 		"babashka": nreplInterrupts,
-		"clojure-clr": checktest.Merged(noJava, noInterrupts, map[string]check.Verdict{"op.unknown-echo": W, "session.across-connections": W,
-			"session.isolated": F, "session.unknown": F, "session.closed": F, "stdin.need-input": S,
-			"stdin.roundtrip": S, "stdin.eof": S}),
-		"basilisp": checktest.Merged(noJava, noInterrupts, map[string]check.Verdict{"op.unknown-echo": W, "session.across-connections": W,
-			"session.isolated": F, "session.close": F, "session.unknown": F, "session.closed": F, "eval.stderr": F,
-			"eval.multiple-forms": F, "eval.unknown-ns": F, "stdin.need-input": S, "stdin.roundtrip": S, "stdin.eof": S}),
-		"jank": checktest.Merged(noJava, noInterrupts, map[string]check.Verdict{"describe.required-ops": F, "op.unknown-echo": W,
+		"nbb": checktest.Merged(noJava, noInterrupts, noStdin, map[string]check.Verdict{"op.unknown-echo": W,
+			"session.isolated": F, "session.unknown": F, "session.closed": F, "eval.stderr": F, "eval.error-status": F,
+			"eval.unknown-ns": F, "load-file.error": F}),
+		"clojure-clr": checktest.Merged(noJava, noInterrupts, noStdin, map[string]check.Verdict{"op.unknown-echo": W,
+			"session.across-connections": W, "session.isolated": F, "session.unknown": F, "session.closed": F}),
+		"basilisp": checktest.Merged(noJava, noInterrupts, noStdin, map[string]check.Verdict{"op.unknown-echo": W,
 			"session.across-connections": W, "session.isolated": F, "session.close": F, "session.unknown": F,
-			"session.closed": F, "eval.multiple-forms": F, "eval.unknown-ns": F, "stdin.need-input": S,
-			"stdin.roundtrip": S, "stdin.eof": S, "wire.canonical": W}),
+			"session.closed": F, "eval.stderr": F, "eval.multiple-forms": F, "eval.unknown-ns": F}),
+		"jank": checktest.Merged(noJava, noInterrupts, noStdin, map[string]check.Verdict{"describe.required-ops": F,
+			"op.unknown-echo": W, "session.across-connections": W, "session.isolated": F, "session.close": F,
+			"session.unknown": F, "session.closed": F, "eval.multiple-forms": F, "eval.unknown-ns": F, "wire.canonical": W}),
+		"let-go": checktest.Merged(noJava, noInterrupts, noStdin, map[string]check.Verdict{"describe.versions": W,
+			"op.unknown-echo": W, "session.isolated": F, "session.unknown": F, "session.closed": F, "eval.stderr": F,
+			"eval.error-status": F, "eval.error-report": W, "eval.multiple-forms": F, "eval.unknown-ns": F,
+			"load-file.error": F}),
 		"dialtone": checktest.Merged(noJava, noLanguage, map[string]check.Verdict{"op.unknown-echo": W, "session.close": F,
 			"eval.stderr": F, "eval.multiple-forms": F}),
 		"repartee": checktest.Merged(noJava, noLanguage, map[string]check.Verdict{"op.unknown-echo": W, "session.close": F,