Router: add the Go HTTP server, structured JSON logging and a healthcheck subcommand (not shipped) #10326

Part of github/gh-aw-firewall#10261
(step 3 of 7).

Prerequisites

containers/router/go/ contains the Go module (one flat main package) from #10312,
merged in #10313: the strict request decoder, table loader, classification/routing service
and CLI, while the image still runs Python. The parity fixtures under
tests/fixtures/parity/ are merged (#10307). Their README.md lists the approved decisions
for the Go port, including the ones this step implements. The fixtures themselves don't cover HTTP or serve: this step tests them by
porting test_http.py and the serve tests in test_cli.py to Go. The differential CI
job keeps comparing the CLI commands only; it doesn't compare HTTP.

Summary

This step adds three things to the existing Go core:

• A serve command with the four endpoints and the Python service's safeguards (body
  limit, concurrency limit, timeouts), built on net/http.
• Structured JSON logs on stderr.
• A healthcheck subcommand, so the later Go image can check its health without Python
  or a shell.

CI builds and tests all of this, but the image still runs Python.

Background

Today cli.py _serve runs the FastAPI app from http.py under Uvicorn with
limit_concurrency=64 and timeout_keep_alive=5. http.py wraps the app in two layers:
DeadlineMiddleware (30 s, outermost) and BodyLimitMiddleware (1 MiB). It also maps
failures to the error envelope ({"code", "detail"}) as follows:

Uvicorn's own logs are plain text, so there are no structured logs today. AWF and the image
both check health by running a Python one-liner against GET /healthz.

Scope

• serve and its HTTP handler in Go, built on net/http as it is: its parser, ServeMux,
  MaxBytesReader, server timeouts and Server.Shutdown.
• Structured JSON logging for serve, written with log/slog.
• A gh-aw-router healthcheck subcommand.
• Tests ported from test_http.py and the serve tests in test_cli.py.

Specification

Use the standard library's HTTP behavior rather than reproducing Uvicorn's or Starlette's.
The only client is the api-proxy (routing-router-client.js and routing-controller.js).
It sends well-formed POST requests with Content-Type: application/json and a
Content-Length, and acts on only three outcomes: a 2xx success, a 204 from /healthz, and
a 422 with no_route. Any other status of 400 or above is the same contract error to it.
So keep the endpoints, the success bodies, the error envelope for the application-level
errors in the table above, and the 422 no_route. Where net/http differs from Uvicorn on
framing, path cleaning or timeouts, take net/http's behavior and record it in the fixture
README's decisions table. Don't write code to reproduce Uvicorn.

The decisions already in the README that apply here are the trailing-slash 404, stricter bind
ports, invalid log levels as usage errors, JSON serve startup errors, and the 501 for an
unsupported transfer coding. JSON serve logging is explicitly required by the parent issue.

serve options

• --bind HOST:PORT.
  • Env fallback GH_AW_ROUTER_BIND, default 127.0.0.1:8737.
  • The host is everything before the last colon. The port must be ASCII digits from 1
    to 65535.
  • Errors exit 2 with a structured ERROR record and one of the fixed reasons
    bind must use HOST:PORT syntax or
    bind must contain a host and port from 1 through 65535.
• --log-level.
  • Env fallback GH_AW_ROUTER_LOG_LEVEL, default info. An empty bind or log-level env
    var counts as unset, as GH_AW_ROUTER_ROUTING_TABLES already does (a decision).
  • Accepts critical, error, warning, info, debug and trace. An invalid value
    from either the flag or the env var is a usage error (exit 2).
  • Map trace to debug, and critical to error and above.
• Startup order, matching run().
  1. Parse the arguments.
  2. Load and validate the tables. On failure, exit 2.
  3. Parse the bind address.
  4. Listen. If listening fails (address in use, permission denied), log an error and
     exit 1, as Uvicorn does.

Server

Use one http.Server with:

• ReadHeaderTimeout of 10 s and ReadTimeout of 30 s, so a slow upload can't hold a
  connection past the 30 s deadline Python enforces;
• WriteTimeout of 30 s and IdleTimeout of 5 s, which match Uvicorn's keep-alive;
• the default header size limit, and no HTTP/2.

Leave framing to net/http's HTTP/1.1 parser. It rejects a bad, signed or conflicting
Content-Length and a missing Host with a bare 400 before any handler runs, and answers an
unsupported transfer coding with a 501 (an existing decision). It also rejects whitespace
before a header colon with a 400, and closes the connection without a response on an invalid
chunk. Don't add checks for the cases where it is more lenient than Uvicorn or differs from
it, such as OPTIONS *, an absolute-form target or an invalid chunk; record them as
decisions instead.

Request pipeline

1. Concurrency. Allow 64 requests in flight across all routes, /healthz included, with
   a buffered channel used as a semaphore. Answer the 65th with a bare 503 that has no error
   envelope.
2. Body limit. If the declared Content-Length is over 1,048,576 bytes, return 413
   without reading the body. Otherwise wrap the body in http.MaxBytesReader, and return
   413 when reading fails with *http.MaxBytesError, which covers chunked uploads. The
   detail is request body exceeds the 1048576-byte limit. Only the planning endpoints read
   the body, so an oversized chunked upload to another path gets that path's 405 or 404,
   where Python buffers POST, PUT and PATCH bodies first and answers 413 (a decision).
3. Routing. Register the four paths on an http.ServeMux without method patterns, plus a
   catch-all / that returns 404 not_found.
   • Each handler checks the method itself and returns 405 method_not_allowed with an
     Allow header, for HEAD and OPTIONS too. Method patterns would answer a wrong method
     with a plain-text 405 and allow HEAD implicitly.
   • Take the rest from ServeMux: percent-encoded paths are decoded (/%68ealthz is
     /healthz), the query string is ignored, /healthz/ is 404, and //healthz and
     /a/../healthz get a 307 redirect to the cleaned path.
4. Endpoints.
   • GET /healthz returns 204 with no body.
   • GET /capabilities returns 200 with the same JSON as Python's for the same tables.
   • POST /classify and POST /route:
     1. Check the media type with mime.ParseMediaType. A missing type,
        application/json, and any application/*+json, with or without parameters, are
        accepted; anything else returns 415.
     2. Decode the request, which can return 400 or 422.
     3. Run the service, which returns 200 or 422.
5. Panics. Recover them, return 500 internal_error, and log at error level with a stack
   trace. Never log the request body.

There's no separate deadline middleware. Planning is CPU-bound and takes milliseconds on a
body of at most 1 MiB, the server timeouts bound slow clients, and the api-proxy has its own
timeout. So the 504 request_timeout response goes away: a request that passes a server
timeout has its connection closed. Record this as a decision, and remove the 504 from
openapi.yaml at the cut-over, when the Go server becomes the one that ships.

Responses

• JSON. Use Content-Type: application/json. encoding/json/v2 doesn't escape <, >
  and & by default, which matters because the classifier prompt is large and XML-escaped
  and the api-proxy limits router responses to 1 MiB.
• Optional fields. Leave absent optional fields out, as Python's exclude_none does.
• Error details.
  • A schema violation's detail is request does not match the contract: followed by the
    Go decoder's message. Details aren't compared, so this needs no decision.
  • Replace characters that unicode.IsPrint rejects with spaces, then truncate to 500
    code points.
  • Don't echo caller-supplied values.

Graceful shutdown

On SIGTERM or SIGINT:

1. Log the shutdown and stop accepting connections.
2. Let in-flight requests finish for up to 0.5 s. Router requests are CPU-bound and take
   milliseconds, so this is plenty.
3. Close whatever is left, log a summary, and exit 0.

The process must exit within 1 s of the signal.

• AWF's compose service sets stop_grace_period: 2s, but AWF's teardown
  (runComposeDown in src/container-stop.ts) runs docker compose down -v -t 1. That
  overrides the grace period, so the router gets SIGKILL 1 s after SIGTERM.
• The subsequent Go container tests require exit code 0 after docker stop.
• Uvicorn today waits for in-flight requests with no limit, and AWF kills it after 1 s.

healthcheck subcommand

• gh-aw-router healthcheck [--url URL] sends GET to http://127.0.0.1:8737/healthz by
  default.

• It exits 0 only on a 204. Anything else exits 1: another status, refused, timed out.

• It writes nothing to stdout, and at most one short line to stderr.

• It uses a 1 s timeout, which matches today's image check, and never uses a proxy.

• It doesn't load routing tables. The check runs every few seconds, so it must not
  depend on the tables.

  This differs from run(), which loads tables before every command. Document the
  difference.

Structured logging (serve only)

• Format. One JSON object per line on stderr, written with log/slog's JSON handler.
  Every record has:

  • time: RFC 3339, UTC, millisecond precision;
  • level, msg;
  • service: gh-aw-router;
  • version.

• Events. "(no body)" below means the event has no extra fields.

  Event (msg)	Level	Extra fields
  starting	INFO	bind, log_level, go_version
  configuration invalid	ERROR	error_code: invalid_request, error (a fixed, sanitized reason). The process then exits 2.
  routing tables loaded	INFO	tables_path, profiles, models, label_cells
  routing tables invalid	ERROR	error_code: invalid_model_data, error (the loader message). The process then exits 2.
  listening	INFO	address
  listen failed	ERROR	error. The process then exits 1.
  request	INFO below 500, ERROR for 500 and above	request_id, method, route, status, duration_ms, error_code (error envelopes only), request_bytes, response_bytes
  panic	ERROR	request_id, route, stack
  shutting down	INFO	signal
  stopped	INFO	drained (whether in-flight requests finished before the drain limit), duration_ms

• serve stderr. After the arguments are parsed, serve writes only JSON records:
  table-load failures and bind validation errors are the routing tables invalid and
  configuration invalid records above, with exit 2. Usage errors are still plain usage
  text starting with usage: gh-aw-router, as in the other commands: an unknown option, a
  missing value, or an invalid log level from the flag or the env var. --help and the
  other CLI commands stay human-readable.

• Normalize method to a fixed list of standard HTTP methods or _OTHER. Loader and
  listener error fields must be sanitized operational reasons, not raw error objects.
  Stack traces must omit panic values and request data.

• route is the matched path (/healthz, /capabilities, /classify or /route),
  or unmatched. Never log the raw path or the query string.

• request_id is a random 128-bit hex value generated for each request. It isn't
  taken from request headers and isn't returned to clients, so the API doesn't change.
  The subsequent router OpenTelemetry step adds trace_id and span_id.

• /healthz. Don't log successful health checks. Do log any other /healthz
  response.

• Privacy. Every field comes from the allowlist in the table above. Never log any of
  the following:

  • request or response bodies, prompts, conversation text or tool inputs;
  • model ids or choice ids from requests;
  • headers, including Authorization and User-Agent;
  • query strings, or raw paths for unmatched routes;
  • 4xx detail strings. Log only error_code, because a detail can name a key the
    caller supplied.

• Other commands. The CLI commands other than serve don't log. Their stdout and
  plain one-line stderr stay unchanged.

Tests

Port test_http.py and the serve tests in test_cli.py to Go, so that the Go tests cover
everything the Python tests cover before the cut-over removes them. In addition:

• Handler tests (httptest).
  • Every row of the error table, comparing the status and code; detail is compared only
    where a test checks sanitization.
  • The method and path matrix: unknown paths, trailing slash (404, an approved decision),
    doubled slash, dot segments, percent-encoding (/%68ealthz is /healthz, /%2Fhealthz
    is 404), query strings, and wrong methods including HEAD and OPTIONS, with their Allow
    headers.
  • The content-type matrix: none, application/json; charset=utf-8 and
    application/vnd.example+json are accepted, and text/plain gets 415.
  • Success responses from each endpoint, compared as JSON, with absent optional fields
    left out.
  • Body limit:
    • a declared Content-Length of 1,048,577 on GET /healthz and on a POST to an
      unknown path;
    • a chunked body one byte over the limit;
    • a body exactly at the limit.
• Concurrency. Through a test hook that holds handlers open, 64 concurrent requests
  succeed and the 65th gets a bare 503.
• Server settings. The http.Server has the timeouts and limits above; a unit test
  checks the configured values rather than waiting for them to expire.
• Panics. A forced panic returns 500, and a panic record is logged.
• Sanitization.
  • A detail with control characters gets spaces in their place.
  • A detail over 500 code points is truncated.
  • Distinctive "canary" values placed in the body, the headers and the query never
    appear in any response detail or log line.
• Logs.
  • Parse every stderr line as JSON.
  • Check the required fields and the field allowlist.
  • Check that successful /healthz calls produce no request record.
• Shutdown (Linux and macOS only).
  1. Start the binary and begin a request that a test hook holds for about 0.2 s, which is
     less than the drain budget.
  2. Send SIGTERM.
  3. Assert that the request completes, the process exits 0 within 1 s, and the
     shutting down and stopped records appear.
  4. Repeat with a request held longer than 0.5 s. The connection closes, and the process
     still exits 0 within 1 s.
• healthcheck. Covers healthy, a 500, a refused connection and a timeout. Also check
  that it succeeds when GH_AW_ROUTER_ROUTING_TABLES points to a missing path.
• serve option tests. The CLI entry point needs a test hook that stops just before
  listening and reports the resolved host, port and log level. Cover --bind and
  --log-level from options and from GH_AW_ROUTER_BIND and GH_AW_ROUTER_LOG_LEVEL, the
  defaults, the bind syntax and port-range errors, non-digit ports such as +80, and
  invalid log levels from the flag and the env var. For startup failures, every stderr line
  must parse as JSON, and one ERROR record must carry the expected error_code.
• A new difference from Python that the api-proxy can observe (a status it acts on, a
  success body, the no_route envelope) goes through a reviewed change to the fixture
  README's decisions. Differences in transport-level behavior that net/http owns are
  recorded there once, in this step's PR, without per-case tests.

Out of scope

• The Dockerfile and image changes. The subsequent cut-over switches HEALTHCHECK to the
  healthcheck subcommand.
• OpenTelemetry, which is a subsequent step.
• Changes to the API, the headers or the error codes, apart from dropping the 504, which the
  cut-over removes from openapi.yaml.
• Reproducing Uvicorn's HTTP/1.1 parsing, Starlette's path handling, or its timeout
  responses.

Acceptance criteria

• serve implements the four endpoints and pipeline on net/http without custom
  framing, path or timeout code. The handler and serve option tests above pass on
  Linux and Windows, and test_http.py and the serve tests in test_cli.py are
  ported.
• The body limit (declared and chunked), the 64-request limit, panic recovery and error
  sanitization have tests, and the server's timeouts are set as specified.
• This step's PR updates the fixture README's decisions table with the net/http
  behavior it takes: the redirect for doubled slashes and dot segments, server timeouts
  instead of the 504, OPTIONS *, absolute-form targets, invalid chunked bodies,
  oversized chunked bodies on paths that don't read them, and empty serve env vars.
  (Whitespace before a header colon gets a 400, as in Uvicorn, so it needs no row.)
• Startup validation is tested: invalid tables exit 2 with an ERROR log record, an
  invalid bind or log level exits 2, and a listen failure exits 1.
• SIGTERM drains requests and exits 0 within 1 s, and this is tested.
• gh-aw-router healthcheck works without loading tables, and its behavior is tested.
• Logs are JSON with the documented fields only. Successful /healthz calls aren't
  logged, and the canary test shows that no request content reaches the logs.
• The router README documents serve, healthcheck, the log format and fields, and
  the env vars.

Files likely touched

• containers/router/go/: new files in the same flat main package, such as server.go
  and logging.go with tests beside them, and cli.go for serve and healthcheck (names
  are suggestions)
• containers/router/README.md
• containers/router/tests/fixtures/parity/README.md (decisions)