docs: translate pages in parallel and every language in one run

translations.py translate keeps --jobs pages (default 8) in flight at once
on a thread pool; each page is still its own conversation, so the model
sees exactly what it did before and only the wall-clock time changes.
--lang becomes optional and repeatable like the other commands, so one
invocation refreshes every language under a single concurrency cap and
usage total. Work is ordered page by page across languages so each
language's cached system prompt is written by its first request and read
by the rest. The run prints its plan (pages, languages, concurrency,
model) before the first request, progress lines carry the language code,
and Ctrl-C leaves at once instead of waiting for pages in flight.

The API client retries rate limits and overloads a few more times since
many pages now share one limit, a connection dropping mid-reply fails that
page rather than the run, and the shared Markdown renderer used for
heading ids is serialised behind a lock.

Author: maxisbey

i18n/README.md

@@ -11,10 +11,10 @@ The English pages under `docs/` are the source. This directory holds what steers
 
 ```text
 uv run --frozen python scripts/docs/translations.py status [--lang CODE]
-uv run --frozen --group translate python scripts/docs/translations.py translate --lang CODE [--pages PATH ...]
+uv run --frozen --group translate python scripts/docs/translations.py translate [--lang CODE ...] [--pages PATH ...] [--jobs N]
 uv run --frozen python scripts/docs/translations.py stage [--lang CODE]
 ```
 
-`status` is offline: per language it lists missing, outdated (with the sections that changed), current and removable pages (translations whose English page is gone — `git rm` them). `translate` calls the Claude API (`ANTHROPIC_API_KEY` in the environment; the registry's model, or `DOCS_TRANSLATE_MODEL` to trial another) for the missing and outdated pages, retranslating only the English sections that changed and keeping the rest byte for byte; `--pages` instead re-translates exactly the named pages from scratch, which is also how a glossary or instructions change reaches existing pages (each generated page records the English section hashes it reflects, so editing those inputs invalidates nothing). `stage` assembles the tree each language site is built from (every language's, or one with `--lang`): each generated page exactly as it was generated, under an "outdated" notice linking the current English page when the English has changed since, and the English page where nothing was generated yet; `scripts/docs/build.sh` runs it before building them. Commit the generated pages in an ordinary pull request.
+`status` is offline: per language it lists missing, outdated (with the sections that changed), current and removable pages (translations whose English page is gone — `git rm` them). `translate` calls the Claude API (`ANTHROPIC_API_KEY` in the environment; the registry's model, or `DOCS_TRANSLATE_MODEL` to trial another) for the missing and outdated pages of every language (or just the `--lang` ones), several pages at a time (`--jobs`, default 8; each page is its own request, so this changes how long the run takes, not what the model sees), retranslating only the English sections that changed and keeping the rest byte for byte; `--pages` instead re-translates exactly the named pages from scratch (in every language unless `--lang` narrows it), which is also how a glossary or instructions change reaches existing pages (each generated page records the English section hashes it reflects, so editing those inputs invalidates nothing). `stage` assembles the tree each language site is built from (every language's, or one with `--lang`): each generated page exactly as it was generated, under an "outdated" notice linking the current English page when the English has changed since, and the English page where nothing was generated yet; `scripts/docs/build.sh` runs it before building them. Commit the generated pages in an ordinary pull request.
 
 To add a language, add an entry to `languages.yml`, write `<code>/instructions.md` (the sections the `pt` file has) and `<code>/glossary.json`, then run `translate --lang <code>`.

scripts/docs/translations.py

@@ -13,12 +13,13 @@
 
 Usage (from the repository root):
     python scripts/docs/translations.py status [--lang CODE]
-    python scripts/docs/translations.py translate --lang CODE [--pages PATH ...]
+    python scripts/docs/translations.py translate [--lang CODE ...] [--pages PATH ...] [--jobs N]
     python scripts/docs/translations.py stage [--lang CODE]
 
 Only `translate` calls the model (credentials come from the environment, e.g.
-`ANTHROPIC_API_KEY`) and needs the `translate` dependency group;
-`DOCS_TRANSLATE_MODEL`, if set, replaces the registry's `model` for that run.
+`ANTHROPIC_API_KEY`) and needs the `translate` dependency group; it keeps
+`--jobs` pages in flight at once, and `DOCS_TRANSLATE_MODEL`, if set, replaces
+the registry's `model` for that run.
 Exit codes: 0 done, 1 some page failed, 2 configuration or credential error.
 """
 
@@ -31,12 +32,16 @@
 import re
 import shutil
 import sys
-from collections import Counter
+import threading
+from collections import Counter, deque
 from collections.abc import Callable, Iterator, Sequence
+from concurrent.futures import FIRST_COMPLETED, Future, ThreadPoolExecutor, wait
 from dataclasses import dataclass, field
+from itertools import chain, zip_longest
 from pathlib import Path
 from typing import Any, Literal, Protocol, cast, get_args
 
+import httpx
 import markdown
 import yaml
 import zensical.config
@@ -54,6 +59,12 @@
 OUTPUT_TOKEN_BUDGET = 64_000
 # Repair turns fed back to the model after the first reply before a page fails.
 MAX_REPAIRS = 2
+# Pages in flight at once unless `--jobs` says otherwise: each page is its own
+# conversation, so concurrency changes nothing the model sees, only wall-clock time.
+DEFAULT_JOBS = 8
+# Retries (with backoff) the API client makes on rate limits and overloads before
+# a request fails its page; generous, since many pages share one rate limit.
+API_RETRIES = 6
 NOTICES_PAGE = "i18n/notices.md"
 # The nav page the notices link to for how the translations are made.
 TRANSLATIONS_DOC = "translations.md"
@@ -385,6 +396,8 @@ class Repo:
     prose_pages: list[str]
     translatable: list[str]
     renderer: markdown.Markdown
+    # python-markdown instances are not thread-safe; pages render one at a time.
+    _render_lock: threading.Lock = field(default_factory=threading.Lock, repr=False)
 
     def language(self, code: str) -> Language:
         for language in self.registry.languages:
@@ -413,10 +426,12 @@ def heading_ids(self, body: str) -> list[str]:
         Raises:
             PageError: The renderer sees headings the source scan does not (setext, indented, HTML).
         """
-        self.renderer.reset()
-        self.renderer.convert(body)
-        tokens = cast("list[dict[str, Any]]", getattr(self.renderer, "toc_tokens", []))
-        ids, found = [str(token["id"]) for token in _flatten(tokens)], parse_headings(body)
+        with self._render_lock:
+            self.renderer.reset()
+            self.renderer.convert(body)
+            tokens = cast("list[dict[str, Any]]", getattr(self.renderer, "toc_tokens", []))
+            ids = [str(token["id"]) for token in _flatten(tokens)]
+        found = parse_headings(body)
         if len(ids) != len(found):
             raise PageError(f"the page renders {len(ids)} headings but {len(found)} are ATX headings at column 0")
         return ids
@@ -579,14 +594,23 @@ class Completion:
 
 
 class Translator(Protocol):
-    """Anything that answers a conversation (`ConfigError`: credentials rejected; `PageError`: request failed)."""
+    """Anything that answers a conversation (`ConfigError`: credentials rejected; `PageError`: request failed).
+
+    `complete` is called from several threads at once when pages run in parallel.
+    """
 
     def complete(self, *, model: str, system: str, messages: Sequence[Message], max_tokens: int) -> Completion: ...
 
 
 def anthropic_translator() -> Translator:
     """The Claude Messages API client, streaming, with the system prompt as one cached block.
 
+    One client serves every thread. The system prompt is everything a language's
+    pages share (rules, instructions, glossary), so it is the cacheable prefix:
+    the first request of a language writes it and the rest read it. A page that
+    starts before that first reply has begun streaming writes it again instead;
+    `command_translate` orders the work so that is rare, and nothing waits on it.
+
     `anthropic` lives in the non-default `translate` dependency group, so it is
     imported here, by name: offline commands and type checking never need it.
 
@@ -602,7 +626,7 @@ def anthropic_translator() -> Translator:
     # The SDK resolves every credential source it knows at construction; fail
     # here, before any page work, rather than on the first request.
     try:
-        client = sdk.Anthropic()
+        client = sdk.Anthropic(max_retries=API_RETRIES)
     except sdk.AnthropicError as exc:  # e.g. a credential profile it was pointed at is unreadable
         raise ConfigError(f"cannot set up the API client: {exc}") from exc
     if not (client.api_key or client.auth_token or client.credentials):
@@ -619,6 +643,8 @@ def complete(self, *, model: str, system: str, messages: Sequence[Message], max_
                 raise ConfigError(f"the API rejected the credentials: {exc.message}") from exc
             except sdk.APIError as exc:
                 raise PageError(f"API request failed: {exc.message}") from exc
+            except httpx.HTTPError as exc:  # the connection failing mid-reply is not wrapped by the SDK
+                raise PageError(f"API connection failed: {exc!r}") from exc
             usage = Usage(
                 reply.usage.input_tokens,
                 reply.usage.output_tokens,
@@ -926,29 +952,71 @@ def translate_page(repo: Repo, inputs: Inputs, job: Job, translator: Translator,
 
 
 def command_translate(repo: Repo, args: argparse.Namespace, translator: Translator | None) -> int:
-    language = repo.language(args.lang)
-    inputs = repo.inputs(language)
-    jobs = select_jobs([classify(page) for page in repo.pages(language)], args.pages)
-    if not jobs:
-        print(f"{language.code}: nothing to translate")
+    codes = list(dict.fromkeys(args.lang))  # each language once, in the order given
+    languages = [repo.language(code) for code in codes] if codes else repo.registry.languages
+    per_language: list[list[tuple[Inputs, Job]]] = []
+    for language in languages:
+        inputs = repo.inputs(language)
+        jobs = select_jobs([classify(page) for page in repo.pages(language)], args.pages)
+        if not jobs:
+            print(f"{language.code}: nothing to translate")
+        per_language.append([(inputs, job) for job in jobs])
+    # Page-major across languages (every language's first page, then every second page, ...),
+    # so each language's first request is under way, and its cached prefix written, before
+    # its next page goes out, however many pages are in flight.
+    work = [item for item in chain.from_iterable(zip_longest(*per_language)) if item is not None]
+    if not work:
         return 0
     model = os.environ.get("DOCS_TRANSLATE_MODEL") or repo.registry.model  # never recorded in the generated files
     # Only a job with open sections calls the model; a run without one needs no client and no
     # credentials. Otherwise both are set up here, so bad credentials fail before any page work.
-    if translator is None and any(job.open for job in jobs):
+    if translator is None and any(job.open for _, job in work):
         translator = anthropic_translator()
-    usage, failed = Usage(), False
-    for job in jobs:
-        page = job.state.page
+
+    def produce(inputs: Inputs, job: Job) -> tuple[str | PageError, Usage]:
+        """One page's body (or why it failed) and the tokens it cost; runs on a pool thread."""
+        spent = Usage()
         try:
-            body = translate_page(repo, inputs, job, translator, model, usage) if translator else reassemble(repo, job)
+            body = translate_page(repo, inputs, job, translator, model, spent) if translator else reassemble(repo, job)
         except PageError as exc:
-            failed = True
-            print(f"error: {page.key}: {exc}", file=sys.stderr)
-            continue
-        page.target.parent.mkdir(parents=True, exist_ok=True)
-        page.target.write_text(with_provenance(body, job.state.hashes), encoding="utf-8", newline="\n")
-        print(f"translated: {page.key} ({len(job.open)} of {len(job.state.hashes)} sections)", flush=True)
+            return exc, spent
+        return body, spent
+
+    active = sorted({inputs.language.code for inputs, _ in work}, key=[lang.code for lang in languages].index)
+    print(f"translating {len(work)} pages ({', '.join(active)}), {args.jobs} at a time, with {model}", flush=True)
+    usage, failed = Usage(), False
+    queue = deque(work)
+    running: dict[Future[tuple[str | PageError, Usage]], tuple[Language, Job]] = {}
+    pool = ThreadPoolExecutor(max_workers=args.jobs)
+    try:
+        # The window is refilled here rather than handing the pool every page up front, so
+        # once a page raises (rejected credentials) no further page starts; the ones already
+        # in flight run out unreported, and the process exits when they have.
+        while queue or running:
+            while queue and len(running) < args.jobs:
+                inputs, job = queue.popleft()
+                running[pool.submit(produce, inputs, job)] = (inputs.language, job)
+            for future in wait(running, return_when=FIRST_COMPLETED).done:
+                language, job = running.pop(future)
+                result, spent = future.result()
+                usage.add(spent)
+                page = job.state.page
+                if isinstance(result, PageError):
+                    failed = True
+                    print(f"{language.code}: error: {page.key}: {result}", file=sys.stderr)
+                    continue
+                page.target.parent.mkdir(parents=True, exist_ok=True)
+                page.target.write_text(with_provenance(result, job.state.hashes), encoding="utf-8", newline="\n")
+                done = f"{len(job.open)} of {len(job.state.hashes)} sections"
+                print(f"{language.code}: translated {page.key} ({done})", flush=True)
+    except KeyboardInterrupt:
+        # The pool's threads are not daemons: left to them, the process would sit until every
+        # page in flight finished (and was thrown away). Leave now instead.
+        print(f"interrupted: {len(running)} pages in flight abandoned, {len(queue)} not started", file=sys.stderr)
+        print(f"usage: {usage}", flush=True)
+        os._exit(130)
+    finally:
+        pool.shutdown(wait=False)
     print(f"usage: {usage}")
     return 1 if failed else 0
 
@@ -1072,6 +1140,12 @@ def command_status(repo: Repo, args: argparse.Namespace) -> int:
 # ---- Command line ----
 
 
+def _positive(text: str) -> int:
+    if (value := int(text)) < 1:
+        raise argparse.ArgumentTypeError("must be at least 1")
+    return value
+
+
 def _parser() -> argparse.ArgumentParser:
     parser = argparse.ArgumentParser(
         prog="translations.py", description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter
@@ -1080,9 +1154,14 @@ def _parser() -> argparse.ArgumentParser:
     status = commands.add_parser("status", help="what each language is missing")
     status.add_argument("--lang", metavar="CODE")
     translate = commands.add_parser("translate", help="translate missing and outdated pages (calls the model)")
-    translate.add_argument("--lang", metavar="CODE", required=True)
     translate.add_argument(
-        "--pages", nargs="+", metavar="PATH", default=[], help="re-translate exactly these pages from scratch"
+        "--lang", nargs="+", action="extend", metavar="CODE", default=[], help="only these (default: every language)"
+    )
+    translate.add_argument(
+        "--pages", nargs="+", action="extend", metavar="PATH", default=[], help="re-translate exactly these, afresh"
+    )
+    translate.add_argument(
+        "--jobs", type=_positive, metavar="N", default=DEFAULT_JOBS, help=f"pages in flight (default {DEFAULT_JOBS})"
     )
     staged = commands.add_parser("stage", help="assemble .build/i18n/CODE/docs for the site build")
     staged.add_argument("--lang", metavar="CODE", help="stage this language only (default: every language)")