cli: hold stdout for the --json envelope, and make card batches survivable

[jvsena42]

Closes #229, except item 7 (image search) — the Unsplash half was ruled out, and the Wikipedia half with it.

Everything here is cli/ and docs. No shared, Android or iOS code is touched, so no journeys/ script applies and RESULTS.md is unchanged.

1. --json was unparseable, and 2>/dev/null could not fix it

libpubkycore installs a tracing subscriber whose default writer is stdout, so a DHT bootstrap error — routine on a host that reaches the homeserver perfectly well over HTTPS — landed ahead of the envelope. Every JSON call in the reporter's scripts needed | grep -a '^{' in front of it.

Fixed at the descriptor rather than at the log level: dup2(2, 1) points fd 1 itself at stderr before the FFI loads, and System.out is re-pointed at a dup of the real stdout taken first (StdoutGuard.kt). That catches anything writing to the raw descriptor rather than only the layer we know about today, and leaves RUST_LOG deciding how much the SDK says rather than where.

Proven by A/B against staging, since a green build says nothing about this:

$ RUST_LOG=debug loopky login --env staging --json     # without the guard
stdout lines: 20      # every one of them a tracing line
$ RUST_LOG=debug loopky login --env staging --json     # with it
stdout lines: 1       # the envelope; all 20 on stderr

2. A batch that could not finish said nothing about what it did

A 665-row card edit --from-file 500'd after 35 writes, reported nothing about the 35, and had no way to pick up where it stopped — so a naive retry was the only move, and it rewrote every row.

• Everything is validated before the first write. A bad row 400 fails with the homeserver untouched rather than 399 rows in.
• A row already holding what it asks for is skipped. That makes re-running the same file the resume, which is better than the --resume the report asked for: no cursor to keep, nothing to pass, and no updated_at churn on rows that did not change.
• One refused row does not end the batch — when a batch fails and the same rows apply singly, the row is not the problem. A failure that will refuse everything identically (expiry, full disk, unsupported host) does stop it, as does a run of five, and the result says how many were never reached. The code reported is the failure that ended the run: a full disk after an unrelated 500 is a full disk.
• A failed batch still reports what it wrote. The same result shape travels on the failure envelope as data: written / skipped / failed / not_attempted, and per failed row its file line, card id, exit code and message.
• A homeserver 500 is retried twice with backoff first. withWriteRetry already recovers an expiry, a 429 and an unreachable session round trip; a 500 was the gap.
• Exit 12 server_error, new. It used to be 1 internal, which the CLI documents as "worth reporting as a bug" — a 500 is not the client's bug, not the caller's input, and may well work next time. Deliberately not 5, which promises the request never arrived.

3. The round trip

A JSONL row may now also be the shape card list --json emits, so a deck can be read, edited with jq and fed straight back. The tri-state survives the translation — an absent key leaves the field alone, an explicit null clears it — and since card list --json writes explicit nulls, feeding its output back sets every field to exactly what it read. A blob picture (sha256, no url) is the one thing a card file cannot name, so those are left unchanged rather than cleared, and counted in one note: clearing would strip every picture off an .apkg-imported deck the moment someone round-tripped it.

4. --check-images

Three failures produce exactly the same silent blank card and no rule over the URL can see any of them: a Wikipedia lead image that resolves to .stl or .webm, a renamed or deleted file, and a host refusing an unfamiliar client. One HEAD per distinct URL, opt-in on deck create, card add, card edit and import (--dry-run included, where it is worth the most). Warns, never refuses. Sends a real user agent — 403 Please set a user-agent is Wikimedia's answer to a generic client and is the very failure it catches — and falls back to a one-byte ranged GET where HEAD is refused. A URL that is fine produces no row at all.

5. The /thumb/ advice was inverted for a vector

"Drop the /thumb/ segment for the original, which is always served" is right for a raster and exactly backwards for an SVG: the original is image/svg+xml, and the thumbnail is the only rendered raster there is. To answer the question in the issue directly — no, neither client decodes SVG: Android loads with Coil and ships no SVG decoder here, iOS with UIImage.

So a Wikimedia SVG original is now rewritten in the warning as .../Name.svg/500px-Name.svg.png, a bad width over an SVG source keeps the width advice and withholds the drop-the-thumb line, and .tif, .webm, .ogv, .stl and friends are named. Only the SVG rewrite is spelled out as a URL — Wikimedia renders the others under /thumb/ with prefixes of their own, and an address invented here that 404s would be worse than the sentence.

6. card list can page

--limit / --cursor walk the manifest's chunk table and fetch only the records a page needs, so deciding which of 4,000 cards still want a picture no longer costs ~700 KB per pass; --json carries next_cursor. --missing-image / --has-image narrow what comes back.

Honest about the limit: there is no server-side filter to offer instead — the homeserver stores opaque records and Nexus indexes tags, not cards — so a filter without --limit saves the output and the caller's work, not the fetch. Both the README and Architecture.md say so. A cursor is a place in the deck rather than a snapshot: one naming a chunk compaction has since folded away resumes at the next one that exists.

Two more found while testing this on staging

import never ran the static image checks at all — only card add/card edit did — so neither the /thumb/ warning nor the new format advice reached the path most decks arrive by. And an http:// address in an image column reached remoteImage, whose requireNotNull threw an IllegalArgumentException no classifier recognises: exit 1 "internal" plus a Kotlin assertion, for an unrenderable URL in someone's own file. Now exit 9 with the scheme named, before anything is built.

Also: an image/ prefix is not a decodable picture. Wikimedia serves an SVG original as image/svg+xml with an ordinary 200, so the probe called every flag and colour swatch fine — the exact deck the static rule was written for.

Verification

./gradlew :cli:test detektAll green. 40 new tests across six files, including the descriptor swap driven against a fake libc (a real dup2(2, 1) in the test JVM would send Gradle's own stdout to stderr).

Against a live staging homeserver, end to end, on a throwaway 5-card deck (published, exercised, then deck deleted):

• Item 1 — the A/B above. Without the guard, RUST_LOG=debug loopky login --env staging --json puts 20 tracing lines on stdout; with it, stdout is one line and all 20 are on stderr.
• Item 3 — card list --json | jq -c '{id, front: {text, image: {url}}}' > edits.jsonl fed straight into card edit --from-file: written 5, skipped 0, failed 0. card list --has-image afterwards reports 5 of 5 and echoes back the stored ref, so the round trip is checkable from the envelope alone.
• Item 2 — the same file again: written 0, skipped 5. That is the resume. And a file with one unknown id in row 2 exits 6 with the homeserver untouched — the good row 1 was still unapplied when re-read.
• Item 6 — card list --missing-image --limit 3 → count 3, card_count 5, next_cursor "0:3", plus the hint on stderr; --cursor 0:3 --limit 10 → the remaining 2 and next_cursor: null. --missing-image goes 5 → 0 after the pictures land.
• Items 4 + 5 — import --dry-run --check-images over real upload.wikimedia.org URLs flagged the SVG original (200 image/svg+xml, with the .../500px-Name.svg.png rewrite offered), an 800px thumbnail (400) and a deleted file (404), and left the real JPEG alone. On a live card edit --back-image <...>.svg --check-images, the static advice and the probe both fire; a real JPEG gives "every picture URL answered with an image."
• The http:// image column now exiting 9 with the scheme named, where it used to be exit 1 plus a Kotlin assertion.

Envelope changes, all additive under schema: 1

data on the failure envelope; failed / failures / not_attempted / image_checks on CardWriteResult; card_count / next_cursor on CardListResult; image_checks on the import result and preview; exit code 12. deck create returns a DeckCreateResult rather than a DeckShowResult — same deck field, plus image_checks, so a caller reading the deck out of the envelope is unaffected.

🤖 Generated with Claude Code

https://claude.ai/code/session_0116NZ92Pe7aGQpqtiRkP4Ye