Skip to content

Fix/terminal image rendering - #1084

Closed
smarpitm wants to merge 8284 commits into
CodebuffAI:mainfrom
smarpitm:fix/terminal-image-rendering
Closed

smarpitm wants to merge 8284 commits into
CodebuffAI:mainfrom
smarpitm:fix/terminal-image-rendering

Conversation

@smarpitm

Copy link
Copy Markdown

What

Fixes getKittyFormat() which invented format ids (101/102/103/104) that don't exist in the Kitty graphics protocol spec. The spec only defines f=24 (RGB), f=32 (RGBA), and f=100 (PNG).

Non-PNG payloads (JPEG, WebP, GIF) produced by the compression pipeline are now converted to PNG via Jimp before transmission, so the bytes always match f=100 . This prevents kitty/WezTerm/Ghostty terminals from silently dropping images.

Why

The original getKittyFormat() returned fabricated ids — f=102 for JPEG, f=103 for WebP, f=104 for GIF. Since image-handler compresses large images to JPEG, sending those bytes with f=102 to a real terminal causes the terminal to either error or silently drop the image (it has no such format). The tests only asserted the escape sequence contained f=102 , not that any real terminal accepts it.

Changes

  • cli/src/utils/terminal-images.ts — getKittyFormat() always returns 100 ; new convertToPngIfNeeded() converts non-PNG payloads to PNG via Jimp; generateKittyImageSequence() and renderInlineImage() are now async.
  • cli/src/components/blocks/image-block.tsx — useMemo → useEffect + useState to handle async renderInlineImage .
  • cli/src/components/image-card.tsx — await renderInlineImage(...) in the existing async loadThumbnail .

New tests

  • getKittyFormat block: returns 100 for all media types.
  • test.each (5 media types): validates f= in the escape sequence is always in {24, 32, 100} .
  • no fabricated format ids : multi-chunk JPEG payload — no f=101 / 102 / 103 / 104 in any chunk.
  • non-PNG converted to PNG : JPEG payload is re-encoded (original base64 absent from output).

Validation

  • bun run typecheck (cli) clean.
  • 25 new tests + 152 related tests pass.

Note

This PR builds on #1 (terminal detection + pwsh fallback). Merge that first.

DeepSeek V4 Flash (the default model) is multimodal: false and reads images via the server-side describe pipeline — for best results with photos, use MiniMax M3 / GPT-5.6 Luna / MiMo 2.5.

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:triaged Classified by the community triage bot pr:needs-work Right idea, not mergeable as written

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants