Skip to content

feat: Source Map v3 generation with runtime traceback remapping - #570

Open
tinovyatkin wants to merge 16 commits into
mainfrom
feat/source-maps
Open

feat: Source Map v3 generation with runtime traceback remapping#570
tinovyatkin wants to merge 16 commits into
mainfrom
feat/source-maps

Conversation

@tinovyatkin

@tinovyatkin tinovyatkin commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

Closes #493

Summary

Adds opt-in Source Map v3 generation (the language-agnostic JS-ecosystem format) plus an injected Python runtime that remaps uncaught-exception tracebacks back to the original source files — analogous to node --enable-source-maps. Full design: docs/source-maps.md.

CLI

  • --sourcemap[=linked|inline|external] (esbuild-style; bare flag = linked, or inline with --stdout; linked/external + --stdout is rejected with a suggestion)
  • --sources-content=<bool> overrides the mode-dependent sourcesContent default (omitted for inline, embedded for linked/external)
  • cribo.toml equivalents: sourcemap, sources-content

How it works

  • Mapping extraction: ruff's codegen emits no positions, so the final bundle is re-parsed once and walked in parallel with the bundled AST, which carries node provenance (module-ordinal node-index ranges + original TextRanges). Each aligned statement yields one line-level mapping; divergent subtrees are skipped defensively. Serialization via oxc_sourcemap (pinned =8.1.2).
  • Runtime: injected as parsed AST statements ahead of user code (after __future__ imports). Lazy by design — zero file access, env reads, parsing, or decoding until the first uncaught exception. The decoder streams the map in constant memory (backward EOF scan for inline data URLs, chunk-aligned base64, escape-aware JSON field scanner, six-int VLQ state machine that only resolves the traceback's needed lines and exits early), and falls back streaming → json.loads → previous hook, never masking the original error.
  • Activation per mode: inline always on (CRIBO_SOURCE_MAPS=0 kill switch); linked active iff the sibling .map exists at run time; external gated on CRIBO_SOURCE_MAPS=1 (or a path to the map).
  • Hook coverage: sys.excepthook, threading.excepthook, sys.unraisablehook; CPython-style chain rendering and repeated-frame collapsing.

Testing

  • 11 Rust unit tests (builder, provenance, line index) + end-to-end mapping assertions down to exact file:line for inlined, wrapper, and entry modules
  • 25 integration tests driving the binary: all delivery modes, --stdout interplay, config keys, sourcesContent matrix, runtime activation matrix, thread/unraisable remapping, and a duress suite — RecursionError (collapsed + remapped), MemoryError under a 512 MB RLIMIT_AS (still remaps), FD exhaustion under RLIMIT_NOFILE (clean fallback), and a laziness test (unreadable map + successful run = silent)
  • 12 pure-Python decoder unit tests (VLQ machine, adversarial sourcesContent, backward EOF scan, base64 chunk alignment, json fallback parity)
  • Snapshot harness: sourcemap_ fixtures opt into --sourcemap=linked and gain a normalized, path-free source_map@<fixture> mapping snapshot; two fixtures added

cargo clippy --workspace --all-targets is clean; full suite 333/334 (the one failure, test_cli_stdout::test_directory_entry_empty_fails, is a pre-existing environment-specific snapshot mismatch, confirmed on the pristine tree)

Known limitations (documented)

  • User code formatting tracebacks itself (traceback.format_exc()) is not remapped — only the installed hooks re-render
  • Under hard OOM no pure-Python hook can run; the guarantee is non-interference
  • Mappings are statement/line-level (column 0) by design — Python tracebacks are line-oriented

Summary by CodeRabbit

  • New Features

    • Added Source Map v3 generation for bundled Python output.
    • Added linked, inline, and external source-map delivery modes.
    • Added traceback remapping to original source files and lines for uncaught exceptions.
    • Added optional embedding of original source contents.
    • Added CLI, configuration, and environment-variable options for source maps.
  • Documentation

    • Documented source-map configuration, runtime behavior, examples, and limitations.
  • Tests

    • Added comprehensive coverage for source-map generation, delivery modes, decoding, and traceback handling.

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SourceMap

3 participants