Skip to content

feat(functions-compiler): opt-in external source maps with app paths - #677

Open
egidijusj wants to merge 3 commits into
mainfrom
feat/functions-compiler-source-maps
Open

egidijusj wants to merge 3 commits into
mainfrom
feat/functions-compiler-source-maps

Conversation

@egidijusj

@egidijusj egidijusj commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Note

Description

Adds an opt-in sourceMap flag to @base44/functions-compiler's /v1/bundle and /v1/bundle-app requests. When set, the compiler returns an external source map (source_map) beside module, with its sources rewritten so a minified stack frame maps back to the file a user actually wrote. It is off by default and the public CLI never sets it — with it off the output is unchanged, so this is purely additive for existing callers. Only Base44's bundler service turns it on, so Cloudflare can remap uncaught-exception stacks in Tail Worker events and consumers can remap logged stacks themselves.

Related Issue

None

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactoring (no functional changes)
  • Other (please describe):

Changes Made

  • New src/source-map.ts: rewrites esbuild's sources list. A user:<key> source that is one of the request's own files becomes that file's path; in a per-app bundle the fn_<index>/ prefix is stripped and the path is validated against that function's files (via Object.hasOwn, so inherited keys like constructor can't masquerade as app files). Everything else — the shim, generated entry/wrappers, virtual runtime modules, npm and remote modules — stays in the list (the mappings index into it) behind a base44-internal: prefix. sourceRoot and sourcesContent are stripped.
  • entryPath in the request contracts: optional per-function field (bundleRequestSchema, appFunctionSchema) carrying a function's real stored path for a function submitted as main.ts. Only the source map reads it; compilation still uses entry.
  • sourceMap in the request contracts and source_map?: string on BundleResponse / BundleAppResponse.
  • deno-bundle.ts: emits sourcemap: "linked" with sourcesContent: false and outfile: MAIN_MODULE when asked — "linked" rather than "external" so the trailing //# sourceMappingURL=_bundled.mjs.map comment Cloudflare reads is kept. Output-file validation now expects exactly a module plus its map in map mode. NORMALIZED_MAIN_MODULE moved here and exported as MAIN_MODULE, since the map's filename derives from it.
  • bundler.ts: the per-app build flags are threaded as one AppBuild object (telemetry, runtimeSecrets, sourceMap) instead of positional booleans, replacing several compileApp(entries, telemetry, runtimeSecrets) call sites. The per-app map is built from a sparse functions array indexed by fn_<index>, so original positions survive the exclude-and-rebuild path. Per-function attribution probes skip the map (their module is discarded), as they already skip telemetry.
  • Tests: test/source-map.test.ts (unit — source-index stability, entryPath substitution, unknown/missing function slots, sourceRoot/sourcesContent removal) and test/source-map.e2e.test.ts (real builds: off-by-default parity, app-file naming, generated→original position mapping, workerd execution, pinned-sources snapshots, and a Miniflare Tail Worker crash whose exceptions stack and self-logged stack are both remapped by hand through source_map).
  • README: new "Source maps" section covering what gets remapped and by whom, the base44-internal: convention, and the caveats (local workerd does no remapping; two functions in one bundle can both list main.ts, so frame→function attribution comes from the log line).

Testing

  • I have tested these changes locally
  • I have added/updated tests as needed
  • All tests pass (npm test)

Checklist

  • My code follows the project's style guidelines
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation (if applicable)
  • My changes generate no new warnings
  • I have updated docs/ (AGENTS.md) if I made architectural changes

Additional Notes

Scoped entirely to packages/functions-compiler — no CLI-side changes, and no package version bump is included here. The feature is inert unless a caller passes sourceMap: true; actor entries and compileFunctionShards never ask for one. Cloudflare's own remapping of Tail Worker exception stacks can only be confirmed against a deployed Worker — the e2e test proves the positions workerd reports line up with the emitted map, which is as far as a local check goes.


🤖 Generated by Claude | 2026-10-23 15:08 UTC | e0cc2a5

egidijusj and others added 3 commits October 20, 2026 09:42
`sourceMap: true` on `bundle`/`bundleApp` returns `map` beside `module`:
an esbuild map of the final compile (not the per-function probes) with no
sourcesContent and no sourceRoot. The module keeps the trailing
`//# sourceMappingURL=_bundled.mjs.map` comment Cloudflare reads to remap
stack traces, and otherwise matches the option-off build byte for byte.

The map's sources name the request's files: per-app `fn_<index>/` is
stripped and checked against that function's files, and a function's
entry becomes its optional `entryPath` (its real stored path). Every
other source stays in place, so the mappings' indices hold, behind a
`base44-internal:` prefix.

Off by default; the public CLI never sets it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e_map

Cloudflare remaps only the stacks of uncaught exceptions it captures;
a stack the Worker logs itself (the per-app entry's console.error before
rethrowing, or user logs) stays minified. Run a throwing function in
workerd with a Tail Worker, for both entries, and map the exception's
frames back to the app's files by hand — local workerd does no remapping
— and pin that the logged per-app crash line stays minified. Say in the
README what is remapped automatically and what a consumer must remap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

🚀 Package Preview Available!


Install this PR's preview build with npm:

npm i @base44-preview/cli@0.2.2-pr.677.e0cc2a5

Prefer not to change any import paths? Install using npm alias so your code still imports base44:

npm i "base44@npm:@base44-preview/cli@0.2.2-pr.677.e0cc2a5"

Or add it to your package.json dependencies:

{
  "dependencies": {
    "base44": "npm:@base44-preview/cli@0.2.2-pr.677.e0cc2a5"
  }
}

Preview published to npm registry — try new features instantly!

@egidijusj

Copy link
Copy Markdown
Collaborator Author

Description

Adds an opt-in external source map to @base44/functions-compiler, so the Production Agent can remap a backend function's crash stack (read by a Cloudflare Tail Worker) to the files the user wrote.

The option defaults to off, and the public base44 CLI never sets it. Only Base44's bundler service (apper's base44-userapp-bundler) will, after it bumps its pin. With it off, the generated entry and the compiled module are byte-identical to main (evidence below).

Replaces base44/cli#668, which was opened from a fork, where CI's secret-dependent jobs (claude-review, generate-description, publish-preview) can't run; the earlier review discussion is there. base44/cli#678 (the _b44_error record) is stacked on this branch. The first commit's message says the response field is map; it's source_map, as below, so take this description if squash-merging.

Request and response changes

Field Notes
bundle(req) sourceMap?: boolean default false
bundle(req) entryPath?: string 1–200 chars; the function's real stored path when entry is the name it was submitted under (a flat function sent as main.ts). Read only by the map; compilation still uses entry
bundleApp(req) sourceMap?: boolean default false
bundleApp(req).functions[] entryPath?: string same as above, per function
BundleResponse / BundleAppResponse (ok) source_map?: string beside module; present only when sourceMap was on. snake_case like the response's other fields (main_module, handler_name)

Successful response with the option on:

{ "ok": true, "module": "…\n//# sourceMappingURL=_bundled.mjs.map\n", "source_map": "{\"version\":3,\"sources\":[…],\"mappings\":\"…\",\"names\":[…]}", "main_module": "_bundled.mjs", … }

What gets remapped, and by whom

  • Uncaught exceptions (the stack in a Tail Worker's exceptions): Cloudflare remaps these with the uploaded map, after the invocation and outside the Worker.
  • Stacks the Worker logs itself stay minified. Cloudflare's docs say a console.log() of error.stack inside a Worker isn't remapped. That covers the per-app entry's console.error(e) before it rethrows, user logs, and feat(functions-compiler): every build records a returned 5xx as one _b44_error line #678's _b44_error.stack. The Production Agent should read crash stacks from exceptions, and remap any logged stack itself with the same version's source_map.
  • Before rollout: a staging check in apper, after the map upload is wired, that deploys a throwing function with its map and asserts that the Tail event's exception names app files. It can't be tested here because local workerd doesn't remap.

Changes

  • esbuild, final compile only (src/deno-bundle.ts): with sourceMap, sourcemap: "linked", sourcesContent: false, outfile: "_bundled.mjs"; minify: true is unchanged. The module and map are picked from outputFiles by extension, and the single-output assertion expects exactly those two. The per-function probe compiles bundleApp runs to attribute errors skip the map, as they already skip telemetry.
    • "linked", not "external". esbuild's "external" writes the same map but does not append the //# sourceMappingURL= comment. "linked" is the mode that writes the map and keeps the comment. Apart from that trailing comment, the module is byte-identical to the build with the option off (asserted in a test).
  • sources rewrite (src/source-map.ts):
    • user:<path> where <path> is a key of the request's files → <path>; per-app user:fn_<index>/<path> → <path>, checked against that function's own files (by original index, so it survives an exclude-and-rebuild);
    • the function's entry → its entryPath when one is sent;
    • everything else (shim, generated entry, __base44_fn_<n> wrappers, activation shim, virtual runtime modules, npm and remote modules) stays in the list, so the mappings' indices hold, prefixed base44-internal:;
    • no sourceRoot, no sourcesContent.
  • src/bundler.ts: threads the flag; the per-app helpers now take one AppBuild object instead of positional telemetry, runtimeSecrets (the third flag would otherwise make it positional (…, false, true, false)).
  • src/contracts.ts: the new request fields.
  • README: a "Source maps" section.

Not touched: the generated entries, the crash path, response headers, actor entries (an actor request with sourceMap compiles as before and returns no map), and compileFunctionShards (it forwards only postResponseTelemetry/runtimeSecrets).

Evidence that output with the option off is unchanged

  • Existing snapshots unchanged. test/__snapshots__/worker-entry.test.ts.snap has no diff in this PR.
  • main vs this branch, compiled bytes. I compiled the same requests with origin/main's package and with this branch's package (both at the same directory depth, since the shim embeds node_modules-relative paths), then compared the outputs: 60/60 byte-identical. That covers 12 package-matrix functions through bundle, plus all 12 through bundleApp, plus both generated entries, each under telemetry × runtimeSecrets on/off. The result is the same with sourceMap omitted and with sourceMap: false, and neither response has a source_map key.

Tests

  • test/source-map.e2e.test.ts (fixture: a TypeScript function with a shared module and an npm:slugify@1.6.6 import):
    • option off: no source_map key and no comment, for both bundle and bundleApp;
    • single bundle: sources holds exactly main.ts, lib/shared.ts plus base44-internal: entries; no app-looking source names a __base44_ file; no sourceRoot/sourcesContent; the module equals the option-off module plus the comment;
    • one known generated position (a string in lib/shared.ts) maps back to lib/shared.ts, line 2, via Node's built-in node:module SourceMap (no new dependency);
    • entryPath replaces a flat entry;
    • the module with its comment still runs in workerd (Miniflare);
    • two-function per-app bundle: fn_<index>/ stripped, one function's entryPath applied and the other's main.ts kept, __base44_fn_0/1 internal, and a position in the second function maps back to its file and line;
    • an exclude-and-rebuild still maps the survivor's fn_1/ files to that function;
    • snapshots of the rewritten sources for the single-function and per-app entries.
  • A real crash through a Tail Worker (added after review), in both the single-function and the per-app entry: a function that throws from a shared module runs in workerd with a Tail Worker attached. The exception's frames, which local workerd leaves minified, are mapped through source_map by hand back to lib/fail.ts:2 (the throw) and main.ts:2 (the caller). That shows the positions the runtime actually reports line up with the map, including the per-app banner's line shift. A second test pins that the stack the per-app entry logs before rethrowing stays minified and needs explicit remapping.
  • test/source-map.test.ts: unit tests for the rewrite's edges esbuild doesn't produce on its own: a sourceRoot/sourcesContent to drop, inherited object keys such as constructor, and an unknown or sparse function index.
  • Full suite: 437/439. The two failures also fail on main here:
    • Stagehand in package-matrix.e2e.test.ts hits the 120 s test timeout. On another run, dayjs@1.11.10 also failed to resolve through the Wix npm mirror.
    • shared-dep-conflict.e2e.test.ts › "threads runtime-secrets mode through the conflict fallback" is flaky. It asserts only when the date-fns-tz race on a fresh DENO_DIR fires, and when it does, the survivors' rebuild also fails. I reproduced the same failure on an unmodified copy of main (1 of 15 runs; 5 of 17 on this branch, small samples).
  • typecheck, lint and knip are clean.

Size and time

Largest fixtures in the repo, on an Apple-silicon laptop with a warm dependency cache:

Fixture Module Map Map / module Compile, off → on (median of 7, interleaved)
Single: jimp@0.16.13 (largest single package-matrix module) 433 KiB (139 KiB gzip) 689 KiB (174 KiB gzip) 1.59 160 ms → 152 ms (same within noise)
Per-app: 34 package-matrix functions 2,252 KiB (703 KiB gzip) 3,071 KiB (698 KiB gzip) 1.36 1,329 ms → 2,506 ms (samples: off 1.2–1.8 s, on 1.1–3.0 s)

The per-app timing is noisy on a laptop. Treat it as "up to about 2× on a very large app" until Base44 measures real apps from S3.

Notes for review

  • No version bump or changelog entry in this PR. That follows this package's convention: the Manual Functions Compiler Publish workflow bumps package.json and src/version.ts in its own release commit (see 6e765fb), and the root CHANGELOG.md is the CLI's. The README documents the option.
  • In a per-app bundle, two functions can each list main.ts (or the same shared path). That follows the spec, which strips fn_<index>/. A consumer that needs to know which function a frame belongs to reads that from the log line's _b44_function.
  • npm sources keep esbuild's temp-dir-relative path to the Deno cache behind the prefix (for example base44-internal:../../…/deno/npm/…/slugify.js). So a map built on another host differs in those entries even though the module doesn't.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant