@@ -46,11 +46,8 @@ run it separately without replacing the wrapper's build/runtime dependency:
4646npm exec --yes --package=tslint@6.1.3 --package=typescript@5.9.2 -- tslint --project .
4747```
4848
49- ` SEMMLE_TYPESCRIPT_HOME ` still overrides the runtime dependency. It must point to
50- the installed TypeScript package directory (containing ` package.json ` ), with the
51- native ` typescript/sync ` and ` typescript/ast ` entry points and matching executable.
52- A legacy 5.x module or a directory from which the bundled package merely happens
53- to be resolvable is not a valid override.
49+ The wrapper always uses the bundled TypeScript dependency.
50+ ` SEMMLE_TYPESCRIPT_HOME ` is no longer supported and is ignored.
5451
5552Distribution packaging
5653----------------------
@@ -59,12 +56,12 @@ The Bazel build produces two archives: `:typescript` (`javascript.zip`) contains
5956the compiled ESM wrapper and JavaScript API; ` :native ` (` native.zip ` ) contains the
6057matching native packages for the target CodeQL distribution:
6158
62- | Distribution | Native packages |
63- | --- | --- |
64- | ` linux64 ` | ` @typescript/typescript-linux-x64 ` |
65- | ` linux-arm64 ` | ` @typescript/typescript-linux-arm64 ` |
66- | ` osx64 ` | ` @typescript/typescript-darwin-arm64 ` and ` @typescript/typescript-darwin-x64 ` |
67- | ` win64 ` | ` @typescript/typescript-win32-x64 ` |
59+ | Distribution | Native packages |
60+ | ------------- | -------------------------------------------------------------------------- --- |
61+ | ` linux64 ` | ` @typescript/typescript-linux-x64 ` |
62+ | ` linux-arm64 ` | ` @typescript/typescript-linux-arm64 ` |
63+ | ` osx64 ` | ` @typescript/typescript-darwin-arm64 ` and ` @typescript/typescript-darwin-x64 ` |
64+ | ` win64 ` | ` @typescript/typescript-win32-x64 ` |
6865
6966The build installs the selected packages explicitly with ` --force --no-save ` ,
7067using the pinned versions and integrities in the lockfile. Merely passing
@@ -97,12 +94,13 @@ Linux arm64, and Windows x64, including Windows 2022.
9794Native lifecycle and memory
9895---------------------------
9996
100- Each file owns a fresh native API/process. AST projection and JSON serialization
101- finish before the program is disposed and the API is closed, including on
102- failure. No native process or compiler cache is retained between files, during
103- stdout backpressure, or across reset/quit. This deliberately favors bounded
104- resource lifetime over reusing a compiler process; the measured cost is described
105- below.
97+ A single native API/process is created lazily and reused across files and
98+ prepare-files batches. Each file still owns a separate program: AST projection
99+ and JSON serialization finish before the program is disposed, including on
100+ failure. Program disposal releases its snapshot and reference-counted source-file
101+ caches without restarting the compiler. Reset closes the API, and the next parse
102+ creates a new one. Quit, EOF, failure, and completion of single-file mode also
103+ close the API.
106104
107105` SEMMLE_TYPESCRIPT_MEMORY_THRESHOLD ` remains a positive megabyte budget, defaulting
108106to 1000 when the Java extractor does not supply it. At native-process creation,
@@ -111,9 +109,9 @@ and passes the remaining bytes, with a minimum of one, as the child's
111109` GOMEMLIMIT ` . The parent's original ` GOMEMLIMIT ` is restored after spawning.
112110This is a Go garbage-collector soft limit, not a hard combined RSS ceiling:
113111Node allocations during extraction, native non-Go allocations, and oversized
114- individual files can exceed the budget. Closing the native process after every
115- file prevents native caches and heap capacity from accumulating across files.
116- Node RSS exceeding the budget produces a warning on stderr.
112+ individual files can exceed the budget. The child's budget is fixed until the
113+ API is closed; later changes to Node RSS do not resize it. Node RSS exceeding
114+ the budget at native-process creation produces a warning on stderr.
117115
118116Native stderr is inherited separately from the JSON stdout protocol. Syntax
119117diagnostics remain successful AST responses; native, I/O, and protocol failures
@@ -137,7 +135,7 @@ Projection checks only the whitelisted properties applicable to each syntax kind
137135including optional properties, and falls back to the full whitelist for unknown
138136kinds. ` src/ast_extractor.ts ` loads the private
139137` dist/api/node/protocol.generated.js ` metadata once from the locally installed,
140- pinned TypeScript dependency, independently of ` SEMMLE_TYPESCRIPT_HOME ` .
138+ pinned TypeScript dependency.
141139Updating TypeScript may require adapting this internal path or metadata format.
142140Scalar getters and kind-specific booleans are handled separately.
143141The Java converter consumes native fields directly, including ` postfixToken ` ,
@@ -146,8 +144,8 @@ spelling. Namespace classification and variable declaration kinds are derived
146144in Java without synthetic flags or properties.
147145
148146The tests cover AST projection, metadata, UTF-16 source positions, syntax
149- diagnostics, TSX, repeated parses, native process disposal, the child memory
150- budget, reset/quit/EOF, cancelled prefetch, runtime overrides , and fatal native
147+ diagnostics, TSX, repeated parses, native process reuse and disposal, the child memory
148+ budget, reset/quit/EOF, cancelled prefetch, the ignored legacy override , and fatal native
151149failures. Run them with Node.js 16.20.0 to verify the minimum supported runtime.
152150
153151Compatibility and regression coverage
@@ -200,19 +198,56 @@ codeql test run -j4 --search-path=javascript/extractor-pack:. \
200198Performance comparison
201199----------------------
202200
203- On macOS arm64 with Node.js 18.15.0, a comparison against the pre-migration
204- wrapper from ` 1971c368e1d ` compiled with TypeScript 5.9.2 gave these medians over three
205- runs:
201+ The current shared-API implementation was compared with refreshed ` main ` at
202+ ` fb0ba8c8a8a ` on October 21, 2026. Both ran on macOS arm64 with Node.js 18.15.0;
203+ ` main ` was built and run with TypeScript 5.9.2, and this branch used the pinned
204+ 7.1 native compiler. These are medians of five fresh-process runs after one
205+ warm-up per variant and workload, alternating variant order:
206206
207- | Workload | 5.9.2 time | 7.1 time | 5.9.2 combined RSS | 7.1 combined RSS |
207+ | Workload | Main time | Branch time | Main combined RSS | Branch combined RSS |
208208| --- | --- | --- | --- | --- |
209- | 103 TypeScript fixtures (33,589 bytes) | 0.237 s | 1.349 s | 124.1 MiB | 113.9 MiB |
210- | Five parses of TypeScript's 588,128-byte declaration file | 0.991 s | 1.116 s | 269.3 MiB | 249.1 MiB |
209+ | 104 TypeScript fixtures (33,779 bytes) | 0.081 s | 0.120 s | 105.5 MiB | 110.4 MiB |
210+ | Five parses of TypeScript's 588,128-byte declaration file | 0.886 s | 0.992 s | 263.7 MiB | 247.2 MiB |
211+
212+ The branch takes 47.3% more time on the fixtures and 11.9% more on the declaration
213+ workload. Sampled combined RSS is 4.7% higher and 6.3% lower, respectively.
214+ Both variants parse identical inputs. Timing starts after the metadata handshake
215+ and includes prepare-files, AST extraction, serialization, and protocol responses,
216+ including the branch's first native-process startup. It excludes Java conversion
217+ and database import. RSS is the maximum sampled sum of Node and its native
218+ descendants, sampled at a nominal 5 ms interval; short-lived peaks may be missed.
219+ Current ` main ` includes parser cleanup absent from the older baseline below.
220+
221+ ### Historical and API-lifetime comparisons
222+
223+ The initial implementation created a native process per file. On macOS arm64
224+ with Node.js 18.15.0, a comparison against the pre-migration wrapper from
225+ ` 1971c368e1d ` compiled with TypeScript 5.9.2 gave these medians over three runs:
226+
227+ | Workload | 5.9.2 time | 7.1 time | 5.9.2 combined RSS | 7.1 combined RSS |
228+ | --------------------------------------------------------- | ---------- | -------- | ------------------ | ---------------- |
229+ | 103 TypeScript fixtures (33,589 bytes) | 0.237 s | 1.349 s | 124.1 MiB | 113.9 MiB |
230+ | Five parses of TypeScript's 588,128-byte declaration file | 0.991 s | 1.116 s | 269.3 MiB | 249.1 MiB |
211231
212232Timing covers prepare-files, extraction, serialization, and protocol responses
213233after the metadata handshake, not Java conversion or database import. Combined
214234RSS is the maximum sampled sum of Node and its native descendants, sampled every
2152355 ms; short-lived peaks may be missed. The per-file native process costs about
2162365.7x on tiny files and 1.13x on the declaration workload, while measured memory is
217- slightly lower. The bounded lifecycle is retained deliberately; this migration
218- does not claim a speedup.
237+ slightly lower.
238+
239+ A follow-up comparison on October 21, 2026, using the same platform and Node.js
240+ version, measured the per-file and shared-API lifetimes directly (medians of three
241+ runs):
242+
243+ | Workload | Per-file API time | Shared API time | Per-file combined RSS | Shared combined RSS |
244+ | --- | --- | --- | --- | --- |
245+ | 104 TypeScript fixtures (33,779 bytes) | 1.017 s | 0.118 s | 115.8 MiB | 109.4 MiB |
246+ | Five parses of the same declaration file | 1.044 s | 0.985 s | 250.2 MiB | 260.2 MiB |
247+
248+ AST responses were byte-identical between lifetimes. A longer shared-API run on
249+ Node.js 26.7.0 processed 50,000 distinct generated TypeScript files: native RSS
250+ was 27.9 MiB after 1,000 files and 30.4 MiB after 50,000, while combined RSS
251+ stabilized around 176 MiB. Another run completed 500 parses of the declaration
252+ file with peak native RSS of 56.5 MiB. These workloads showed no unbounded
253+ per-file retention, so the wrapper reuses the API without a fixed batch limit.
0 commit comments