Skip to content

lib: optimize TextEncoder.encode for short strings - #66831

Open
mertcanaltin wants to merge 3 commits into
nodejs:mainfrom
mertcanaltin:mert/textencoder-small-string
Open

mertcanaltin wants to merge 3 commits into
nodejs:mainfrom
mertcanaltin:mert/textencoder-small-string

Conversation

@mertcanaltin

Copy link
Copy Markdown
Member

This pr improvement short strings for TextEncoder.encode.

Benchmark results:

(.venv) ➜  node git:(mert/textencoder-small-string) βœ— npx node-benchmark-compare te-clean.csv                                                        
                                                                          confidence improvement accuracy (*)   (**)  (***)
util/text-encoder.js op='encode' type='ascii' n=1000000 len=256                    *      1.67 %       Β±1.45% Β±1.94% Β±2.54%
util/text-encoder.js op='encode' type='ascii' n=1000000 len=32                   ***    127.18 %       Β±1.93% Β±2.58% Β±3.40%
util/text-encoder.js op='encode' type='one-byte-string' n=1000000 len=256                 1.39 %       Β±2.07% Β±2.79% Β±3.68%
util/text-encoder.js op='encode' type='one-byte-string' n=1000000 len=32         ***     85.44 %       Β±2.43% Β±3.27% Β±4.31%
util/text-encoder.js op='encode' type='two-byte-string' n=1000000 len=256                -0.12 %       Β±0.86% Β±1.15% Β±1.49%
util/text-encoder.js op='encode' type='two-byte-string' n=1000000 len=32         ***     74.83 %       Β±1.45% Β±1.94% Β±2.54%

Be aware that when doing many comparisons the risk of a false-positive result increases.
In this case, there are 6 comparisons, you can thus expect the following amount of false-positive results:
  0.30 false positives, when considering a   5% risk acceptance (*, **, ***),
  0.06 false positives, when considering a   1% risk acceptance (**, ***),
  0.01 false positives, when considering a 0.1% risk acceptance (***)

I got help from Claude Opus 5.5. I reviewed the changes and ran the tests and benchmarks myself.

Assisted-by: Claude opus 5.5
Signed-off-by: Mert Can Altin <mertgold60@gmail.com>
@nodejs-github-bot nodejs-github-bot added encoding Issues and PRs related to the TextEncoder and TextDecoder APIs. needs-ci PRs that need a full CI run. labels Oct 18, 2026
Comment thread test/parallel/test-whatwg-encoding-custom-textencoder-short.js Outdated
mertcanaltin and others added 2 commits October 18, 2026 16:51
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
@mertcanaltin
mertcanaltin requested a review from aduh95 October 18, 2026 13:58
@codecov

codecov Bot commented Oct 18, 2026

Copy link
Copy Markdown

Codecov Report

βœ… All modified and coverable lines are covered by tests.
βœ… Project coverage is 90.49%. Comparing base (e2d5522) to head (5f342bd).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #66831      +/-   ##
==========================================
- Coverage   90.50%   90.49%   -0.01%     
==========================================
  Files         797      797              
  Lines      277793   277809      +16     
  Branches    53366    53366              
==========================================
+ Hits       251405   251414       +9     
- Misses      16801    16806       +5     
- Partials     9587     9589       +2     
Files with missing lines Coverage Ξ”
lib/internal/encoding.js 96.74% <100.00%> (+0.08%) ⬆️

... and 33 files with indirect coverage changes

πŸš€ New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • πŸ“¦ JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

encoding Issues and PRs related to the TextEncoder and TextDecoder APIs. needs-ci PRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants