Repository navigation
Conversation
f73bebb to
f7a0dc8
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #66575 +/- ##
==========================================
+ Coverage 90.51% 90.52% +0.01%
==========================================
Files 797 797
Lines 278252 279085 +833
Branches 53472 53694 +222
==========================================
+ Hits 251868 252650 +782
- Misses 16761 16820 +59
+ Partials 9623 9615 -8
🚀 New features to boost your workflow:
|
hamidrezaghavami
left a comment
There was a problem hiding this comment.
The implementation in lib/fs.js and lib/internal/fs/promises.js is incredibly clean. I didn't review the C++ bindings, but LGTM on the JS/Promises side!
| `fsPromises.glob()`, `fs.watch()`, `fs.watchFile()`, and `dir.read()` do not | ||
| publish events. |
Qard
left a comment
There was a problem hiding this comment.
Rather than having the getFsChannel, shouldTraceFs, and the traceFs* functions, we should be defining the channels at the top level. By using these functions which take a string name for the channels at publish time you're eliminating the performance benefit diagnostic channels have of splitting the lookup cost out of the hot publishing path, leaving it with basically the same performance you'd get with a plain event emitter.
The diagnostics channel design is intended to be used with explicit top-level defined channels which V8 can then optimize the calling of as simply a held pointer from the outer scope which is very fast to reference as V8 will optimize it, rather than doing channel lookups at every publish or check.
The checks looked each channel up by name on every call, through a dictionary and a helper that took the operation name. The channels and their context field names are now module-level constants, one tracer object per operation, so a check is a property read on a held object. Refs: nodejs#66575 (review) Assisted-by: claude:fable-5.1 Signed-off-by: Tim Fish <tim@timfish.uk>
Qard
left a comment
There was a problem hiding this comment.
Why this FsTracer wrapper type rather than just using the tracing channel instance directly? You're wasting a bunch of computation on every publish of recomputing the message shape for each channel when that would not change between publishes to that channel. Seems to me like it'd be better to just use the tracing channel instances directly, have each publish site craft its own particular shape based on the fields relevant to that channel, and use hasSubscribers in place of the shouldTrace wrapper function. I don't really see a reason for wrapping like this. 🤔
The fs modules now hold the TracingChannel instances themselves, and each publish site builds its own context literal, so the shape of the context is fixed per site instead of computed from a field table on every publish. The check at the start of each function is `hasSubscribers` on the held channel. The trace helpers call the traced function again with a private `this`, so that the second entry skips the check. This replaces the re-entry flag and the wrapper that consumed it. No traced function uses `this` for anything else. Refs: nodejs#66575 (review) Assisted-by: claude:fable-5.1 Signed-off-by: Tim Fish <tim@timfish.uk>
|
Done in c9571ff. Each operation now has a Cost of the check at a call site, with the previous two versions for comparison. The
Against |
Each node:fs operation publishes to its own TracingChannel, named fs.<operation>. The sync, callback, and promise forms of an operation share a channel, and the event context carries the form in `api`, the caller's arguments in `args`, and the leading arguments by name. The events are published from JavaScript with the TracingChannel helpers, so all events of a call share one context object, stores bound with bindStore() reach the callback and nested operations, and the error event is always followed by end or asyncEnd. Each traced function starts with a check that calls the function again inside the trace when the channel has subscribers. The whole body then runs inside the start scope, so requests capture the bound stores, and captured function references still publish. Assisted-by: claude:opus-5.5 Signed-off-by: Tim Fish <tim@timfish.uk>
The tracing checks of realpath.native and realpathSync.native passed the functions to the trace helpers as property reads. When user code replaces the property, the trace helper called the replacement again, so it ran twice for each call. The functions now pass themselves by their own name. The new checks assert from setImmediate(), because an exception thrown in a realpath.native() callback or in a nextTick from it is lost. That is a separate bug. Assisted-by: claude:fable-5.1 Signed-off-by: Tim Fish <tim@timfish.uk>
Say which events each form of an fs operation publishes, and explain the function names in the channel table. Assisted-by: claude:fable-5.1 Signed-off-by: Tim Fish <tim@timfish.uk>
The checks looked each channel up by name on every call, through a dictionary and a helper that took the operation name. The channels and their context field names are now module-level constants, one tracer object per operation, so a check is a property read on a held object. Refs: nodejs#66575 (review) Assisted-by: claude:fable-5.1 Signed-off-by: Tim Fish <tim@timfish.uk>
The fs modules now hold the TracingChannel instances themselves, and each publish site builds its own context literal, so the shape of the context is fixed per site instead of computed from a field table on every publish. The check at the start of each function is `hasSubscribers` on the held channel. The trace helpers call the traced function again with a private `this`, so that the second entry skips the check. This replaces the re-entry flag and the wrapper that consumed it. No traced function uses `this` for anything else. Refs: nodejs#66575 (review) Assisted-by: claude:fable-5.1 Signed-off-by: Tim Fish <tim@timfish.uk>
fs.mkstemp(), fs.mkstempSync() and fsPromises.mkstemp() landed after this branch, so the coverage test failed on them. Add the fs.mkstemp channel with the same context as fs.mkdtemp. Assisted-by: claude:fable-5.1 Signed-off-by: Tim Fish <tim@timfish.uk>
c9571ff to
87bf499
Compare
This adds a
diagnostics_channelTracingChannel for eachnode:fsoperation. It is a different approach to #65370, based on the review feedback there.Each operation publishes to
tracing:fs.<operation>:*, for exampletracing:fs.stat:start. The sync, callback and promise forms of an operation share a channel, so a subscriber tofs.readFileseesfs.readFile(),fs.readFileSync(),fsPromises.readFile()andfilehandle.readFile(). The context hasapi('sync','callback'or'promise'),args(a copy of the arguments as passed), and the leading arguments by name, such aspath,fdanddest. The docs have the full list.The events are published from JS with
traceSync(),traceCallback()andtracePromise(), so all events of one call share one context object,erroris always followed byendorasyncEnd,asyncStarthasresult, andbindStore()works. Buffers and the other arguments are passed by reference inargs. ThereadFileandwriteFileone-round-trip paths,fsPromises.readFile(),readFileSync()with an encoding,existsSync()and the FileHandle methods all publish.Each traced function starts with one line, for example
if (shouldTraceFs('stat')) return traceFsCallback('stat', stat, this, arguments);. When the channel has subscribers, the helper calls the same function again inside the trace, and a flag makes that inner call skip the check. The whole body then runs inside thestartscope. This matters forbindStore(): theFSReqCallbackor binding promise must be created inside the scope, because a request created before it does not carry the store to the callback. Nested fs calls also run inside the parent's store. Because the check is in the function body, captured references and ESM named imports still publish. When nothing subscribes, the rest of the body is the same as on main.Calls with invalid arguments publish
start,errorandend. Calls that a VFS handler serves also publish.fsPromises.glob(),fs.watch(),fs.watchFile()anddir.read()do not publish. An operation that uses other fs operations publishes those too, inside its own events. For example,fs.readFileSync()without an encoding publishesfs.open,fs.readandfs.closeinsidefs.readFile.With plain
compare.js, the async and stream benchmarks inbenchmark/fsdo not change, but some short sync benchmarks are slower. With 15 runs and no subscribers,fstatSyncwas about 5% slower,statSyncabout 3%, and failingreadFileSynccalls 5% to 9%. Most of this is the cost ofTracingChannel.hasSubscribersbefore V8 optimizes it. Many of these benchmarks do 10,000 calls in a new process, so most checks run while the getter is still in the interpreter or in Sparkplug. There, the getter callsBoundedChannel.hasSubscribersandChannel.hasSubscribersfor all five channels, which costs 60 to 100 ns per check. Once it is optimised, it costs about 3 ns. Next to a 300 to 900 ns sync call, the difference shows.Once
hasSubscribershas been optimized, calling it becomes insignificant apart from sync calls that throw. These are 1-3% slower. When an exception passes through a function that runs as Sparkplug code, V8 finds the bytecode offset with a linear walk from the start of the function (Code::GetBytecodeOffsetForBaselinePC, fromIsolate::ThrowandLookupExceptionHandlerInTable). So any code before the binding call makes each throw slower. A singleif (x.active !== 0) return;costs about 50 ns per failingopenSync()there. I did not find a way around this.I did not change the benchmarks to warm the
hasSubscribersgetters. A follow-up could makeTracingChannel.hasSubscriberscheaper before optimisation.The new tests cover the event order, the shared context,
bindStore()across callbacks, promises and nested calls, and the FileHandle methods. A coverage test fails if a public fs function does not publish, or publishes more than once.Fixes: #65330
Refs: #65370
CC: @nodejs/diagnostics
This PR was partly AI generated but everything was reviewed by a water based human