Skip to content

Send usage telemetry to Scarf as well as App Insights - #135

Draft
MatiasFernandez wants to merge 4 commits into
mainfrom
mfernandez/add-scarf-telemetry
Draft

MatiasFernandez wants to merge 4 commits into
mainfrom
mfernandez/add-scarf-telemetry

Conversation

@MatiasFernandez

@MatiasFernandez MatiasFernandez commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

Sends GemDB's seven usage events to Scarf as well as App Insights. App Insights masks IP addresses, so it can't tell us which companies and organisations use GemDB. Scarf can, and anyone can read its dashboards. App Insights is unchanged.

How it works

  • One send site, two sinks. send() in telemetry.ts still goes to App Insights and now also calls a second TelemetryLogger created with a ScarfSender. Opt-out is VS Code's own telemetry setting, enforced by createTelemetryLogger for both, so there's no consent check of our own.
  • Production only. Scarf's logger exists only when extensionMode is Production, so F5 and test runs never reach it and everything in Scarf is production data.
  • One GET per event, sent to https://gemdb.gateway.scarf.sh/gemdb/code/telemetry?event=<name>&…. Every GemDB property and measure goes as a query parameter, so a new one needs no change for Scarf. Of the common.* properties, only five go, from an allowlist: machine id, extension version, OS, VS Code version and remote name. Everything arrives as a string.
  • common.os is added explicitly. It isn't one of VS Code's own common properties: @vscode/extension-telemetry adds it, to its own logger only. The Scarf logger passes it through additionalCommonProperties, the same way the package does.
  • The event name has a prefix that gets stripped. VS Code adds gemtalksystems.gemdb/ to the front of every event name before a sender sees it. ScarfSender removes it, so event=activated, not event=gemtalksystems.gemdb/activated.
  • The network sits behind its own module. scarfTransport.ts holds the https.get (3 s timeout, fire-and-forget, every error swallowed, including a synchronous throw). src/__mocks__/setup.ts swaps it for a recorder in every test file. fakeExtensionContext is a Production context, so without this, a dozen existing suites would send real events to Scarf. Only scarfTransport.test.ts runs the real one, through vi.importActual with https mocked.
  • The fake logger behaves more like VS Code. It now hands events to a real sender with the name prefix, adds only the additionalCommonProperties each logger was created with, and lets common properties win a key collision. That is what makes a missing common.os fail a test.
  • The shell bundle is still guarded. The eslint import restriction and the esbuild metafile check now cover scarfTransport.ts as well as telemetry.ts, since the transport sends with no consent check of its own. grep gateway.scarf.sh out/gemdb-shell.js finds nothing.

Worth knowing when reading Scarf's data

In a remote window (SSH, Codespaces) GemDB runs on the remote machine, so a cloud host reads as Amazon, Microsoft or Google rather than the user's employer. docs/telemetry.md says to leave out events whose common.remotename is set when counting companies.

Before merge

USAGE_DATA.md has TODO comments for things we need Scarf to confirm:

  • the raw IP address is discarded after the company and location matching
  • the region the data is stored in, and how long it is kept
  • Scarf can delete events by a common.vscodemachineid value
  • our Scarf plan includes company enrichment

Not yet tested

  • An end-to-end run with a packaged .vsix in a normal profile, checking that events arrive in Scarf, that nothing arrives with telemetry.telemetryLevel: off, and that nothing arrives from dev:fresh.

🤖 Generated with Claude Code

@MatiasFernandez
MatiasFernandez force-pushed the mfernandez/add-scarf-telemetry branch from f65b803 to dc0e252 Compare October 15, 2026 20:17
MatiasFernandez and others added 2 commits October 15, 2026 17:24
The fake createTelemetryLogger ignored its sender and recorded every
event to __telemetry. A second sink needs its sender exercised, so when
one is passed the fake now calls sendEventData the way VS Code does:
the name prefixed with the extension id, and the common properties
mixed into data.properties, or into the top level when there is none.
The TelemetryReporter mock passes no sender, so __telemetry still
records exactly what App Insights is sent and no suite changes.

Like VS Code, each fake logger adds only the additionalCommonProperties
it was created with, and common properties win over an event's own on
a collision. So a sink missing a property only another logger adds
fails a test instead of passing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
scarfUrl turns one event into a GET on the Scarf gateway: event first,
then every GemDB property and measure as a query parameter, plus an
allowlist of five common.* properties. A property VS Code adds later
never reaches Scarf unreviewed, while a new GemDB property needs no
change here.

ScarfSender is the TelemetrySender that builds it. VS Code prefixes
every event name with the extension id before a sender sees it, so the
sender strips that and Scarf's event reads as EVENT spells it.

The request itself lives in scarfTransport.ts: fire-and-forget,
three-second timeout, every error swallowed, including one https.get
throws before a request exists, since VS Code does not catch what a
sender throws. It is a module of its own so setup.ts can swap it for a
recorder in every test file. The suites that call initTelemetry use a
Production context, the mode Scarf will be sent from, and must never
reach the real gateway. Only scarfTransport.test.ts asks for the real
one, with https mocked.

It sends with no consent check of its own, so both shell-bundle guards
(the eslint import restriction and the esbuild metafile check) now
cover it as they cover telemetry.ts.

Nothing calls it yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MatiasFernandez
MatiasFernandez force-pushed the mfernandez/add-scarf-telemetry branch from dc0e252 to fd18292 Compare October 15, 2026 20:25
MatiasFernandez and others added 2 commits October 15, 2026 18:38
Scarf's dashboards are readable by people who never open App Insights.
Every event now goes to both through the one send().

Scarf hears only from Production windows. Its readers never filter by
extensionMode, so an F5 or test run must not reach it at all rather
than be filtered out later. Its data is passed flat, so VS Code mixes
common.* into the top level where scarfUrl reads it. common.os is not one of VS
Code's own common properties; @vscode/extension-telemetry adds it to
its own logger only, so the Scarf logger adds it the same way.
ignoreUnhandledErrors is set because its default would forward the
extension host's unhandled errors to the sender.

docs/telemetry.md says what differs in Scarf, and that remote windows
send from the remote machine, so a cloud host reads as its provider
and should be left out when counting companies.

The fake logger's common properties gain common.vscodemachineid, so a
test can see the id reach Scarf through the allowlist.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
USAGE_DATA.md names Scarf as a second processor: what it receives, the
IP-to-company and location matching, why we want it, and that deletion
through Copy Telemetry ID covers both. The "no other third party" line
and the Disclosure section are corrected to match.

Three facts about Scarf are left as TODO comments to confirm before
merge: that the raw IP is discarded, its storage region and retention,
and that it can delete by machine id.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MatiasFernandez
MatiasFernandez force-pushed the mfernandez/add-scarf-telemetry branch from fd18292 to 87240ac Compare October 15, 2026 21:44

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant