feat: add opt-in GCF output format for MCP tool results - #2286
feat: add opt-in GCF output format for MCP tool results#2286blackwell-systems wants to merge 2 commits into
Conversation
Set DARLING_OUTPUT_FORMAT=gcf to have the MCP server return Graph Compact Format instead of JSON. A single call-tool filter (registered once via WithRequestFilters) re-encodes each tool's JSON result as a GCF generic wire, factoring the repeated field names of the record arrays these tools return (blocking pairs, wait stats, alerts, config, ...) into one header and dropping the indentation — roughly halving the token cost of a result. Covers every tool with no per-tool changes. Conservative per result: used only when the GCF wire is both smaller than the JSON (never-grow) and a stable round-trip of it, comparing number-exactly (int64 preserved, not float-rounded); on any parse/encode/mismatch the JSON is kept, so a result is never grown, dropped, or garbled. Default JSON output is unchanged. BlackwellSystems.Gcf is zero-dependency, pinned exact.
erikdarlingdata
left a comment
erikdarlingdata
left a comment
There was a problem hiding this comment.
Thanks for this — it's careful work, and it's in the right place. I didn't want to review it from the diff alone, so I built a small console harness against BlackwellSystems.Gcf 0.2.0 and pushed realistic Darling payloads through your GcfOutput.TryEncode to see the actual wire. Most of what I went looking for, you'd already handled. One thing I found is blocking, and a few are cleanup.
What's right
The seam. One call-tool filter registered once in DarlingMcpHostService, covering every tool with no per-tool changes, is exactly where this belongs. If I were adding it myself I'd have put it there.
The guards, mostly. Opt-in and default off; never-grow; fall back to JSON on exception at every step; only a lone text block re-encoded so an image alongside it can't be dropped; IsError left alone. Int64 survives exactly — I confirmed 9007199254740993 comes out intact rather than as a rounded double.
Three things I specifically tried to break and could not:
- Sentinel collision. A row whose string value is literally
-or~gets quoted and round-trips byte-exact. That was my first guess at a silent corruption and it's handled. - Ragged rows. Darling's serializers omit nulls on some paths, so a record array can have a field on later rows that the first row lacks. I built exactly that case, expecting the field to fall off a first-row-derived header. It doesn't — the header picks up keys absent from row 0 and the missing cells get
~. Round-trips identical. - Key reordering between rows normalizes to header order, which is semantically fine for JSON objects.
The package checks out too: MIT, and genuinely zero-dependency (net8.0 and netstandard2.0 groups, no dependencies in either), so the README claim is accurate.
And the wire is legible. ## events [12]{blocked_session_id,blocking_session_id,...} followed by pipe-delimited rows is something a consuming model reads without trouble — that was my main worry going in, since JSON has universal priors and a new format does not, and I'm satisfied on the clean case.
Blocking: "lossless" isn't, and the round-trip guard can't see it
The check is:
if (Gcf.EncodeGeneric(Gcf.DecodeGeneric(wire)) != wire)
return null;That compares the wire against itself. It's a fixed-point test on the GCF codec, and it will pass for any wire the encoder produces, including one that has already lost data — decode gives back whatever the wire says, re-encode reproduces the wire, guard satisfied. The lossy step is upstream of everything it examines, on the JSON→native leg in FromJson:
case JsonValueKind.Number:
if (e.TryGetInt64(out var l))
return l;
return e.GetDouble(); // <- hereAny number that isn't an Int64 goes through a double. Measured, straight off the wire:
input 33.333333333333333 -> wire 33.333333333333336
input 0.1234567890123456789 -> wire 0.12345678901234568
Both produce a wire that passes the round-trip guard and gets returned to the caller. This is reachable rather than theoretical — Darling's tools return numeric values (percentages, ratios, average durations, cost figures), and a computed percentage is exactly the shape above.
I want to be fair about severity: nobody triages a performance problem on the sixteenth significant digit, and I'm not claiming operational harm. What I can't accept is the claim. The README says "a lossless round-trip of the JSON" and the PR says "the wire is decoded and must round-trip", and the guard is the entire reason a reviewer would trust this feature to sit in the output path of every tool. A guard that tests the wrong leg is worse than no guard, because it reads as proof.
Save yourself a detour: the obvious fix doesn't work. I tried putting a decimal into the native model directly, and GCF's generic profile emits it as a quoted string — OrdersDB|"33.333333333333333"|... — which preserves the digits but changes the JSON type from number to string. So TryGetDecimal before GetDouble trades a precision change for a type change. The generic profile appears to have no exact decimal number type.
Two ways out that I'd accept:
-
Minimal and verifiable. Decline payloads containing a number the model can't hold exactly, letting them stay JSON:
case JsonValueKind.Number: if (e.TryGetInt64(out var l)) return l; throw new NotSupportedException("non-integer number"); // caught above -> JSON fallback
Two lines, makes the losslessness claim true, and costs you the decimal-bearing results — which is honest about what the format can carry.
-
Thorough. Make the guard compare against the input: decode the wire, convert back to a JSON model, and check it semantically against a re-parse of
json. More code, and it subsumes case 1 plus anything else either of us hasn't thought of.
Either way the README and PR wording should say what's actually guaranteed.
The savings are payload-shaped, and the headline number is the good case
I measured 67% on a clean record array, close to your figure. But on a payload where real query text dominates — multi-line SQL with commas, quotes and pipes, which gets correctly quoted and escaped — it's 28%:
clean get_blocking (short repeated query): 3555 -> 1175 chars (67% smaller)
10 rows of real multi-line query text: 4236 -> 3069 chars (28% smaller)
Your 48.5–51.4% comes from a get_blocking fixture whose blocked_query is one short repeated string. That's a legitimate shape, but the tools where token cost actually hurts — get_top_queries_by_cpu, analyze_plan_xml, deadlock XML — are precisely the text-heavy ones where factoring field names buys least. Worth stating in the README so nobody enables this expecting half their context back on the big results.
The sentinels need documenting
The wire uses - for null and ~ for absent. Nothing in the wire, the README paragraph, or the tool descriptions says so. A model reading resolved_at = - will probably infer null, but "probably" is doing real work when the reader is an LLM during an incident. Since the point of this feature is to be read by a model, the meaning should be somewhere the model sees.
Please drop the Dashboard version bump
deprecated/Dashboard/Dashboard.csproj goes <Version>3.3.0</Version> → 3.3.1 while AssemblyVersion, FileVersion and InformationalVersion stay at 3.3.0.0/3.3.0, which leaves those fields disagreeing with each other in a deprecated project, for a change that has nothing to do with GCF.
If you added it to satisfy check-version-bump: that workflow is on: pull_request: branches: [main] with if: github.head_ref == 'dev', so it doesn't run on a PR into dev at all. Nothing here needed a bump.
Nits
using System.Linq;inGcfCallToolFilter.csis unused. Won't fail the build, but it's there.Transformassignsresult.Contentin place on the success path, so it mutates theCallToolResultthe tool returned rather than producing a new one. Harmless as things stand — the tools build a fresh result per call — but the comment reads like it returns a transformed copy, and the in-place version is the one that would surprise someone later.
One thing that isn't a defect, and is my call rather than yours
You're the format's vendor contributing your own package — that's fine, and stating it plainly is better than either of us pretending otherwise. What I'm weighing is dependency freshness in a monitoring product: BlackwellSystems.Gcf first published 2026-08-09, four releases in six days, and this PR pins 0.2.0, which went up 2026-08-15. A pre-1.0 package with a one-week history in the output path of every MCP tool is a different risk from the same code vendored or behind an interface. MIT and zero-dependency both help. I'm not asking you to solve this; I'm telling you it's part of my decision and it's separate from the quality of the work.
Where this leaves us
Fix the losslessness guard (either option above) and the wording, drop the Dashboard bump, document the sentinels, and I'll take another look. Also note that CI hasn't run on this — only check-branches reported, because a fork PR needs maintainer approval for the workflows. I'll approve the run so you get build and test signal, since neither of us has seen this compile in CI yet.
The engineering instinct here is good — conservative guards, fail-safe fallbacks, the right registration point, and comments that explain why rather than what. The gap is that the feature measures tokens and claims losslessness, and only one of those was actually tested.
- The fail-safe now compares the decoded wire against the parsed input model (ValuesEqual), not the wire against itself, so a value that survived only as a rounded double is caught and the result falls back to JSON. Key order is compared order-insensitively (header-order normalization is semantically equal for JSON objects). - FromJson declines any non-integer number a double cannot hold exactly (checked via TryGetDecimal); 33.5 / 0.25 still encode, a high-precision decimal or a uint64 past Int64 stays JSON. - Transform returns a new CallToolResult (carrying StructuredContent / IsError / Meta) instead of mutating the tool's result; drop the unused System.Linq. - README: savings are payload-shaped, document the - (null) / ~ (absent) sentinels, and scope the losslessness claim to what round-trips. - Revert the unrelated deprecated/Dashboard version bump.
|
Thanks for the depth here — the harness especially. You found a real one, and you were exactly right about where it lived: the guard tested the codec's fixed point, not the JSON→value leg, so it couldn't see the Losslessness guard (blocking). Two changes:
Wording. The README now says the savings are payload-shaped (≈half on record arrays, less on text-dominated results like query text and plan/deadlock XML), documents the Dashboard bump. Reverted — you're right it didn't belong, and the check doesn't run on a PR into Nits. Dropped the unused On the dependency-freshness call — that's yours to make and I won't argue it. Two things that might bear on it, though. First, the integration is built so the SDK isn't load-bearing on the hot path: every result is verified against its input and any mismatch or exception falls back to JSON, so a codec defect costs compression, not correctness. Second, its version number undersells it. |
erikdarlingdata
left a comment
erikdarlingdata
left a comment
There was a problem hiding this comment.
Re-ran the harness against this head rather than taking the fix on trust, since that is how the original defect surfaced. The blocking issue is fixed, and the fix is the right shape. Clearing my requested changes.
Verified against the new code
The two things that matter most were that the defect is actually gone and that fixing it did not break the cases I had already tried to break it with. Both hold — 13/13:
lossy decimals now DECLINE
PASS 33.333333333333333 / 0.1234567890123456789 -> falls back to JSON
PASS a lossy decimal nested in a record declines the whole payload
exact non-integers still encode
PASS 33.5 / 0.25 / 0.5 encode, and the values appear on the wire
int64 boundary
PASS 9007199254740993 exact (past 2^53)
PASS a ulong past Int64.MaxValue stays JSON
the three I tried to break it with STILL work
PASS ragged rows (a later row carrying a field row 0 lacks)
PASS literal '-' and '~' string values
PASS key reorder between rows (order-insensitive compare)
savings intact
PASS clean record array: json 8735 -> wire 2617 (71% smaller)
guards unchanged
PASS invalid JSON falls back; tiny payload (never-grow) falls back
The guard change is exactly the correction that was needed — ValuesEqual(DecodeGeneric(wire), native) verifies against the input model, so a value that only survived as a rounded double now fails the compare instead of passing a fixed-point test that could never see it. And declining in FromJson via (decimal)d != exact is the better half of the fix: it fails before a wire exists rather than catching it afterwards.
Making the key comparison order-insensitive was the right call too — it keeps the header-order normalization from reading as data loss, which is what would otherwise have taken ragged rows and key reordering down with the real bug.
Two small things, neither blocking
The word "lossless" is still unqualified in two code comments. The README now scopes it properly, but GcfOutput's class comment still reads "Opt-in, lossless, and never larger than the JSON" and GcfCallToolFilter's says "never larger, never lossy". That is the exact wording that sent me looking in the first place — worth matching the README's precision, since the code comment is what the next reader hits first.
The remaining double-domain corner is real, and you already named it. A non-integer outside the decimal range keeps its double, and the guard cannot see that particular loss because native itself holds the already-rounded value — the same structural blind spot as before, now narrowed to a corner instead of covering every decimal. Your comment says so plainly ("A token outside the decimal range is inherently double-domain"), which is the right way to handle it. Noting it only so it stays a known limit rather than becoming a surprise: it is unreachable for this codebase in practice, because Darling's numerics are int64 counts and microsecond durations plus decimal-range ratios and percentages, none of which carry 29+ significant digits.
Nits confirmed fixed: unused using System.Linq gone, and Transform now returns a new CallToolResult carrying StructuredContent/IsError/Meta instead of mutating the tool's own. Dashboard version bump reverted — the diff no longer touches that file at all.
On the dependency call
That one is Erik's, not mine, and I am not going to pre-empt it — but your framing changes the input to it, so I will relay it accurately rather than paraphrase it away. "0.2.0, published the day before the PR" reads very differently once it is the seventh binding of a spec already hardened by six siblings against a shared 281-fixture conformance suite and a cross-SDK differential fuzz, versus a package written last week. That is a fair correction to how I characterised it, and the exact-version pin does keep the upgrade in this repo's control.
The 0.x API-stability question stands on its own, as you say. Worth adding one thing in your favour that I checked rather than assumed: the integration really is built so a codec defect costs compression and not correctness — every result is verified against its input and any mismatch or exception returns the original JSON, and Transform bails before touching a result that is an error or carries anything other than a lone text block. So the blast radius of the dependency being wrong is "results stay JSON", which is the property that makes the freshness question a judgement call rather than a risk.
Nice work on the turnaround, and thanks for taking the harness seriously rather than arguing with it.
2 participants
Summary
Adds an opt-in output format,
DARLING_OUTPUT_FORMAT=gcf, that makes the MCP server returnGraph Compact Format instead of JSON. A single call-tool filter
(registered once via
WithRequestFilters/AddCallToolFilter) re-encodes each tool's JSONresult as a GCF generic wire: the repeated field names of the record arrays these tools return
(blocking pairs, wait stats, alerts, config rows, session/query stats, …) are factored into a
single header and the pretty-print indentation is dropped.
It hooks the SDK's call-tool filter pipeline in one place, so it covers every tool with no
per-tool changes — the ~60 data-read tools all flow through it.
Measured on the tool result shapes
o200k tokenizer, GCF vs the current
WriteIndented = trueJSON, lossless round-trip verified,on a representative
get_blockingresult (15-field event records):Roughly half the tokens, and it scales with row count. The win comes from factoring the
repeated JSON keys and dropping the indentation; wider record sets save more.
Deliberately conservative — it can only ever help
stays JSON.
long(not
double), so large ids/counts/durations are never float-rounded. On any parse, encode, ormismatch the JSON is kept — a result is never dropped or garbled.
(or an error) is left untouched.
BlackwellSystems.Gcfis a zero-dependency package, pinned exact.Why GCF, beyond size
The harder question than size is whether a model reads the compact form as accurately as JSON.
GCF is designed and evaluated for that:
GPT-5.5, Gemini 2.5 Pro / 3.1 Pro / 3.5 Flash), 27 runs, 11 models, 4 providers; equal to or
better than JSON on every model. https://gcformat.com/guide/benchmarks
shapes what a transformer can represent, then validated by controlled from-scratch training at
1.3B scale, measured causally rather than by an LLM judge. Four papers (2026, under review):
GCF is already in production across observability, developer tooling, and network automation:
Chrome DevTools MCP (Google) merged it
as an experimental format; Speakeasy (customers include Google, Verizon,
Mistral AI) ships it in their
oqCLI; OmniRoute vendored it into itsgateway; NetClaw benchmarked it against TOON on
real network data and replaced TOON; and many others across the MCP and agent ecosystem.
The change
GcfOutput(env gate + JSON→GCF encode with the never-grow and lossless guards) andGcfCallToolFilter(the call-tool filter), registered inDarlingMcpHostService.BlackwellSystems.Gcfpackage reference (central version pin).fallbacks, and the filter's enable / disable / error / multi-block behavior.
Darling/README.mdmcp section) documents the variable; Dashboard version bumped.dotnet buildclean (0 warnings); targetsdevper the contribution flow.GCF is open source and MIT-licensed. Format, spec, SDKs, and benchmarks: https://gcformat.com
Happy to adjust the variable name or extend beyond the single-text-block case; reproducible
numbers and harness on request.