Skip to content

Record how rapid identification matched, so adoption is measurable - #44

Merged
brandonbk merged 1 commit into
mainfrom
feat/matched-by-telemetry
Aug 30, 2026
Merged

Record how rapid identification matched, so adoption is measurable#44
brandonbk merged 1 commit into
mainfrom
feat/matched-by-telemetry

Conversation

@brandonbk

Copy link
Copy Markdown
Member

The gap

matchedBy shipped in v1.10.0 as the adoption observable for deterministic customer matching —
and then went nowhere. Clients select it in the mutation document and discard it
(rapid-identify.js destructures only { id, encryptedCustomerId }), and the service never records
it. Outside DEBUG=omeda on a laptop, nothing sees it.

The consequence is asymmetric: refusals are observable — every fallback path calls
noticeError — but there is no denominator, so you can count divergent members and not the
adoption rate. The verification plan for #43 said "facet matchedBy and watch adoption go email
customerId". That was not actually possible. This makes it possible.

Why three attributes and not one

matchedBy: 'email' conflates three populations that need completely different responses:

Situation What it means What you'd do
member has no stored id client rollout still landing, or a genuinely new member nothing — resolves itself
ids all dead / malformed / unreachable data or infrastructure question investigate
ids point at two active customers a real duplicate pair merge them in Omeda

So resolveOmedaCustomerId now returns { customerId, outcome } rather than a bare id, and the
resolver records:

omedaMatchedBy         customerId | email     -- what Omeda actually matched on
omedaIdResolution      resolved | diverged | none-active | unresolvable | too-many |
                       malformed-only | none-supplied
omedaCandidateIdCount  how many ids the caller sent -- the adoption denominator

The reason is returned rather than only reported because telling those cases apart by parsing
noticeError message strings would be fragile.

resolved alongside matchedBy: 'email' is meaningful on its own: that is the lookup-to-post race,
where the id resolved cleanly and SCAO then rejected it. The two attributes are complementary, not
redundant.

Queries this enables

-- Adoption: is the fleet actually sending ids, and are they being used?
SELECT count(*) FROM Transaction WHERE omedaMatchedBy IS NOT NULL
FACET omedaMatchedBy, omedaIdResolution SINCE 1 day ago

-- The denominator -- how much of "email" is simply "nothing stored yet"
SELECT percentage(count(*), WHERE omedaCandidateIdCount > 0) FROM Transaction
WHERE omedaMatchedBy IS NOT NULL SINCE 1 week ago

-- The work queue for the audience team
SELECT count(*) FROM Transaction WHERE omedaIdResolution = 'diverged' SINCE 1 week ago

Scope

Additive and internal. No schema change, no client change, so no coordinated deploy — unlike
#43, this one ships on its own. The New Relic agent no-ops when disabled or outside a transaction,
so the call is inert rather than fatal, which matters on the blocking path of authentication.

Tests

14 specs, up from 13. Every outcome in the vocabulary is now asserted, including a new
malformed-only case that nothing previously covered — deliberately distinct from none-supplied,
since "the client stored something unusable" and "the client stored nothing" are different problems.
Lint clean; schema verified to still build via introspection.

After merge

Bump lerna.json + services/graphql/package.json to 1.10.1 and push the v1.10.1 tag — that
tag triggers the deploy.

🤖 Generated with Claude Code

`matchedBy` shipped in v1.10.0 as the adoption observable and then went nowhere: clients select
it and discard it, and nothing records it. The only signal was `noticeError`, which counts
failures with no denominator -- you could see refusals but not the rate.

Records three attributes on the transaction instead. One is not enough: `matchedBy: 'email'`
conflates three populations that need completely different responses --

  - the member has no stored id yet (client rollout, or a genuinely new member)
  - the member's ids are all dead, malformed, or unreachable
  - the member's ids point at two simultaneously-active customers

The first resolves itself as the fleet adopts, the second is an infrastructure or data question,
and the third is a duplicate pair that needs merging in Omeda. So `resolveOmedaCustomerId` now
returns `{ customerId, outcome }` rather than a bare id, and the resolver reports:

  omedaMatchedBy        customerId | email     -- what Omeda actually matched on
  omedaIdResolution     resolved | diverged | none-active | unresolvable | too-many |
                        malformed-only | none-supplied
  omedaCandidateIdCount how many ids the caller sent -- the adoption denominator

The reason is returned rather than only reported because distinguishing those cases by parsing
`noticeError` message strings would be fragile.

`resolved` alongside `matchedBy: 'email'` is meaningful on its own: that is the lookup-to-post
race, where the id resolved cleanly and SCAO then rejected it.

Additive and internal -- no schema change, no client change, so no coordinated deploy. The agent
no-ops when disabled or outside a transaction, which matters on the blocking path of
authentication.

14 specs (was 13): every outcome in the vocabulary is now asserted, including a new
malformed-only case that nothing previously covered.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@brandonbk
brandonbk merged commit d67ddee into main Aug 30, 2026
5 checks passed
@brandonbk
brandonbk deleted the feat/matched-by-telemetry branch August 30, 2026 16:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant