feat(settings): run the QR promo value proposition experiment by vbudhram · Pull Request #21112 · mozilla/fxa

vbudhram · 2026-08-26T14:30:08Z

Because

The mobile promo in the auth flow has never been A/B tested, so we do not know which value proposition converts desktop users to app downloads
FxA has had Nimbus plumbing for a while but no experiment has ever used it; this is the first one
The current promo fits one line of copy, so the longer test strings have nowhere to go

This pull request

Declares the promo-qr-mobile feature in configs/nimbus.yaml with two variables, enabled and branch. Every channel defaults to the control, so a Cirrus failure is a no-op
Adds branches.ts, mapping each branch slug to its heading and its own QR code. One slug drives both, because each QR scans to a distinct Bitly link and a mismatched pair would misattribute the download. An unknown slug falls back to the control
Declares all 9 headings as literal ids in en.ftl so l10n extraction finds them, and picks one at runtime with useFtlMsgResolver. The CTA moves below the QR and changes wording, so it takes a new promo-qr-mobile-description-v2 id
Rebuilds index.tsx to the updated design. Drops the logo above the heading, since the Bitly QR already embeds the Firefox glyph. Replaces the 230KB qr-mobile-kit.svg with fox-body.svg, fox-tail.svg, and a per-branch QR, roughly a 10x reduction
Adds branch and nimbus_user_id extra keys to promo_qr_mobile.view. Cirrus reports enrollment itself, so the app must stamp nimbus_user_id on its own events for analysis to join them
Exports NimbusContext so tests and stories can supply a branch without a network fetch

Issue that this pull request solves

Closes: https://mozilla-hub.atlassian.net/browse/FXA-14273

Checklist

My commit is GPG signed.
If applicable, I have modified or added tests which pass locally.
I have added necessary documentation (if appropriate).
I have verified that my changes render correctly in RTL (if appropriate).
I have manually reviewed all AI generated code.

Other information

npx jest src/components/PromoQrMobile: 52 passed, 0 failed.
npx jest src/lib/glean src/models/contexts: 188 passed, 0 failed.

How to test:

Set NIMBUS_ENABLED=true in the root .env and bring up infrastructure so the cirrus pm2 process runs
Create the experiment in Experimenter against app accounts.cirrus, feature promo-qr-mobile, 9 branches
Load the app with ?nimbusPreview=true at 1024px or wider, on /, /signin, or /signup
Confirm the heading and QR change together per branch, and that promo_qr_mobile.view carries branch and nimbus_user_id

Branch records come from Remote Settings, so enrollment cannot be hand-authored locally. Storybook has Control, TreatmentB, and TreatmentA stories covering 1, 2, and 3-line headings without a Nimbus fetch.

Needs a decision before merge: the CTA reads "Scan to download mobile app" per the Figma design, but the ticket table says "Scan to get the app" in all 9 rows. The CTA is constant across branches so it does not confound the experiment, but it does change shipped copy in every branch including the control.

Needs data review: the two new Glean extra keys.

Copilot

Pull request overview

Adds a Nimbus experiment for QR mobile-promo messaging and branch-specific attribution.

Changes:

Adds nine experiment branches with localized headings and QR mappings.
Redesigns the promo and expands Glean telemetry.
Adds component tests and Storybook variants.

Reviewed changes

Copilot reviewed 11 out of 23 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
`configs/nimbus.yaml`	Declares the experiment feature.
`packages/fxa-shared/metrics/glean/fxa-ui-metrics.yaml`	Defines telemetry extras.
`packages/fxa-shared/metrics/glean/web/index.ts`	Adds the branch event property.
`packages/fxa-shared/metrics/glean/web/promoQrMobile.ts`	Updates generated event typing.
`packages/fxa-settings/src/lib/glean/index.ts`	Forwards experiment telemetry.
`packages/fxa-settings/src/models/contexts/NimbusContext.ts`	Exports Nimbus context.
`packages/fxa-settings/src/components/PromoQrMobile/branches.ts`	Maps branches to copy and QR assets.
`packages/fxa-settings/src/components/PromoQrMobile/index.tsx`	Implements experiment rendering and reporting.
`packages/fxa-settings/src/components/PromoQrMobile/index.test.tsx`	Tests branches, visibility, and telemetry.
`packages/fxa-settings/src/components/PromoQrMobile/index.stories.tsx`	Adds visual branch scenarios.
`packages/fxa-settings/src/components/PromoQrMobile/en.ftl`	Adds localized experiment copy.
`packages/fxa-settings/src/components/PromoQrMobile/control.svg`	Adds the control QR asset.
`packages/fxa-settings/src/components/PromoQrMobile/treatment-b.svg`	Adds a treatment QR asset.
`packages/fxa-settings/src/components/PromoQrMobile/fox-body.svg`	Adds decorative artwork.
`packages/fxa-settings/src/components/PromoQrMobile/fox-tail.svg`	Adds decorative artwork.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

+  const cmsQr =
+    promoQrImageUrl && isValidCmsUrl(promoQrImageUrl) ? promoQrImageUrl : null;
+
+  const feature = experiments?.features?.['promo-qr-mobile'];
+  const enrolled = !cmsQr && feature?.['enabled'] === true;


-      GleanMetrics.promoQrMobile.view();
+      GleanMetrics.promoQrMobile.view({
+        event: {
+          nimbusUserId: experiments?.nimbusUserId,


+    extra_keys:
+      nimbus_user_id:
+        description: Nimbus user ID
+        type: string
+      branch:
+        description: The value proposition experiment branch shown to the user


+
+      <FtlMsg id="promo-qr-mobile-description-v2">
+        <p className="py-1 text-sm text-grey-900 dark:text-grey-100">
+          Scan to download mobile app


+      promoQrMobile.view.record({
+        branch: gleanPingMetrics?.event?.['branch'] || '',
+        nimbus_user_id: gleanPingMetrics?.event?.['nimbusUserId'] || '',


Copilot

🟡 Changes recommended

Correct initial exposure recording, resolve the CTA copy decision, and complete the telemetry data review and identifier documentation.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

packages/fxa-settings/src/components/PromoQrMobile/index.tsx:62

Both upstream states start with loading: false and only flip to true in effects (NimbusContext.ts:87,125 and models/hooks.ts:615-630). On the initial render this therefore paints and records the control before either request starts; the promo is then hidden and later swaps to the assigned branch, but hasLoggedView prevents recording that real exposure. Make the providers report the initial request as pending synchronously (or expose an explicit settled state), and cover the provider-integrated initial render.

  // The CMS image decides enrollment, so reporting before it lands would stamp a
  // treatment on a user who ends up seeing the CMS control.
  const loading = nimbusLoading || cmsLoading;

Files reviewed: 14/26 changed files
Comments generated: 2
Review effort level: Balanced

+      nimbus_user_id:
+        description: Nimbus user ID
+        type: string


+# Value proposition variant. Refers to a calmer browsing experience on the phone.
+promo-qr-mobile-heading-treatment-h = Take a calmer way to browse with you
+# Appears below a QR code that a user can scan to download the Firefox mobile app
+promo-qr-mobile-description-v2 = Scan to download mobile app


dschom · 2026-09-03T21:12:18Z

+      variants:
+        control:
+          description: Control.
+        treatment-a:


Why do these have over lapping descriptions. e.g. a & b are the same?

Let me clean this up, it is what appears in nimbu sto give description of the treatment. I will also add text of what is in the QR.

dschom · 2026-09-03T21:18:17Z

            setExperiments({
              features: features,
-              nimbusUserId: uniqueUserId,
+              // The ID Cirrus enrolled with, not uniqueUserId: analysis joins


I'm not sure what 'analysis' means in this context.

dschom · 2026-09-03T21:19:20Z

  'mobile_device_count',
  'supported',
  'has_passkey_option',
+  'branch',


should this be more explicit? This refers to the branch of the nimbus experiment right?

Because: - The mobile download rate from desktop auth traffic needs a lift, and no FxA Nimbus experiment has run before. - The updated promo design makes room for the longer headings the test needs. This commit: - Declares promo-qr-mobile in the FML, with branch typed as a BranchSlug enum so Experimenter rejects a misspelled arm instead of serving the control. - Names each enum variant by its value proposition and CTA, so the nine arms are distinguishable in the Experimenter form. - Adds 9 literal FTL heading ids and a per-branch QR, resolved at runtime. - Rebuilds the promo to the updated design: no logo, CTA below the QR, and real fox vectors in place of the 230KB kit. - Reports branch and nimbus_user_id on promo_qr_mobile.view, and holds the render until Nimbus and the CMS both resolve.

vbudhram requested review from a team as code owners August 26, 2026 14:30

vbudhram changed the title ~~Fxa 14273~~ feat(settings): run the QR promo value proposition experiment Aug 26, 2026

bcolsson approved these changes Aug 26, 2026

View reviewed changes

vbudhram requested a balanced review from Copilot August 31, 2026 21:27

Copilot started reviewing on behalf of vbudhram August 31, 2026 21:27 View session

Copilot AI reviewed Aug 31, 2026

View reviewed changes

vbudhram force-pushed the fxa-14273 branch from 1baa046 to 307beb1 Compare September 2, 2026 14:27

vbudhram requested a balanced review from Copilot September 3, 2026 16:05

Copilot started reviewing on behalf of vbudhram September 3, 2026 16:05 View session

Copilot AI reviewed Sep 3, 2026

View reviewed changes

dschom reviewed Sep 3, 2026

View reviewed changes

dschom approved these changes Sep 4, 2026

View reviewed changes

vbudhram force-pushed the fxa-14273 branch from 307beb1 to 924c872 Compare September 4, 2026 16:41

vbudhram merged commit 113caf6 into main Sep 4, 2026
18 of 20 checks passed

vbudhram deleted the fxa-14273 branch September 4, 2026 17:22

Navigation Menu

Uh oh!

feat(settings): run the QR promo value proposition experiment - #21112

feat(settings): run the QR promo value proposition experiment#21112
vbudhram merged 1 commit into
mainfrom
fxa-14273

vbudhram commented Aug 26, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

dschom Sep 3, 2026

Uh oh!

vbudhram Sep 4, 2026

Uh oh!

dschom Sep 3, 2026

Uh oh!

dschom Sep 3, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

vbudhram commented Aug 26, 2026

Because

This pull request

Issue that this pull request solves

Checklist

Other information

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Suppressed comments (1)

Uh oh!

dschom Sep 3, 2026

Choose a reason for hiding this comment

Uh oh!

vbudhram Sep 4, 2026

Choose a reason for hiding this comment

Uh oh!

dschom Sep 3, 2026

Choose a reason for hiding this comment

Uh oh!

dschom Sep 3, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants