Fix: resolve TestHub build creation from the normalised product flags

[kamal-kaur04]

Summary

Build creation is gated on a strict enabled === true, while every other site reads the same setting for truthiness. Any config supplying a truthy-but-not-boolean value — "true" (what a YAML- or env-derived config produces), 1, or the key omitted entirely (the plugin's own default is ON) — therefore produces a run that:

• never requests a TestHub build (zero traffic to the collector),
• reports observability as enabled everywhere downstream, and
• stamps every Automate session with testhubBuildUuid: "", so the session can never be linked to reporting data.

The tests pass and nothing warns. The only symptom is missing data.

// src/utils/helper.js — before
exports.isTestHubBuild = (pluginSettings = {}, isBuildStart = false) => {
  if (isBuildStart) {
    return pluginSettings?.test_reporting?.enabled === true
        || pluginSettings?.test_observability?.enabled === true
        || pluginSettings?.accessibility === true;   // strict
  }
  return this.isTestObservabilitySession() || this.isAccessibilitySession();
};

// src/testObservability.js — configure(), before
process.env.BROWSERSTACK_TEST_OBSERVABILITY = observabilityConfig.enabled;  // raw value
// …and src/testObservability.js:103 reads the same setting with `||`

The two sites disagree about what "enabled" means: one silently skips build creation, the other goes on to declare the product active.

This arrived in v3.7.0 — the identical config creates a build on 3.5.0 and creates nothing from 3.8.0 through current 3.11.3, so upgrading is not a remedy for an affected user.

What this changes

1. helper.parseBooleanSetting() — one place that resolves a flag, accepting the boolean / string / numeric forms a nightwatch.conf.js, YAML or env-derived config actually produces. Returns undefined when the setting is absent, so "not set" stays distinguishable from "set to false".
2. configure() normalises before writing the env, and warns when a non-boolean value had to be interpreted. The top-level testObservability / testReporting flags get the same treatment — previously a top-level "false" was ignored outright.
3. The build-start decision now reads the resolved state instead of re-interpreting raw config, so it can no longer disagree with the product map. This is the fix proper; 1 and 2 are what stop it diverging again.
4. A warning when a run intended a TestHub build but ends without a uuid. This case was completely silent before, in both the never-attempted and the attempt-failed variant. The second needed an explicit intent marker, because handleErrorForObservability turns the product flags back off before the capabilities are written — so a naive "product enabled but no uuid" check stays quiet exactly when a build-start failure loses the data.

Resolution table after this change:

test_observability.enabled	Before	After
true	build created	build created
"true"	nothing sent	build created
1 / "1"	nothing sent	build created
omitted	nothing sent	build created
false / "false" / 0	nothing sent	nothing sent (opt-out preserved)

No behaviour change for anyone already passing a real boolean.

Test plan

• npm test — 102 passing, 0 failing. New unit tests in test/src/utils/booleanSettings.js cover the parser, the build-start gate, and configure()'s normalisation.

• npm run eslint — clean.

• Wire-level A/B, no credentials — stub W3C hub + stub TestHub collector, nightwatch core 3.16.0, same test file and same config across arms. Signal is POST /api/v[12]/builds reaching the collector plus the testhubBuildUuid the hub actually received:

  enabled	3.5.0 (pre-gate)	3.11.3 published	this branch
  true	build	build	build
  "true"	build	none, uuid=""	build
  1	none	none, uuid=""	build
  omitted	build	none, uuid=""	build
  false	—	none	none

• Live A/B against BrowserStack — one test, test_observability.enabled: 'true' in both arms, only the plugin swapped:

  	published 3.11.3	this branch
  Automate build	e1fca429ba1454eed2524786b2a9a8478c6b7def	da5f372e86e50aa917f2916255d723f97c317d8a
  Automate session	8815e6dc54f12b9c1c8cb724b841fdedf32aabce	8c5bab3080c01a9611cc25c1b7f8fba1d7616327
  Test result	PASSED	PASSED
  TestHub build	none created	mxwufkw6jspxiiljalnblsznazkas7bdepicnjxx
  Session linked to a test run	n/a — no build exists	yes

  The pre-fix run prints no build-report URL at all; the post-fix run prints one, and that build's test run carries session_id = 8c5bab3080c01a9611cc25c1b7f8fba1d7616327 — i.e. the same Automate session, now linked.

• Opt-out re-checked: enabled: false and enabled: 'false' create no build on this branch, as before.

Measured production impact before the fix: a four-figure count of Automate sessions per week, spread over five accounts, reporting observability as enabled with an empty testhubBuildUuid — and three of those accounts are on 3.11.2, which is what confirms this is config-dependent rather than version-dependent. Accounts on the same versions with a boolean in their config are unaffected.

Not included here, on purpose

An accessibility-only run still never creates a build: launchTestSession() is nested inside if (helper.isTestObservabilitySession()) (nightwatch/globals.js), so the accessibility branch of the build-start decision is unreachable whenever reporting is explicitly off — and BROWSERSTACK_ACCESSIBILITY='true' is only ever set from the build-start response, so accessibility: true with test_observability.enabled: false silently disables accessibility as well. Hoisting the call would route those runs into the existing observability-denied path, which never stops the build and leaves the process wedged; that wants fixing first. Tracked separately (internal ref SDK-7538).

🤖 Generated with Claude Code