Problem
The smoke test step in .ci/smoke-tests-common-validation.yml uses
continueOnError: true.
As a result, smoke test failures do not fail the PR pipeline. This can allow a
PR with failing smoke tests to appear successful.
Removing continueOnError alone may prevent diagnostic screenshots from being
published when a smoke test fails because subsequent steps do not run by
default.
Proposed Change
- Remove
continueOnError: true from the smoke test step so failures propagate
to the pipeline result.
- Add
condition: always() to the screenshot publishing step so diagnostic
screenshots are still uploaded after a failure.
Expected Behavior
- A smoke test failure causes the corresponding pipeline job to fail.
- Smoke test screenshots are published regardless of the test result.
Acceptance Criteria
- The pipeline fails when
npm run smoke-tests exits with a non-zero status.
- The screenshot publishing step runs after both successful and failed smoke
test executions.
- Successful smoke test runs continue to pass without behavioral changes.
Problem
The smoke test step in
.ci/smoke-tests-common-validation.ymlusescontinueOnError: true.As a result, smoke test failures do not fail the PR pipeline. This can allow a
PR with failing smoke tests to appear successful.
Removing
continueOnErroralone may prevent diagnostic screenshots from beingpublished when a smoke test fails because subsequent steps do not run by
default.
Proposed Change
continueOnError: truefrom the smoke test step so failures propagateto the pipeline result.
condition: always()to the screenshot publishing step so diagnosticscreenshots are still uploaded after a failure.
Expected Behavior
Acceptance Criteria
npm run smoke-testsexits with a non-zero status.test executions.