diff --git a/projects/web-site/public/article-images/b3874dc571f7da.svg b/projects/web-site/public/article-images/b3874dc571f7da.svg new file mode 100644 index 0000000..9da9e87 --- /dev/null +++ b/projects/web-site/public/article-images/b3874dc571f7da.svg @@ -0,0 +1 @@ +The Library Grew by 0.1KB, but the Angular App's Initial JavaScript Shrank by 685KBCover image for The Library Grew by 0.1KB, but the Angular App's Initial JavaScript Shrank by 685KB🐕The Library Grew by 0.1KB, but theAngular App's Initial JavaScriptShrank by 685KBrdlabo.dev · English developer article diff --git a/projects/web-site/public/article-images/npm-beta-per-pull-request.svg b/projects/web-site/public/article-images/npm-beta-per-pull-request.svg new file mode 100644 index 0000000..a4ed374 --- /dev/null +++ b/projects/web-site/public/article-images/npm-beta-per-pull-request.svg @@ -0,0 +1 @@ +Let Users Try This PR with npm install: Building Per-PR Beta Releases for OSSCover image for Let Users Try This PR with npm install: Building Per-PR Beta Releases for OSS🧪Let Users Try This PR with npminstall: Building Per-PR BetaReleases for OSSrdlabo.dev · English developer article diff --git a/projects/web-site/public/article-images/playwright-visual-regression-ci-baseline.svg b/projects/web-site/public/article-images/playwright-visual-regression-ci-baseline.svg new file mode 100644 index 0000000..833fb26 --- /dev/null +++ b/projects/web-site/public/article-images/playwright-visual-regression-ci-baseline.svg @@ -0,0 +1 @@ +Visual Regression Testing for OSS: Mac-Independent Baselines Updated Only in CICover image for Visual Regression Testing for OSS: Mac-Independent Baselines Updated Only in CI📸Visual Regression Testing for OSS:Mac-Independent Baselines UpdatedOnly in CIrdlabo.dev · English developer article diff --git a/projects/web-site/public/images/npm-beta-per-pull-request/pr-beta-published-comment.png b/projects/web-site/public/images/npm-beta-per-pull-request/pr-beta-published-comment.png new file mode 100644 index 0000000..e811f60 Binary files /dev/null and b/projects/web-site/public/images/npm-beta-per-pull-request/pr-beta-published-comment.png differ diff --git a/projects/web-site/public/images/playwright-visual-regression-ci-baseline/radio-actual.png b/projects/web-site/public/images/playwright-visual-regression-ci-baseline/radio-actual.png new file mode 100644 index 0000000..fd2b89b Binary files /dev/null and b/projects/web-site/public/images/playwright-visual-regression-ci-baseline/radio-actual.png differ diff --git a/projects/web-site/public/images/playwright-visual-regression-ci-baseline/radio-diff.png b/projects/web-site/public/images/playwright-visual-regression-ci-baseline/radio-diff.png new file mode 100644 index 0000000..1205af2 Binary files /dev/null and b/projects/web-site/public/images/playwright-visual-regression-ci-baseline/radio-diff.png differ diff --git a/projects/web-site/public/images/playwright-visual-regression-ci-baseline/radio-expected.png b/projects/web-site/public/images/playwright-visual-regression-ci-baseline/radio-expected.png new file mode 100644 index 0000000..6c026e0 Binary files /dev/null and b/projects/web-site/public/images/playwright-visual-regression-ci-baseline/radio-expected.png differ diff --git a/projects/web-site/public/sitemap.xml b/projects/web-site/public/sitemap.xml index 92bf1f3..3d9b599 100644 --- a/projects/web-site/public/sitemap.xml +++ b/projects/web-site/public/sitemap.xml @@ -9,6 +9,9 @@ https://rdlabo.dev/articles/archive/2022 https://rdlabo.dev/articles/archive/2021 https://rdlabo.dev/articles/archive/2020 + https://rdlabo.dev/articles/playwright-visual-regression-ci-baseline + https://rdlabo.dev/articles/npm-beta-per-pull-request + https://rdlabo.dev/articles/b3874dc571f7da https://rdlabo.dev/articles/ionic-themes-ionic9-major-update https://rdlabo.dev/articles/ionic-theme-md3 https://rdlabo.dev/articles/ionic-9-components-got-better diff --git a/projects/web-site/src/app/generated/article-catalog.generated.ts b/projects/web-site/src/app/generated/article-catalog.generated.ts index c4e52e2..fcdb7b5 100644 --- a/projects/web-site/src/app/generated/article-catalog.generated.ts +++ b/projects/web-site/src/app/generated/article-catalog.generated.ts @@ -1,5 +1,44 @@ // Generated by scripts/generate-articles.ts. Do not edit. export const ARTICLE_SUMMARIES = [ + { + "slug": "playwright-visual-regression-ci-baseline", + "title": "Visual Regression Testing for OSS: Mac-Independent Baselines Updated Only in CI", + "description": "How an Ionic theme project uses Playwright in a fixed Linux container, maintainer-authorized screenshot updates, and separate read/write jobs to keep visual baselines reproducible and reviewable.", + "image": "https://rdlabo.dev/article-images/playwright-visual-regression-ci-baseline.svg", + "imageWidth": 1200, + "imageHeight": 630, + "emoji": "📸", + "sourceName": "Zenn", + "originalUrl": "https://zenn.dev/rdlabo/articles/playwright-visual-regression-ci-baseline", + "publishedAt": "2026-08-30T03:52:15.000Z", + "publishedDate": "2026-08-30" + }, + { + "slug": "npm-beta-per-pull-request", + "title": "Let Users Try This PR with npm install: Building Per-PR Beta Releases for OSS", + "description": "How I built a secure per-PR npm beta release flow with GitHub Actions, OIDC trusted publishing, immutable versions, and maintainer-approved /beta commands.", + "image": "https://rdlabo.dev/article-images/npm-beta-per-pull-request.svg", + "imageWidth": 1200, + "imageHeight": 630, + "emoji": "🧪", + "sourceName": "Zenn", + "originalUrl": "https://zenn.dev/rdlabo/articles/npm-beta-per-pull-request", + "publishedAt": "2026-08-30T03:26:43.000Z", + "publishedDate": "2026-08-30" + }, + { + "slug": "b3874dc571f7da", + "title": "The Library Grew by 0.1KB, but the Angular App's Initial JavaScript Shrank by 685KB", + "description": "Why replacing Ionic root imports with granular entrypoints in a provider library cut an Angular app's initial synchronous JavaScript closure by 36.5%, even though the library itself became slightly larger.", + "image": "https://rdlabo.dev/article-images/b3874dc571f7da.svg", + "imageWidth": 1200, + "imageHeight": 630, + "emoji": "🐕", + "sourceName": "Zenn", + "originalUrl": "https://zenn.dev/rdlabo/articles/b3874dc571f7da", + "publishedAt": "2026-08-28T09:52:27.000Z", + "publishedDate": "2026-08-28" + }, { "slug": "ionic-themes-ionic9-major-update", "title": "Bringing iOS 26 and Material Design 3 to Ionic: Both Themes Reach v9", diff --git a/projects/web-site/src/app/generated/article-loaders.generated.ts b/projects/web-site/src/app/generated/article-loaders.generated.ts index 5c13255..b0543fc 100644 --- a/projects/web-site/src/app/generated/article-loaders.generated.ts +++ b/projects/web-site/src/app/generated/article-loaders.generated.ts @@ -1,5 +1,8 @@ // Generated by scripts/generate-articles.ts. Do not edit. export const ARTICLE_LOADERS: Record Promise<{ default: { html: string; headings: readonly { id: string; text: string; level: 2 | 3 }[] } }>> = { + "playwright-visual-regression-ci-baseline": () => import('./articles/playwright-visual-regression-ci-baseline.generated'), + "npm-beta-per-pull-request": () => import('./articles/npm-beta-per-pull-request.generated'), + "b3874dc571f7da": () => import('./articles/b3874dc571f7da.generated'), "ionic-themes-ionic9-major-update": () => import('./articles/ionic-themes-ionic9-major-update.generated'), "ionic-theme-md3": () => import('./articles/ionic-theme-md3.generated'), "ionic-9-components-got-better": () => import('./articles/ionic-9-components-got-better.generated'), diff --git a/projects/web-site/src/app/generated/articles/b3874dc571f7da.generated.ts b/projects/web-site/src/app/generated/articles/b3874dc571f7da.generated.ts new file mode 100644 index 0000000..3fc25f2 --- /dev/null +++ b/projects/web-site/src/app/generated/articles/b3874dc571f7da.generated.ts @@ -0,0 +1,2 @@ +// Generated by scripts/generate-articles.ts. Do not edit. +export default {"html":"

To speed up an Angular app's startup, I was changing Ionic imports to more granular entrypoints.

\n

Making the same change in the app itself barely affected the initial bundle. However, making it in a shared library used from ApplicationConfig.providers produced a dramatically different result.

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
SeatKeep production buildInitial JS synchronous closure
Before1,875,956 bytes
After1,190,987 bytes
Difference−684,969 bytes (−36.5%)
\n

Interestingly, the shared library itself did not get smaller.

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
@rdlabo/ionic-angular-kitnpm tarballRoot FESM
22.0.0-5397.0KB114.0KB
22.0.0-6397.1KB114.3KB
\n

The library grew by 0.1KB, yet the initial JavaScript of the app using it shrank by about 685KB.

\n

Why did an import change that had almost no effect in the app itself make such a difference in a shared library?

\n

As I investigated, I found that the important factor was not the amount of code in the library, but the dependency graph the library passed on to the consuming app. In particular, a library registered with Angular's providers and loaded at startup affects initial JavaScript differently from a component on a lazy-loaded page.

\n

This article uses the actual change to separate Angular DI from JavaScript loading, then examines the Ionic 9 granular entrypoints that made the improvement possible.

\n

The only change was where Ionic was imported from

\n

The library in question is @rdlabo/ionic-angular-kit, which is used by multiple Ionic Angular apps. It provides authentication, HTTP interceptors, overlays, storage, and other features, and apps register it as follows.

\n
import {\n  provideKitHttp,\n  provideKitOverlay,\n} from '@rdlabo/ionic-angular-kit';\n\nexport const appConfig: ApplicationConfig = {\n  providers: [\n    provideKitHttp(() => ({ /* ... */ })),\n    provideKitOverlay(),\n  ],\n};\n

In broad terms, the kit's public FESM previously imported Ionic controllers from the root entrypoint.

\n
import {\n  ActionSheetController,\n  AlertController,\n  LoadingController,\n  ModalController,\n  NavController,\n  PopoverController,\n  ToastController,\n} from '@ionic/angular';\n

I changed these to the individual entrypoints exposed by Ionic 9.

\n
import { ActionSheetController } from '@ionic/angular/action-sheet-controller';\nimport { AlertController } from '@ionic/angular/alert-controller';\nimport { LoadingController } from '@ionic/angular/loading-controller';\nimport { ModalController } from '@ionic/angular/modal-controller';\nimport { PopoverController } from '@ionic/angular/popover-controller';\nimport { ToastController } from '@ionic/angular/toast-controller';\nimport { NavController } from '@ionic/angular/common';\n

You can review the change in the pull request.

\n

https://github.com/rdlabo-dev/ionic-angular-library/pull/91

\n

The figures in this article come from rebuilding SeatKeep commits 3d905d17364c and 36943f34cf3e with the following command. The only change between these commits is the kit update from 22.0.0-5 to 22.0.0-6.

\n
npx ng build app --configuration production --stats-json\n

Within stats.json, I treated outputs whose entrypoints were src/main.ts and angular:polyfills as roots. From there, I recursively followed only import-statement edges, excluding dynamic-import, source maps, and CSS. The table reports the sum of stats.outputs[*].bytes for the JavaScript reached this way. These are uncompressed raw bytes, not gzip or Brotli sizes.

\n

The public API and Angular DI tokens did not change. The consuming app only needed to update the kit version.

\n
変更前\napp.config\n  └─ kit root\n       └─ @ionic/angular root\n            ├─ Controller\n            ├─ Component\n            ├─ generated proxies\n            └─ Ionic Core utility\n\n変更後\napp.config\n  └─ kit root\n       ├─ @ionic/angular/alert-controller\n       ├─ @ionic/angular/modal-controller\n       ├─ @ionic/angular/loading-controller\n       └─ @ionic/angular/common\n

The kit's FESM stayed almost the same size. From the consuming app's perspective, however, the entrypoint for static imports became narrower, removing Ionic's standalone root index from the initial JavaScript's synchronous closure.

\n

Here, “synchronous closure” means the JavaScript reached by recursively following static imports from main and polyfills in the production build's stats.json. This compares raw bytes; it does not mean that compressed transfer size or actual browser startup time improved by 36.5%.

\n

Even so, the production build confirmed that dependencies unnecessary for the initial route were removed from the synchronous closure instead of merely moving between chunks.

\n

Lazy creation through DI and lazy loading of JavaScript are different

\n

To understand this result, separate instance creation through dependency injection from ES module loading.

\n

Suppose a library exposes the following function.

\n
export const provideLibrary = (): EnvironmentProviders => {\n  return makeEnvironmentProviders([\n    LibraryOverlayService,\n  ]);\n};\n

The app registers it in ApplicationConfig.providers.

\n
import { provideLibrary } from '@example/library';\n\nexport const appConfig: ApplicationConfig = {\n  providers: [\n    provideLibrary(),\n  ],\n};\n

The LibraryOverlayService instance might not be created until it is actually injected. But calling provideLibrary() first requires loading the JavaScript for @example/library.

\n
app.config.ts\n  └─ @example/library\n       ├─ provideLibrary\n       ├─ LibraryOverlayService\n       └─ Serviceが実行時にimportする依存\n

Allowing DI to create a service lazily is not the same as keeping the JavaScript that defines the class out of the initial bundle.

\n

Angular's official documentation explains that providers can be registered at application bootstrap, on a component or directive, or on a route, with different scopes and lifecycles.

\n

https://angular.dev/guide/di/defining-dependency-providers

\n

From the bundle's perspective, a library statically imported by an initial file such as app.config.ts enters the initial dependency graph regardless of when the service is instantiated.

\n

The cause is not the provider mechanism itself. What matters is that registering the provider requires statically importing the library from an initial file.

\n

Ionic 9 root and granular entrypoints

\n

Ionic 9's package structure made this improvement possible.

\n

The root of @ionic/angular in Ionic 9 points to the standalone-oriented index. It is a convenient entrypoint, but it re-exports not only controllers, but also navigation features, standalone components, generated proxies, and Ionic Core utilities.

\n

At the same time, Ionic 9 exposes separate entrypoints for components, controllers, common, and provide.

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
PurposeExample import
Convenient root entrypoint@ionic/angular
Component@ionic/angular/ion-button
Overlay controller@ionic/angular/alert-controller
Shared Angular features@ionic/angular/common
Bootstrap provider@ionic/angular/provide
\n

The complete export list is available in Ionic 9.0.0's package.json.

\n

https://github.com/ionic-team/ionic-framework/blob/v9.0.0/packages/angular/package.json

\n

The root entrypoint is not inherently bad. Importing multiple components from one place offers a good developer experience, and tree-shaking is sufficient in many situations.

\n

On the other hand, narrowing runtime value imports can be especially valuable in a shared library that must be loaded at startup. Controllers and providers have clear usage targets, making them a good fit for granular entrypoints.

\n

A type-only import type disappears from the JavaScript output. Type imports and runtime value imports can therefore be audited separately.

\n
import type { AlertOptions } from '@ionic/angular';\nimport { AlertController } from '@ionic/angular/alert-controller';\n

Why was the difference small on the app's lazy-loaded pages?

\n

Pages in an app are often lazy-loaded by route.

\n
export const routes: Routes = [\n  {\n    path: 'settings',\n    loadComponent: () =>\n      import('./settings.page').then((m) => m.SettingsPage),\n  },\n];\n

An Ionic component referenced only by SettingsPage will normally remain in the lazy chunk.

\n
import { IonButton } from '@ionic/angular';\n

With Ionic 9, this can also be changed to an individual entrypoint.

\n
import { IonButton } from '@ionic/angular/ion-button';\n

However, if the Angular builder was already removing unused exports and splitting dependencies into appropriate lazy chunks, the initial bundle may barely change. A dependency may also enter a shared or initial chunk if it is used on the startup path as well.

\n

Tree-shaking and code splitting are different. Removing unused code from the final bundle does not guarantee that chunk boundaries will be divided exactly as desired. An Ionic issue also discussed cases where tree-shaking of standalone components worked but code splitting was less than ideal because of Webpack or esbuild constraints.

\n

https://github.com/ionic-team/ionic-framework/issues/30114

\n

This does not mean deep imports have no value in app code. The difference can simply be small relative to the change when lazy loading and tree-shaking are already working on a page.

\n

In contrast, every app loaded this kit FESM at startup. Fixing one root import narrowed the initial dependencies of multiple apps at once. That is why the library-side change had such a large effect.

\n

Moving a provider to route providers alone does not make it lazy-loaded

\n

Moving a heavy provider into route scope may look like it would remove it from the initial bundle. However, statically importing that provider from the root route definition keeps its JavaScript in the root dependency graph.

\n
// app.routes.ts\nimport { provideAdminFeature } from '@example/admin';\n\nexport const routes: Routes = [\n  {\n    path: 'admin',\n    providers: [provideAdminFeature()],\n    loadChildren: () => import('./admin/admin.routes'),\n  },\n];\n

The provider's scope is limited to the admin route, but provideAdminFeature is still statically imported from app.routes.ts. DI scope and bundle boundaries are different.

\n

To lazy-load the JavaScript that defines the provider as well, register it in the routes file that is itself lazy-loaded.

\n
// app.routes.ts\nexport const routes: Routes = [\n  {\n    path: 'admin',\n    loadChildren: () => import('./admin/admin.routes'),\n  },\n];\n
// admin/admin.routes.ts\nimport { provideAdminFeature } from '@example/admin';\n\nexport const routes: Routes = [\n  {\n    path: '',\n    providers: [provideAdminFeature()],\n    loadComponent: () => import('./admin.page'),\n  },\n];\n

Angular Router creates an EnvironmentInjector from a route's providers for use by that route and its children.

\n

https://angular.dev/guide/routing/define-routes

\n

You therefore need to inspect the file boundary as well, depending on whether the goal of moving a provider is only to change the instance scope or also to lazy-load its JavaScript.

\n

Measure the effect in the consuming app, not the library

\n

In this case, the npm tarball and root FESM grew slightly, while the consuming app's initial JavaScript shrank.

\n

Library package size, public FESM size, and the consuming app's initial chunk are different metrics. When optimizing a library that provides providers, you need to production-build a real consuming app and compare the synchronous closure from stats.json.

\n

Actual browser startup time should be measured separately.

\n
// main.ts\nperformance.mark('script-start');\n\nbootstrapApplication(AppComponent, appConfig);\n

This mark records when the body of main.ts begins executing. With ES modules, static import resolution and evaluation of imported modules happen before that point.

\n
navigation start\n  ├─ HTML/module download\n  ├─ 依存グラフの解決\n  ├─ parse/compile/link\n  ├─ import先moduleの評価\n  └─ main.ts本文\n       └─ script-start\n

The interval from navigation start to script-start can include not only network transfer, but also loading the dependency graph, parsing, compilation, linking, and evaluating imported modules.

\n

A 36.5% reduction in the synchronous closure does not guarantee a 36.5% reduction in startup time. Check Evaluate Script and Compile Script in a Chrome Performance trace, and compare before and after using fresh browser profiles.

\n

Use the production build to verify that an initial dependency was actually removed, and a real browser to determine how many seconds changed. Keeping these measurements separate makes it less likely that moving code between bundles will be mistaken for a speedup.

\n

What to check in libraries registered with providers

\n

Based on this experience, I now check the following in libraries registered with startup providers.

\n\n

Rather than enforcing deep imports uniformly across every app, I think it is easier to find meaningful improvements by first auditing shared libraries statically imported from startup files such as app.config.ts.

\n

When using ESLint, it also seems more practical to target runtime imports in shared libraries loaded at startup instead of banning @ionic/angular throughout the entire app.

\n

Conclusion

\n

The key to this improvement was not when DI creates a service, but which JavaScript the initial files statically import. Even before a service is instantiated, a library used by startup providers enters the initial dependency graph.

\n

The size of a library by itself is also insufficient for judging the effect of a change. Here, the library grew by 0.1KB, but production-building the consuming app showed that its initial JavaScript shrank by 685KB. Use a production build to verify that initial dependencies were removed, then a browser to measure the actual change in seconds.

\n

There is no need to mechanically ban every root import. For libraries registered with Angular's providers, however, the dependency graph passed to consuming apps deserves as much attention as the public API. Start with shared libraries loaded at startup, and you may find a substantial improvement.

\n

See you next time.

\n","headings":[{"id":"the-only-change-was-where-ionic-was-imported-from","text":"The only change was where Ionic was imported from","level":2},{"id":"lazy-creation-through-di-and-lazy-loading-of-javascript-are-different","text":"Lazy creation through DI and lazy loading of JavaScript are different","level":2},{"id":"ionic-9-root-and-granular-entrypoints","text":"Ionic 9 root and granular entrypoints","level":2},{"id":"why-was-the-difference-small-on-the-app's-lazy-loaded-pages%3F","text":"Why was the difference small on the app's lazy-loaded pages?","level":2},{"id":"moving-a-provider-to-route-providers-alone-does-not-make-it-lazy-loaded","text":"Moving a provider to route providers alone does not make it lazy-loaded","level":2},{"id":"measure-the-effect-in-the-consuming-app%2C-not-the-library","text":"Measure the effect in the consuming app, not the library","level":2},{"id":"what-to-check-in-libraries-registered-with-providers","text":"What to check in libraries registered with providers","level":2},{"id":"conclusion","text":"Conclusion","level":2}]} as const; diff --git a/projects/web-site/src/app/generated/articles/npm-beta-per-pull-request.generated.ts b/projects/web-site/src/app/generated/articles/npm-beta-per-pull-request.generated.ts new file mode 100644 index 0000000..f5d15f4 --- /dev/null +++ b/projects/web-site/src/app/generated/articles/npm-beta-per-pull-request.generated.ts @@ -0,0 +1,2 @@ +// Generated by scripts/generate-articles.ts. Do not edit. +export default {"html":"

“Could users put the changes from this PR into a real app and try them?”

\n

With an open source npm library, some things only become apparent after the package is installed in a real app, even after the code has passed review and CI. But if you publish a full release first just for testing, any problem you discover has already reached regular users. If you wait for the next full release, users cannot get even a small fix.

\n

I therefore introduced per-PR beta releases for the npm packages I maintain. Once a PR passes CI, an owner or maintainer can comment /beta to make that exact commit installable from npm.

\n

I am currently using this mechanism to test PR #131, “fix: support radio groups across inset lists,” for the iOS 26 theme.

\n

When I commented /beta on the CI-approved commit 8d96f41c7cb3, the comment received an 👀 reaction and the workflow replied with an install command for an immutable version.

\n

\"A

\n

GitHub Actions posted this command to the PR automatically after publishing. With this one line, users can put the PR's changes into their own app without cloning and building the repository.

\n

This lets them test changes in a real app before the full release. For a small change like this one, users who cannot wait for the full release can also choose to adopt just that fix early, with a clear understanding that it is a candidate build.

\n

Keep the roles of latest, next, and beta separate

\n

I treated npm dist-tags with the greatest care.

\n

Without an explicit --tag, npm publish updates latest. That is the tag many users get when they install a package without specifying a version. If the publication path for a PR package were to touch latest by mistake, code that was still being tested would reach regular users. An OSS release flow must prevent that completely.

\n

I divided publication into three explicit paths.

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
PurposeTriggernpm dist-tag
Full releasevX.Y.Z created by npm run releaselatest
Revision/prereleasevX.Y.Z-* created by npm run releasenext
PR candidateMaintainer /beta command or PR mergebeta
\n

The beta workflow verifies once more immediately before publishing that DIST_TAG is literally the string beta. It never offers latest or next, which belong to the full release flow, as selectable tags for a candidate release.

\n
if [ \"$DIST_TAG\" != \"beta\" ]; then\n  echo \"Refusing unsupported dist-tag: $DIST_TAG\"\n  exit 1\nfi\n\nnpm publish \"$PACKAGE_ARCHIVE\" \\\n  --ignore-scripts \\\n  --provenance \\\n  --access public \\\n  --tag beta\n

Publishing beta or next does not move latest. The existing full-release path and the path for distributing PR candidates remain separate.

\n

Implementing the design with GitHub Actions and OIDC

\n

That covers the publication-path design. Now, how does it work in practice?

\n

The workflow has four main stages.

\n
PRのcommit\n\n  ├─ リポジトリごとの必須CI\n\n  └─ build → npm pack → immutable artifact\n\nmaintainerが /beta            │\n  └─ 👀 reaction              │\n       └─ CI・権限・SHAを検証 ┘\n\n                    └─ OIDCでnpmへbeta publish\n\n                                   └─ PRへnpm installをコメント\n

The key design choice is that building the PR's code and publishing to npm do not happen in the same job.

\n

The YAML below omits some validation to make the relationship between the stages easier to see. You can inspect the actual workflows in Package Candidate and Release.

\n

1. Build the artifact on the PR side without publication permissions

\n

The Package Candidate workflow runs whenever a PR is opened or updated. It checks out the target head SHA, installs dependencies, builds the package, and runs npm pack.

\n
on:\n  pull_request:\n    branches: [main]\n    types: [opened, synchronize, reopened, ready_for_review]\n\npermissions:\n  contents: read\n  pull-requests: read\n\njobs:\n  pack:\n    steps:\n      - uses: actions/checkout@v6\n        with:\n          repository: ${{ github.event.pull_request.head.repo.full_name }}\n          ref: ${{ github.event.pull_request.head.sha }}\n          persist-credentials: false\n\n      - run: npm ci\n      - run: npm run build\n      - run: npm pack --ignore-scripts --pack-destination \"$RUNNER_TEMP/npm-candidate\"\n\n      - uses: actions/upload-artifact@v7\n        with:\n          name: npm-candidate-${{ github.event.pull_request.head.sha }}\n          path: ${{ runner.temp }}/npm-candidate/*.tgz\n

This workflow has neither id-token: write nor an npm token. Even if a malicious PR changes a build script, it cannot publish anything to npm from here.

\n

Before the build, the version is rewritten to <base>-beta.pr<PR number>.sha<SHA>. The artifact name also includes the full SHA, making it possible to verify later which workflow run built it and from which commit.

\n

2. First, respond to /beta with an 👀 reaction

\n

I considered using GitHub approval as the publication condition. However, if another commit is added after approval, the reviewed code and the code sent to npm are no longer the same.

\n

Instead, /beta authorizes only the head SHA that was current when the comment was posted, not the PR as a whole. No matter who adds the change, a new commit invalidates the previous authorization. The new SHA must pass CI, and an owner or maintainer must comment /beta again.

\n

The workflow receives /beta through GitHub's issue_comment event because regular comments on a PR are also handled by the Issue Comments API.

\n

After confirming that the comment body is an exact match and that the target is a PR, the workflow uses actions/github-script to check the commenter's permission. If it is admin or maintain, it attempts to add the 👀 reaction with peter-evans/create-or-update-comment.

\n
on:\n  issue_comment:\n    types: [created]\n\njobs:\n  beta-reaction:\n    if: >-\n      github.event.issue.pull_request &&\n      github.event.comment.body == '/beta'\n    permissions:\n      pull-requests: write\n\n    steps:\n      - name: Check owner or maintainer permission\n        id: permission\n        uses: actions/github-script@v8\n        with:\n          script: |\n            const { data } = await github.rest.repos.getCollaboratorPermissionLevel({\n              owner: context.repo.owner,\n              repo: context.repo.repo,\n              username: context.payload.comment.user.login,\n            });\n            return { authorized: ['admin', 'maintain'].includes(data.permission) };\n\n      - name: Add reaction to authorized beta comment\n        if: fromJson(steps.permission.outputs.result).authorized\n        uses: peter-evans/create-or-update-comment@v5\n        continue-on-error: true\n        with:\n          comment-id: ${{ github.event.comment.id }}\n          reactions: eyes\n

The 👀 means “command received”; it does not indicate a successful publication. The workflow then verifies that the PR is open and not a draft, along with the required CI, the PR head SHA, the commenter's permissions, and the candidate artifact. It checks the head SHA and permissions again immediately before publication. If CI is still running, it does not publish and instead attempts to post a comment explaining which checks remain. The maintainer must run /beta again after CI succeeds.

\n

3. Publish through OIDC without storing an npm token

\n

Publication uses npm Trusted Publishing.

\n

On npm, I register the GitHub repository and workflow file as a Trusted Publisher for the package. On the GitHub Actions side, only the publishing job receives id-token: write.

\n
publish-candidate:\n  needs: authorize-candidate\n  if: needs.authorize-candidate.outputs.allowed == 'true'\n  permissions:\n    actions: read\n    contents: read\n    id-token: write\n    pull-requests: read\n\n  steps:\n    - uses: actions/setup-node@v6\n      with:\n        node-version: 24\n        registry-url: https://registry.npmjs.org\n\n    - uses: actions/download-artifact@v8\n      with:\n        name: ${{ needs.authorize-candidate.outputs.artifact }}\n        run-id: ${{ needs.authorize-candidate.outputs.run-id }}\n\n    - run: |\n        npm publish \"$PACKAGE_ARCHIVE\" \\\n          --ignore-scripts \\\n          --provenance \\\n          --access public \\\n          --tag beta\n

GitHub Actions issues a short-lived OIDC ID token, and npm verifies that claims such as the repository and workflow match the registered publisher. There is no need to store a long-lived NPM_TOKEN in GitHub Secrets.

\n

OIDC does not, however, make the artifact's contents safe. OIDC controls who can publish. The artifact, SHA, and package identity checks must separately guarantee what gets published.

\n

The permission boundary would collapse if a job with id-token: write checked out and ran PR code. The publishing job therefore never runs PR code; it only downloads the .tgz created by the read-only workflow. It also compares the package name, version, registry, SHA, and the workflow run that created the artifact, publishing only when every value matches. If the PR changes the workflow used to make the release decision, publication from that PR is blocked as well.

\n

The workflow also uses --provenance, so the npm package records which source repository and GitHub Actions workflow built and published it.

\n

4. Comment with the install command only after publication succeeds

\n

The 👀 reaction and the publication-complete comment serve different purposes.

\n

The completion-comment job declares both authorization and publication as dependencies, and runs only if publish-candidate succeeds.

\n
comment-candidate:\n  needs: [authorize-candidate, publish-candidate]\n  if: >-\n    always() &&\n    needs.authorize-candidate.outputs.allowed == 'true' &&\n    needs.publish-candidate.result == 'success'\n  permissions:\n    issues: write\n    pull-requests: write\n\n  steps:\n    - name: Comment exact install command\n      uses: actions/github-script@v8\n

actions/github-script builds the command from the package name and immutable version.

\n
### npm beta published\n\nCI passed for commit `8d96f41c7cb3`. Install the immutable version with:\n\n```sh\nnpm install @rdlabo/ionic-theme-ios26@9.0.2-beta.pr131.sha8d96f41c7cb3\n```\n

The comment body also contains an HTML marker with the SHA. If the same workflow is rerun, it updates the existing comment for that SHA instead of adding another one.

\n

The beta dist-tag moves whenever the next candidate is published. The PR therefore tells users to install an immutable version containing the PR number and SHA, not @beta. Because the npm registry does not allow the contents of an existing version to be overwritten, the exact package that was tested remains reproducible later.

\n

I kept the comment step separate from the publishing job. If npm publication succeeds but the GitHub API temporarily fails, the release itself should not be rolled back into a failed state. The comment is best effort, and the install command also remains in the Job Summary.

\n

Automatic publication on merge joins the same publishing job. A push to main alone does not satisfy the trigger. After receiving workflow_run, the workflow verifies that the SHA is an actual PR merge commit and the current main HEAD, and that both the repository-specific required CI and Package Candidate have succeeded for the same SHA. Only then does it publish to beta. If the same version already exists on npm, it is not published again.

\n

Rolling it out to npm libraries listed on docs.rdlabo.dev

\n

I first tested this mechanism with the iOS 26 and Material Design 3 themes for Ionic. I then expanded it to the npm libraries listed on docs.rdlabo.dev. The design handles not only single-package repositories, but also repositories that publish multiple packages together, such as the Capacitor Stripe plugin.

\n

The repository-specific configuration varies: a single package, npm workspaces, custom build steps, or multiple required CI workflows. The publication boundaries nevertheless stay the same.

\n\n

Make the README an operating contract, not just the workflow

\n

If someone must read GitHub Actions YAML to learn the publication requirements, the OSS project has not documented its operation well enough. I therefore added a Prerelease channels section to each repository's README.

\n

The README explains that only an exact /beta command from an owner or maintainer is accepted, required CI must pass, and a new commit requires another /beta. The same place documents automatic publication after a PR merge, the fact that a direct push to main does not publish a candidate, and the roles of latest, next, and beta.

\n

A PR that changes a workflow used for the release decision cannot publish beta from itself. The workflow change must first be reviewed and merged into main; it becomes available to the next PR. This condition is documented in the README too, so the reason /beta did not work is not confined to an Actions log.

\n

The implementation enforces the boundary by stopping invalid publications. The README lets maintainers and contributors operate from the same shared assumptions. The code and documentation are maintained together as one release specification.

\n

Test with real users before the full release

\n

A pull request is not only a place to merge code. It can also be a place to hand a change to users and let them test it in their real projects.

\n

For an OSS library in particular, it is difficult for maintainers to reproduce every usage pattern in their own test environments. With per-PR npm packages, we can ask users to “install this one line and try it” instead of merely asking them to read the diff.

\n

This makes it easier to get feedback before the full release. Small fixes that cannot wait for the next release can reach the users who need them first. All the while, the latest tag used by regular users remains untouched.

\n

I have brought this quick path for testing PR changes into my own OSS projects. Going forward, I can validate not only the code review, but also the result of putting the change into a real app before the release.

\n

See you next time.

\n","headings":[{"id":"keep-the-roles-of-latest%2C-next%2C-and-beta-separate","text":"Keep the roles of latest, next, and beta separate","level":2},{"id":"implementing-the-design-with-github-actions-and-oidc","text":"Implementing the design with GitHub Actions and OIDC","level":2},{"id":"1.-build-the-artifact-on-the-pr-side-without-publication-permissions","text":"1. Build the artifact on the PR side without publication permissions","level":3},{"id":"2.-first%2C-respond-to-%2Fbeta-with-an-%F0%9F%91%80-reaction","text":"2. First, respond to /beta with an 👀 reaction","level":3},{"id":"3.-publish-through-oidc-without-storing-an-npm-token","text":"3. Publish through OIDC without storing an npm token","level":3},{"id":"4.-comment-with-the-install-command-only-after-publication-succeeds","text":"4. Comment with the install command only after publication succeeds","level":3},{"id":"rolling-it-out-to-npm-libraries-listed-on-docs.rdlabo.dev","text":"Rolling it out to npm libraries listed on docs.rdlabo.dev","level":2},{"id":"make-the-readme-an-operating-contract%2C-not-just-the-workflow","text":"Make the README an operating contract, not just the workflow","level":3},{"id":"test-with-real-users-before-the-full-release","text":"Test with real users before the full release","level":2}]} as const; diff --git a/projects/web-site/src/app/generated/articles/playwright-visual-regression-ci-baseline.generated.ts b/projects/web-site/src/app/generated/articles/playwright-visual-regression-ci-baseline.generated.ts new file mode 100644 index 0000000..15a0f5d --- /dev/null +++ b/projects/web-site/src/app/generated/articles/playwright-visual-regression-ci-baseline.generated.ts @@ -0,0 +1,2 @@ +// Generated by scripts/generate-articles.ts. Do not edit. +export default {"html":"

When operating visual regression tests in an OSS project, taking the screenshots was not the difficult part. The hard question was, “Whose environment produces the correct baseline?”

\n

For an Ionic theme I maintain, baseline images updated on a Mac did not match screenshots captured on Linux in GitHub Actions exactly. Even with the same code and Chromium version, small differences appeared in areas such as text edges.

\n

Instead of increasing the allowed difference, I decided that every baseline committed to the repository would be generated in one shared Linux container.

\n

Detecting an intentional change as a visual diff

\n

A real example is PR #131 for the iOS 26 theme.

\n

This PR added elements to the demo that correctly display a Radio Group spanning multiple Inset Lists. The code change was intentional, but those elements were absent from the saved baseline, so the visual regression test correctly failed.

\n

The previous Expected image is on the left, and the PR's Actual image is on the right.

\n\n\n\n\n\n\n\n\n\n\n\n\n\n
ExpectedActual
\"Baseline\"Actual
\n

In the Diff generated by Playwright, the added area appears in red.

\n

\"Difference

\n

Nothing is broken here; the diff is the result of intentionally changing the demo and theme. A visual regression test shows the difference, but the people reviewing the PR decide whether to adopt it as the new baseline.

\n

When a maintainer comments /update-screenshots on the PR, CI regenerates the images on Linux and adds them to that PR in a bot commit. Tests run again on that commit and verify that both Ionic 8 and Ionic 9 match the same new baseline.

\n

This PR initially detected visual differences in both Ionic 8 and 9. After the /update-screenshots update, both test suites passed.

\n

Start with Playwright's toHaveScreenshot

\n

The image comparison itself uses Playwright's toHaveScreenshot.

\n
await page.goto(route.path, { waitUntil: 'networkidle' });\nawait prepareScreenShot(page, route.name);\n\nawait expect(page).toHaveScreenshot(`${route.name}.png`, {\n  fullPage: true,\n  animations: 'disabled',\n  mask: [page.locator('ion-spinner')],\n});\n

The tests cover more than Buttons and Inputs. All demo routes and states—including Alerts, Action Sheets, Modals, Popovers, Toasts, and Tabs—are defined in an array. Each route is captured in both light and dark mode, producing more than 90 baseline images today.

\n

Before capturing dark mode, the test enables Ionic's class-based palette.

\n
await page.evaluate(() => {\n  document.documentElement.classList.add('ion-palette-dark');\n});\n\nawait expect(page).toHaveScreenshot(`${route.name}-dark.png`, {\n  fullPage: true,\n  animations: 'disabled',\n  mask: [page.locator('ion-spinner')],\n});\n

Visual regression tests need to minimize anything that changes from one capture to the next. These tests keep the following conditions consistent.

\n\n

The test code can be short; creating a stable screen is the more important part. If every capture includes changing dates, random values, network responses, or animations, the image comparison quickly stops being trustworthy.

\n

Do not turn OS differences into a debate over whose image is correct

\n

Playwright's official documentation also states that browser rendering varies with the OS, hardware, headless mode, and other factors, and recommends running tests in the same environment that generated the baselines. See Visual comparisons.

\n

Running --update-snapshots on a local Mac generates baselines with Chromium on macOS. Taking those images to GitHub Actions means comparing them with Chromium on Linux.

\n

Even when they look identical, font rendering, antialiasing, pixel rounding, and other details do not match perfectly. If fine-grained diffs appear even in unchanged regions, every PR must determine whether each difference is a real regression or merely an OS difference.

\n

Playwright provides maxDiffPixels, maxDiffPixelRatio, and threshold.

\n
await expect(page).toHaveScreenshot('button.png', {\n  maxDiffPixelRatio: 0.01,\n});\n

This could allow a test to pass when, for example, up to 1% of all pixels differ. A pixel count, however, cannot distinguish a small broken area at the edge of an image from a small OS-induced difference.

\n

What I need to protect in this CSS theme includes spacing of just a few pixels, borders, and text positions. I therefore left maxDiffPixels and maxDiffPixelRatio unset instead of widening the tolerance merely to make the tests pass.

\n

This choice is not about being strict for its own sake; it gives an OSS project a shared standard for making decisions. Instead of folding differences that occur only on one person's Mac into the allowed tolerance, comparing every PR in the same Linux environment lets contributors and maintainers inspect the same Diff.

\n

I standardized the comparison environment instead.

\n
jobs:\n  test:\n    runs-on: ubuntu-latest\n    container:\n      image: mcr.microsoft.com/playwright:v1.58.2-noble\n

Both PR tests and baseline updates run in the same Playwright Linux container. Pinning the container tag also limits the impact of browser updates on the runner.

\n

Manage permission to change baselines within the PR

\n

playwright test --update-snapshots can be run locally, but contributors no longer commit images generated on their own computers as the baseline for this OSS project.

\n

When the baseline needs to change, a maintainer posts the following comment on a PR in the same repository.

\n
/update-screenshots\n

GitHub Actions receives the issue_comment event and checks the commenter's repository permission. If they have admin, maintain, or write, the workflow adds an 👀 reaction to the comment. Updating the baseline requires authorization just as pushing code does.

\n
on:\n  issue_comment:\n    types: [created]\n\njobs:\n  pr-metadata:\n    if: >-\n      github.event.issue.pull_request &&\n      contains(github.event.comment.body, '/update-screenshots')\n

The actual update runs in a read-only job inside the pinned Playwright container.

\n
    \n
  1. Check out the PR head SHA that was current when the command was posted
  2. \n
  3. Run --update-snapshots with Ionic 9
  4. \n
  5. Switch to Ionic 8 and run the normal tests against the same images
  6. \n
  7. Save the updated images as an artifact
  8. \n
\n

I do not maintain separate baselines for Ionic 8 and 9. Ionic 9 updates the baseline, and the tests verify that Ionic 8 produces the same appearance.

\n
- name: Update screenshots with Ionic 9\n  run: npm run test:e2e:update\n  env:\n    IONIC_MAJOR: 9\n\n- name: Verify Ionic 8 against the same screenshots\n  run: npm run test:e2e\n  env:\n    IONIC_MAJOR: 8\n

Keeping separate images for the two versions would make both test suites easier to pass. The guarantee I want from this theme, however, is that “the same theme version looks the same on Ionic 8 and Ionic 9.” Sharing one set of baselines makes CI enforce that compatibility.

\n

Keep externally supplied code and write permission out of the same job

\n

A new commit might be added to the PR while the screenshots are being generated. Images created from an old SHA must not be committed to a new head.

\n

I therefore separated the job that generates the images from the job that commits them. The job that checks out and builds PR code has no write access to the repository.

\n
read-only job\n  PRのSHAをcheckout\n  Linuxで画像を更新\n  artifactへ保存\n\n\nwrite job\n  PRの現在のhead SHAを再取得\n  コメント時点のSHAと一致するか確認\n  GitHub APIでbot commit\n

Before committing, the workflow confirms that the PR's repository, branch, and head SHA are unchanged from when image generation began. If any value has changed, it fails and requires another /update-screenshots command.

\n

It also validates filenames so the artifact cannot introduce unexpected files. The commit uses GitHub's Git Data API and updates the branch with force: false.

\n

The workflow never uses permissions from the upstream repository to commit automatically to a fork PR. Automatic image updates work only for branches in the same repository. For a fork, the contributor must update images using the same Playwright Linux container, or a maintainer must bring the change onto a branch in the upstream repository. This limitation is somewhat inconvenient, but I prioritized avoiding a path that writes to a third party's branch with upstream permissions.

\n

After a successful update, the workflow posts the following comment on the PR.

\n
✅ Screenshots have been updated successfully!\n\nThe new screenshots have been committed to this PR.\n

The PR now contains the entire process: inspect the difference, update the baseline, and revalidate the result with Ionic 8 and 9.

\n

Let contributors and maintainers inspect the same Diff on the PR

\n

When a visual regression test fails, a long Actions log does not tell a contributor what changed. Review is not possible if only a maintainer's local environment can display the Diff.

\n

On failure, the workflow therefore saves Playwright's Expected, Actual, Diff, and trace files as artifacts. A summary of the Ionic 9 JSON report is posted as a PR comment, and the HTML report is published to a per-PR GitHub Pages directory. That report directory is removed after the PR is closed.

\n

CI uses a matrix for Ionic 8 and 9.

\n
strategy:\n  fail-fast: false\n  matrix:\n    include:\n      - ionic-major: 8\n        ionic-version: 8.8.19\n      - ionic-major: 9\n        ionic-version: 9.0.0\n

With fail-fast: false, one failure does not prevent the other result from being recorded. The PR can show whether the issue affects only Ionic 8, only Ionic 9, or whether both versions correctly detected the difference.

\n

OSS baselines are decided by review, not implementation

\n

/update-screenshots is useful, but it is not a command to run automatically whenever a test fails.

\n

A visual regression test tells us only one fact: the result differs from before. Contributors and maintainers must inspect Expected, Actual, and Diff to decide whether that difference is a fix or a regression.

\n

The correct order is:

\n
    \n
  1. The visual regression test detects a difference
  2. \n
  3. A person inspects Actual and Diff
  4. \n
  5. If the change is intentional, run /update-screenshots
  6. \n
  7. CI updates the baseline on Linux
  8. \n
  9. Ionic 8 and 9 are revalidated against the same images
  10. \n
\n

Increasing the tolerance makes CI quieter. Quiet CI and correct visuals are not the same thing.

\n

An OSS visual regression test must share not only its images, but also the process used to decide what is correct.

\n

The differences between Mac and Linux were not a reason to make comparisons more permissive; they were a reason to choose one place where baselines are created. Update baselines only in CI. Share the Diff on the PR and have people review it. Keep externally supplied code separate from permission to write to the repository.

\n

With these boundaries in place, no particular maintainer's computer defines correctness. Whoever opens a PR can inspect the same images and protect differences measured in just a few pixels.

\n

See you next time.

\n","headings":[{"id":"detecting-an-intentional-change-as-a-visual-diff","text":"Detecting an intentional change as a visual diff","level":2},{"id":"start-with-playwright's-tohavescreenshot","text":"Start with Playwright's toHaveScreenshot","level":2},{"id":"do-not-turn-os-differences-into-a-debate-over-whose-image-is-correct","text":"Do not turn OS differences into a debate over whose image is correct","level":2},{"id":"manage-permission-to-change-baselines-within-the-pr","text":"Manage permission to change baselines within the PR","level":2},{"id":"keep-externally-supplied-code-and-write-permission-out-of-the-same-job","text":"Keep externally supplied code and write permission out of the same job","level":2},{"id":"let-contributors-and-maintainers-inspect-the-same-diff-on-the-pr","text":"Let contributors and maintainers inspect the same Diff on the PR","level":2},{"id":"oss-baselines-are-decided-by-review%2C-not-implementation","text":"OSS baselines are decided by review, not implementation","level":2}]} as const; diff --git a/projects/web-site/src/articles/b3874dc571f7da.md b/projects/web-site/src/articles/b3874dc571f7da.md new file mode 100644 index 0000000..e50f526 --- /dev/null +++ b/projects/web-site/src/articles/b3874dc571f7da.md @@ -0,0 +1,305 @@ +--- +title: "The Library Grew by 0.1KB, but the Angular App's Initial JavaScript Shrank by 685KB" +description: "Why replacing Ionic root imports with granular entrypoints in a provider library cut an Angular app's initial synchronous JavaScript closure by 36.5%, even though the library itself became slightly larger." +zennSlug: b3874dc571f7da +emoji: "🐕" +publishedDate: "2026-08-28" +originalUrl: "https://zenn.dev/rdlabo/articles/b3874dc571f7da" +--- + +To speed up an Angular app's startup, I was changing Ionic imports to more granular entrypoints. + +Making the same change in the app itself barely affected the initial bundle. However, making it in a shared library used from `ApplicationConfig.providers` produced a dramatically different result. + +| SeatKeep production build | Initial JS synchronous closure | +| --- | --- | +| Before | 1,875,956 bytes | +| After | 1,190,987 bytes | +| Difference | −684,969 bytes (−36.5%) | + +Interestingly, the shared library itself did not get smaller. + +| @rdlabo/ionic-angular-kit | npm tarball | Root FESM | +| --- | --- | --- | +| 22.0.0-5 | 397.0KB | 114.0KB | +| 22.0.0-6 | 397.1KB | 114.3KB | + +The library grew by 0.1KB, yet the initial JavaScript of the app using it shrank by about 685KB. + +Why did an import change that had almost no effect in the app itself make such a difference in a shared library? + +As I investigated, I found that the important factor was not the amount of code in the library, but the dependency graph the library passed on to the consuming app. In particular, a library registered with Angular's `providers` and loaded at startup affects initial JavaScript differently from a component on a lazy-loaded page. + +This article uses the actual change to separate Angular DI from JavaScript loading, then examines the Ionic 9 granular entrypoints that made the improvement possible. + +# The only change was where Ionic was imported from + +The library in question is `@rdlabo/ionic-angular-kit`, which is used by multiple Ionic Angular apps. It provides authentication, HTTP interceptors, overlays, storage, and other features, and apps register it as follows. +```ts +import { + provideKitHttp, + provideKitOverlay, +} from '@rdlabo/ionic-angular-kit'; + +export const appConfig: ApplicationConfig = { + providers: [ + provideKitHttp(() => ({ /* ... */ })), + provideKitOverlay(), + ], +}; +``` +In broad terms, the kit's public FESM previously imported Ionic controllers from the root entrypoint. +```ts +import { + ActionSheetController, + AlertController, + LoadingController, + ModalController, + NavController, + PopoverController, + ToastController, +} from '@ionic/angular'; +``` +I changed these to the individual entrypoints exposed by Ionic 9. +```ts +import { ActionSheetController } from '@ionic/angular/action-sheet-controller'; +import { AlertController } from '@ionic/angular/alert-controller'; +import { LoadingController } from '@ionic/angular/loading-controller'; +import { ModalController } from '@ionic/angular/modal-controller'; +import { PopoverController } from '@ionic/angular/popover-controller'; +import { ToastController } from '@ionic/angular/toast-controller'; +import { NavController } from '@ionic/angular/common'; +``` +You can review the change in the pull request. + +[https://github.com/rdlabo-dev/ionic-angular-library/pull/91](https://github.com/rdlabo-dev/ionic-angular-library/pull/91) + +The figures in this article come from rebuilding SeatKeep commits `3d905d17364c` and `36943f34cf3e` with the following command. The only change between these commits is the kit update from `22.0.0-5` to `22.0.0-6`. +```shell +npx ng build app --configuration production --stats-json +``` +Within `stats.json`, I treated outputs whose entrypoints were `src/main.ts` and `angular:polyfills` as roots. From there, I recursively followed only `import-statement` edges, excluding `dynamic-import`, source maps, and CSS. The table reports the sum of `stats.outputs[*].bytes` for the JavaScript reached this way. These are uncompressed raw bytes, not gzip or Brotli sizes. + +The public API and Angular DI tokens did not change. The consuming app only needed to update the kit version. +```text +変更前 +app.config + └─ kit root + └─ @ionic/angular root + ├─ Controller + ├─ Component + ├─ generated proxies + └─ Ionic Core utility + +変更後 +app.config + └─ kit root + ├─ @ionic/angular/alert-controller + ├─ @ionic/angular/modal-controller + ├─ @ionic/angular/loading-controller + └─ @ionic/angular/common +``` +The kit's FESM stayed almost the same size. From the consuming app's perspective, however, the entrypoint for static imports became narrower, removing Ionic's standalone root index from the initial JavaScript's synchronous closure. + +Here, “synchronous closure” means the JavaScript reached by recursively following static imports from `main` and polyfills in the production build's `stats.json`. This compares raw bytes; it does not mean that compressed transfer size or actual browser startup time improved by 36.5%. + +Even so, the production build confirmed that dependencies unnecessary for the initial route were removed from the synchronous closure instead of merely moving between chunks. + +# Lazy creation through DI and lazy loading of JavaScript are different + +To understand this result, separate instance creation through dependency injection from ES module loading. + +Suppose a library exposes the following function. +```ts +export const provideLibrary = (): EnvironmentProviders => { + return makeEnvironmentProviders([ + LibraryOverlayService, + ]); +}; +``` +The app registers it in `ApplicationConfig.providers`. +```ts +import { provideLibrary } from '@example/library'; + +export const appConfig: ApplicationConfig = { + providers: [ + provideLibrary(), + ], +}; +``` +The `LibraryOverlayService` instance might not be created until it is actually injected. But calling `provideLibrary()` first requires loading the JavaScript for `@example/library`. +```text +app.config.ts + └─ @example/library + ├─ provideLibrary + ├─ LibraryOverlayService + └─ Serviceが実行時にimportする依存 +``` +Allowing DI to create a service lazily is not the same as keeping the JavaScript that defines the class out of the initial bundle. + +Angular's official documentation explains that providers can be registered at application bootstrap, on a component or directive, or on a route, with different scopes and lifecycles. + +[https://angular.dev/guide/di/defining-dependency-providers](https://angular.dev/guide/di/defining-dependency-providers) + +From the bundle's perspective, a library statically imported by an initial file such as `app.config.ts` enters the initial dependency graph regardless of when the service is instantiated. + +The cause is not the provider mechanism itself. What matters is that registering the provider requires statically importing the library from an initial file. + +# Ionic 9 root and granular entrypoints + +Ionic 9's package structure made this improvement possible. + +The root of `@ionic/angular` in Ionic 9 points to the standalone-oriented index. It is a convenient entrypoint, but it re-exports not only controllers, but also navigation features, standalone components, generated proxies, and Ionic Core utilities. + +At the same time, Ionic 9 exposes separate entrypoints for components, controllers, `common`, and `provide`. + +| Purpose | Example import | +| --- | --- | +| Convenient root entrypoint | @ionic/angular | +| Component | @ionic/angular/ion-button | +| Overlay controller | @ionic/angular/alert-controller | +| Shared Angular features | @ionic/angular/common | +| Bootstrap provider | @ionic/angular/provide | + +The complete export list is available in Ionic 9.0.0's `package.json`. + +[https://github.com/ionic-team/ionic-framework/blob/v9.0.0/packages/angular/package.json](https://github.com/ionic-team/ionic-framework/blob/v9.0.0/packages/angular/package.json) + +The root entrypoint is not inherently bad. Importing multiple components from one place offers a good developer experience, and tree-shaking is sufficient in many situations. + +On the other hand, narrowing runtime value imports can be especially valuable in a shared library that must be loaded at startup. Controllers and providers have clear usage targets, making them a good fit for granular entrypoints. + +A type-only `import type` disappears from the JavaScript output. Type imports and runtime value imports can therefore be audited separately. +```ts +import type { AlertOptions } from '@ionic/angular'; +import { AlertController } from '@ionic/angular/alert-controller'; +``` +# Why was the difference small on the app's lazy-loaded pages? + +Pages in an app are often lazy-loaded by route. +```ts +export const routes: Routes = [ + { + path: 'settings', + loadComponent: () => + import('./settings.page').then((m) => m.SettingsPage), + }, +]; +``` +An Ionic component referenced only by `SettingsPage` will normally remain in the lazy chunk. +```ts +import { IonButton } from '@ionic/angular'; +``` +With Ionic 9, this can also be changed to an individual entrypoint. +```ts +import { IonButton } from '@ionic/angular/ion-button'; +``` +However, if the Angular builder was already removing unused exports and splitting dependencies into appropriate lazy chunks, the initial bundle may barely change. A dependency may also enter a shared or initial chunk if it is used on the startup path as well. + +Tree-shaking and code splitting are different. Removing unused code from the final bundle does not guarantee that chunk boundaries will be divided exactly as desired. An Ionic issue also discussed cases where tree-shaking of standalone components worked but code splitting was less than ideal because of Webpack or esbuild constraints. + +[https://github.com/ionic-team/ionic-framework/issues/30114](https://github.com/ionic-team/ionic-framework/issues/30114) + +This does not mean deep imports have no value in app code. The difference can simply be small relative to the change when lazy loading and tree-shaking are already working on a page. + +In contrast, every app loaded this kit FESM at startup. Fixing one root import narrowed the initial dependencies of multiple apps at once. That is why the library-side change had such a large effect. + +# Moving a provider to route providers alone does not make it lazy-loaded + +Moving a heavy provider into route scope may look like it would remove it from the initial bundle. However, statically importing that provider from the root route definition keeps its JavaScript in the root dependency graph. +```ts +// app.routes.ts +import { provideAdminFeature } from '@example/admin'; + +export const routes: Routes = [ + { + path: 'admin', + providers: [provideAdminFeature()], + loadChildren: () => import('./admin/admin.routes'), + }, +]; +``` +The provider's scope is limited to the admin route, but `provideAdminFeature` is still statically imported from `app.routes.ts`. DI scope and bundle boundaries are different. + +To lazy-load the JavaScript that defines the provider as well, register it in the routes file that is itself lazy-loaded. +```ts +// app.routes.ts +export const routes: Routes = [ + { + path: 'admin', + loadChildren: () => import('./admin/admin.routes'), + }, +]; +``` +```ts +// admin/admin.routes.ts +import { provideAdminFeature } from '@example/admin'; + +export const routes: Routes = [ + { + path: '', + providers: [provideAdminFeature()], + loadComponent: () => import('./admin.page'), + }, +]; +``` +Angular Router creates an `EnvironmentInjector` from a route's `providers` for use by that route and its children. + +[https://angular.dev/guide/routing/define-routes](https://angular.dev/guide/routing/define-routes) + +You therefore need to inspect the file boundary as well, depending on whether the goal of moving a provider is only to change the instance scope or also to lazy-load its JavaScript. + +# Measure the effect in the consuming app, not the library + +In this case, the npm tarball and root FESM grew slightly, while the consuming app's initial JavaScript shrank. + +Library package size, public FESM size, and the consuming app's initial chunk are different metrics. When optimizing a library that provides providers, you need to production-build a real consuming app and compare the synchronous closure from `stats.json`. + +Actual browser startup time should be measured separately. +```ts +// main.ts +performance.mark('script-start'); + +bootstrapApplication(AppComponent, appConfig); +``` +This mark records when the body of `main.ts` begins executing. With ES modules, static import resolution and evaluation of imported modules happen before that point. +```text +navigation start + ├─ HTML/module download + ├─ 依存グラフの解決 + ├─ parse/compile/link + ├─ import先moduleの評価 + └─ main.ts本文 + └─ script-start +``` +The interval from navigation start to `script-start` can include not only network transfer, but also loading the dependency graph, parsing, compilation, linking, and evaluating imported modules. + +A 36.5% reduction in the synchronous closure does not guarantee a 36.5% reduction in startup time. Check `Evaluate Script` and `Compile Script` in a Chrome Performance trace, and compare before and after using fresh browser profiles. + +Use the production build to verify that an initial dependency was actually removed, and a real browser to determine how many seconds changed. Keeping these measurements separate makes it less likely that moving code between bundles will be mistaken for a speedup. + +# What to check in libraries registered with `providers` + +Based on this experience, I now check the following in libraries registered with startup `providers`. + +- Whether the root entrypoint exports optional runtime features +- Whether provider implementations import a dependency library's root barrel +- Whether type-only and runtime imports are separated with `import type` +- Whether heavy features can be separated into secondary entrypoints +- Whether route-specific providers can be placed in lazy-loaded route files +- Whether the synchronous closure was compared in a production build of a consuming app +- Whether parse, compile, and evaluate time was measured with a fresh browser profile + +Rather than enforcing deep imports uniformly across every app, I think it is easier to find meaningful improvements by first auditing shared libraries statically imported from startup files such as `app.config.ts`. + +When using ESLint, it also seems more practical to target runtime imports in shared libraries loaded at startup instead of banning `@ionic/angular` throughout the entire app. + +# Conclusion + +The key to this improvement was not when DI creates a service, but which JavaScript the initial files statically import. Even before a service is instantiated, a library used by startup `providers` enters the initial dependency graph. + +The size of a library by itself is also insufficient for judging the effect of a change. Here, the library grew by 0.1KB, but production-building the consuming app showed that its initial JavaScript shrank by 685KB. Use a production build to verify that initial dependencies were removed, then a browser to measure the actual change in seconds. + +There is no need to mechanically ban every root import. For libraries registered with Angular's `providers`, however, the dependency graph passed to consuming apps deserves as much attention as the public API. Start with shared libraries loaded at startup, and you may find a substantial improvement. + +See you next time. diff --git a/projects/web-site/src/articles/npm-beta-per-pull-request.md b/projects/web-site/src/articles/npm-beta-per-pull-request.md new file mode 100644 index 0000000..b3fde1c --- /dev/null +++ b/projects/web-site/src/articles/npm-beta-per-pull-request.md @@ -0,0 +1,285 @@ +--- +title: "Let Users Try This PR with npm install: Building Per-PR Beta Releases for OSS" +description: "How I built a secure per-PR npm beta release flow with GitHub Actions, OIDC trusted publishing, immutable versions, and maintainer-approved /beta commands." +zennSlug: npm-beta-per-pull-request +emoji: "🧪" +publishedDate: "2026-08-30" +originalUrl: "https://zenn.dev/rdlabo/articles/npm-beta-per-pull-request" +--- + +“Could users put the changes from this PR into a real app and try them?” + +With an open source npm library, some things only become apparent after the package is installed in a real app, even after the code has passed review and CI. But if you publish a full release first just for testing, any problem you discover has already reached regular users. If you wait for the next full release, users cannot get even a small fix. + +I therefore introduced per-PR `beta` releases for the npm packages I maintain. Once a PR passes CI, an owner or maintainer can comment `/beta` to make that exact commit installable from npm. + +I am currently using this mechanism to test [PR #131, “fix: support radio groups across inset lists,”](https://github.com/rdlabo-dev/ionic-theme-ios26/pull/131) for the iOS 26 theme. + +When I commented `/beta` on the CI-approved [commit `8d96f41c7cb3`](https://github.com/rdlabo-dev/ionic-theme-ios26/commit/8d96f41c7cb353de42563191f36eb6c7b6ed453e), the comment received an 👀 reaction and the workflow replied with an install command for an immutable version. + +![A /beta comment on a pull request followed by a GitHub Actions reply containing the immutable npm beta install command](/images/npm-beta-per-pull-request/pr-beta-published-comment.png) + +GitHub Actions posted this command to the PR automatically after publishing. With this one line, users can put the PR's changes into their own app without cloning and building the repository. + +This lets them test changes in a real app before the full release. For a small change like this one, users who cannot wait for the full release can also choose to adopt just that fix early, with a clear understanding that it is a candidate build. + +## Keep the roles of `latest`, `next`, and `beta` separate + +I treated npm dist-tags with the greatest care. + +Without an explicit `--tag`, `npm publish` updates `latest`. That is the tag many users get when they install a package without specifying a version. If the publication path for a PR package were to touch `latest` by mistake, code that was still being tested would reach regular users. An OSS release flow must prevent that completely. + +I divided publication into three explicit paths. + +| Purpose | Trigger | npm dist-tag | +| --- | --- | --- | +| Full release | `vX.Y.Z` created by `npm run release` | `latest` | +| Revision/prerelease | `vX.Y.Z-*` created by `npm run release` | `next` | +| PR candidate | Maintainer `/beta` command or PR merge | `beta` | + +The `beta` workflow verifies once more immediately before publishing that `DIST_TAG` is literally the string `beta`. It never offers `latest` or `next`, which belong to the full release flow, as selectable tags for a candidate release. + +```bash +if [ "$DIST_TAG" != "beta" ]; then + echo "Refusing unsupported dist-tag: $DIST_TAG" + exit 1 +fi + +npm publish "$PACKAGE_ARCHIVE" \ + --ignore-scripts \ + --provenance \ + --access public \ + --tag beta +``` + +Publishing `beta` or `next` does not move `latest`. The existing full-release path and the path for distributing PR candidates remain separate. + +## Implementing the design with GitHub Actions and OIDC + +That covers the publication-path design. Now, how does it work in practice? + +The workflow has four main stages. + +```text +PRのcommit + │ + ├─ リポジトリごとの必須CI + │ + └─ build → npm pack → immutable artifact + │ +maintainerが /beta │ + └─ 👀 reaction │ + └─ CI・権限・SHAを検証 ┘ + │ + └─ OIDCでnpmへbeta publish + │ + └─ PRへnpm installをコメント +``` + +The key design choice is that building the PR's code and publishing to npm do not happen in the same job. + +The YAML below omits some validation to make the relationship between the stages easier to see. You can inspect the actual workflows in [Package Candidate](https://github.com/rdlabo-dev/ionic-theme-ios26/blob/main/.github/workflows/package-candidate.yml) and [Release](https://github.com/rdlabo-dev/ionic-theme-ios26/blob/main/.github/workflows/release.yml). + +### 1. Build the artifact on the PR side without publication permissions + +The `Package Candidate` workflow runs whenever a PR is opened or updated. It checks out the target head SHA, installs dependencies, builds the package, and runs `npm pack`. + +```yaml +on: + pull_request: + branches: [main] + types: [opened, synchronize, reopened, ready_for_review] + +permissions: + contents: read + pull-requests: read + +jobs: + pack: + steps: + - uses: actions/checkout@v6 + with: + repository: ${{ github.event.pull_request.head.repo.full_name }} + ref: ${{ github.event.pull_request.head.sha }} + persist-credentials: false + + - run: npm ci + - run: npm run build + - run: npm pack --ignore-scripts --pack-destination "$RUNNER_TEMP/npm-candidate" + + - uses: actions/upload-artifact@v7 + with: + name: npm-candidate-${{ github.event.pull_request.head.sha }} + path: ${{ runner.temp }}/npm-candidate/*.tgz +``` + +This workflow has neither `id-token: write` nor an npm token. Even if a malicious PR changes a build script, it cannot publish anything to npm from here. + +Before the build, the version is rewritten to `-beta.pr.sha`. The artifact name also includes the full SHA, making it possible to verify later which workflow run built it and from which commit. + +### 2. First, respond to `/beta` with an 👀 reaction + +I considered using GitHub approval as the publication condition. However, if another commit is added after approval, the reviewed code and the code sent to npm are no longer the same. + +Instead, `/beta` authorizes only the head SHA that was current when the comment was posted, not the PR as a whole. No matter who adds the change, a new commit invalidates the previous authorization. The new SHA must pass CI, and an owner or maintainer must comment `/beta` again. + +The workflow receives `/beta` through GitHub's `issue_comment` event because regular comments on a PR are also handled by the Issue Comments API. + +After confirming that the comment body is an exact match and that the target is a PR, the workflow uses `actions/github-script` to check the commenter's permission. If it is `admin` or `maintain`, it attempts to add the 👀 reaction with [`peter-evans/create-or-update-comment`](https://github.com/peter-evans/create-or-update-comment). + +```yaml +on: + issue_comment: + types: [created] + +jobs: + beta-reaction: + if: >- + github.event.issue.pull_request && + github.event.comment.body == '/beta' + permissions: + pull-requests: write + + steps: + - name: Check owner or maintainer permission + id: permission + uses: actions/github-script@v8 + with: + script: | + const { data } = await github.rest.repos.getCollaboratorPermissionLevel({ + owner: context.repo.owner, + repo: context.repo.repo, + username: context.payload.comment.user.login, + }); + return { authorized: ['admin', 'maintain'].includes(data.permission) }; + + - name: Add reaction to authorized beta comment + if: fromJson(steps.permission.outputs.result).authorized + uses: peter-evans/create-or-update-comment@v5 + continue-on-error: true + with: + comment-id: ${{ github.event.comment.id }} + reactions: eyes +``` + +The 👀 means “command received”; it does not indicate a successful publication. The workflow then verifies that the PR is open and not a draft, along with the required CI, the PR head SHA, the commenter's permissions, and the candidate artifact. It checks the head SHA and permissions again immediately before publication. If CI is still running, it does not publish and instead attempts to post a comment explaining which checks remain. The maintainer must run `/beta` again after CI succeeds. + +### 3. Publish through OIDC without storing an npm token + +Publication uses npm Trusted Publishing. + +On npm, I register the GitHub repository and workflow file as a Trusted Publisher for the package. On the GitHub Actions side, only the publishing job receives `id-token: write`. + +```yaml +publish-candidate: + needs: authorize-candidate + if: needs.authorize-candidate.outputs.allowed == 'true' + permissions: + actions: read + contents: read + id-token: write + pull-requests: read + + steps: + - uses: actions/setup-node@v6 + with: + node-version: 24 + registry-url: https://registry.npmjs.org + + - uses: actions/download-artifact@v8 + with: + name: ${{ needs.authorize-candidate.outputs.artifact }} + run-id: ${{ needs.authorize-candidate.outputs.run-id }} + + - run: | + npm publish "$PACKAGE_ARCHIVE" \ + --ignore-scripts \ + --provenance \ + --access public \ + --tag beta +``` + +GitHub Actions issues a short-lived OIDC ID token, and npm verifies that claims such as the repository and workflow match the registered publisher. There is no need to store a long-lived `NPM_TOKEN` in GitHub Secrets. + +OIDC does not, however, make the artifact's contents safe. OIDC controls _who_ can publish. The artifact, SHA, and package identity checks must separately guarantee _what_ gets published. + +The permission boundary would collapse if a job with `id-token: write` checked out and ran PR code. The publishing job therefore never runs PR code; it only downloads the `.tgz` created by the read-only workflow. It also compares the package name, version, registry, SHA, and the workflow run that created the artifact, publishing only when every value matches. If the PR changes the workflow used to make the release decision, publication from that PR is blocked as well. + +The workflow also uses `--provenance`, so the npm package records which source repository and GitHub Actions workflow built and published it. + +### 4. Comment with the install command only after publication succeeds + +The 👀 reaction and the publication-complete comment serve different purposes. + +The completion-comment job declares both authorization and publication as dependencies, and runs only if `publish-candidate` succeeds. + +```yaml +comment-candidate: + needs: [authorize-candidate, publish-candidate] + if: >- + always() && + needs.authorize-candidate.outputs.allowed == 'true' && + needs.publish-candidate.result == 'success' + permissions: + issues: write + pull-requests: write + + steps: + - name: Comment exact install command + uses: actions/github-script@v8 +``` + +`actions/github-script` builds the command from the package name and immutable version. + +````markdown +### npm beta published + +CI passed for commit `8d96f41c7cb3`. Install the immutable version with: + +```sh +npm install @rdlabo/ionic-theme-ios26@9.0.2-beta.pr131.sha8d96f41c7cb3 +``` +```` + +The comment body also contains an HTML marker with the SHA. If the same workflow is rerun, it updates the existing comment for that SHA instead of adding another one. + +The `beta` dist-tag moves whenever the next candidate is published. The PR therefore tells users to install an immutable version containing the PR number and SHA, not `@beta`. Because the npm registry does not allow the contents of an existing version to be overwritten, the exact package that was tested remains reproducible later. + +I kept the comment step separate from the publishing job. If npm publication succeeds but the GitHub API temporarily fails, the release itself should not be rolled back into a failed state. The comment is best effort, and the install command also remains in the Job Summary. + +Automatic publication on merge joins the same publishing job. A push to `main` alone does not satisfy the trigger. After receiving `workflow_run`, the workflow verifies that the SHA is an actual PR merge commit and the current `main` HEAD, and that both the repository-specific required CI and Package Candidate have succeeded for the same SHA. Only then does it publish to `beta`. If the same version already exists on npm, it is not published again. + +## Rolling it out to npm libraries listed on docs.rdlabo.dev + +I first tested this mechanism with the iOS 26 and Material Design 3 themes for Ionic. I then expanded it to the npm libraries listed on [docs.rdlabo.dev](https://docs.rdlabo.dev/). The design handles not only single-package repositories, but also repositories that publish multiple packages together, such as the Capacitor Stripe plugin. + +The repository-specific configuration varies: a single package, npm workspaces, custom build steps, or multiple required CI workflows. The publication boundaries nevertheless stay the same. + +- Only full releases use `latest` +- Revisions and prereleases use `next` +- Only PR candidates use `beta` +- Candidate builds come from the exact SHA that passed CI +- If the commit changes after authorization, authorization must be granted again +- Publication permissions never exist where PR code runs + +### Make the README an operating contract, not just the workflow + +If someone must read GitHub Actions YAML to learn the publication requirements, the OSS project has not documented its operation well enough. I therefore added a `Prerelease channels` section to each repository's README. + +The README explains that only an exact `/beta` command from an owner or maintainer is accepted, required CI must pass, and a new commit requires another `/beta`. The same place documents automatic publication after a PR merge, the fact that a direct push to `main` does not publish a candidate, and the roles of `latest`, `next`, and `beta`. + +A PR that changes a workflow used for the release decision cannot publish `beta` from itself. The workflow change must first be reviewed and merged into `main`; it becomes available to the next PR. This condition is documented in the README too, so the reason `/beta` did not work is not confined to an Actions log. + +The implementation enforces the boundary by stopping invalid publications. The README lets maintainers and contributors operate from the same shared assumptions. The code and documentation are maintained together as one release specification. + +## Test with real users before the full release + +A pull request is not only a place to merge code. It can also be a place to hand a change to users and let them test it in their real projects. + +For an OSS library in particular, it is difficult for maintainers to reproduce every usage pattern in their own test environments. With per-PR npm packages, we can ask users to “install this one line and try it” instead of merely asking them to read the diff. + +This makes it easier to get feedback before the full release. Small fixes that cannot wait for the next release can reach the users who need them first. All the while, the `latest` tag used by regular users remains untouched. + +I have brought this quick path for testing PR changes into my own OSS projects. Going forward, I can validate not only the code review, but also the result of putting the change into a real app before the release. + +See you next time. diff --git a/projects/web-site/src/articles/playwright-visual-regression-ci-baseline.md b/projects/web-site/src/articles/playwright-visual-regression-ci-baseline.md new file mode 100644 index 0000000..5ce3b46 --- /dev/null +++ b/projects/web-site/src/articles/playwright-visual-regression-ci-baseline.md @@ -0,0 +1,242 @@ +--- +title: "Visual Regression Testing for OSS: Mac-Independent Baselines Updated Only in CI" +description: "How an Ionic theme project uses Playwright in a fixed Linux container, maintainer-authorized screenshot updates, and separate read/write jobs to keep visual baselines reproducible and reviewable." +zennSlug: playwright-visual-regression-ci-baseline +sourceVerification: pending +sourceUrl: https://zenn.dev/rdlabo/articles/playwright-visual-regression-ci-baseline +publishedAt: "2026-08-30T12:52:15+09:00" +publishedDate: "2026-08-30" +emoji: "📸" +--- + +When operating visual regression tests in an OSS project, taking the screenshots was not the difficult part. The hard question was, “Whose environment produces the correct baseline?” + +For an Ionic theme I maintain, baseline images updated on a Mac did not match screenshots captured on Linux in GitHub Actions exactly. Even with the same code and Chromium version, small differences appeared in areas such as text edges. + +Instead of increasing the allowed difference, I decided that every baseline committed to the repository would be generated in one shared Linux container. + +## Detecting an intentional change as a visual diff + +A real example is [PR #131](https://github.com/rdlabo-dev/ionic-theme-ios26/pull/131) for the iOS 26 theme. + +This PR added elements to the demo that correctly display a Radio Group spanning multiple Inset Lists. The code change was intentional, but those elements were absent from the saved baseline, so the visual regression test correctly failed. + +The previous Expected image is on the left, and the PR's Actual image is on the right. + +| Expected | Actual | +| --- | --- | +| ![Baseline before adding the Radio Group](/images/playwright-visual-regression-ci-baseline/radio-expected.png) | ![Actual screenshot from the PR with the Radio Group added](/images/playwright-visual-regression-ci-baseline/radio-actual.png) | + +In the Diff generated by Playwright, the added area appears in red. + +![Difference between Expected and Actual, with the added Radio Group highlighted in red](/images/playwright-visual-regression-ci-baseline/radio-diff.png) + +Nothing is broken here; the diff is the result of intentionally changing the demo and theme. A visual regression test shows the difference, but the people reviewing the PR decide whether to adopt it as the new baseline. + +When a maintainer comments `/update-screenshots` on the PR, CI regenerates the images on Linux and adds them to that PR in a bot commit. Tests run again on that commit and verify that both Ionic 8 and Ionic 9 match the same new baseline. + +This PR initially detected visual differences in both Ionic 8 and 9. After the `/update-screenshots` update, both test suites passed. + +## Start with Playwright's `toHaveScreenshot` + +The image comparison itself uses Playwright's [`toHaveScreenshot`](https://playwright.dev/docs/test-snapshots). + +```ts +await page.goto(route.path, { waitUntil: 'networkidle' }); +await prepareScreenShot(page, route.name); + +await expect(page).toHaveScreenshot(`${route.name}.png`, { + fullPage: true, + animations: 'disabled', + mask: [page.locator('ion-spinner')], +}); +``` + +The tests cover more than Buttons and Inputs. All demo routes and states—including Alerts, Action Sheets, Modals, Popovers, Toasts, and Tabs—are defined in an array. Each route is captured in both light and dark mode, producing more than 90 baseline images today. + +Before capturing dark mode, the test enables Ionic's class-based palette. + +```ts +await page.evaluate(() => { + document.documentElement.classList.add('ion-palette-dark'); +}); + +await expect(page).toHaveScreenshot(`${route.name}-dark.png`, { + fullPage: true, + animations: 'disabled', + mask: [page.locator('ion-spinner')], +}); +``` + +Visual regression tests need to minimize anything that changes from one capture to the next. These tests keep the following conditions consistent. + +- Use Chromium only +- Capture full-page screenshots +- Disable Ionic and Playwright animations +- Wait for `networkidle` and visible `ion-content` +- Mask spinners that keep moving +- Match the viewport to the content height +- Use the same Playwright container image in CI + +The test code can be short; creating a stable screen is the more important part. If every capture includes changing dates, random values, network responses, or animations, the image comparison quickly stops being trustworthy. + +## Do not turn OS differences into a debate over whose image is correct + +Playwright's official documentation also states that browser rendering varies with the OS, hardware, headless mode, and other factors, and recommends running tests in the same environment that generated the baselines. See [Visual comparisons](https://playwright.dev/docs/test-snapshots). + +Running `--update-snapshots` on a local Mac generates baselines with Chromium on macOS. Taking those images to GitHub Actions means comparing them with Chromium on Linux. + +Even when they look identical, font rendering, antialiasing, pixel rounding, and other details do not match perfectly. If fine-grained diffs appear even in unchanged regions, every PR must determine whether each difference is a real regression or merely an OS difference. + +Playwright provides `maxDiffPixels`, `maxDiffPixelRatio`, and `threshold`. + +```ts +await expect(page).toHaveScreenshot('button.png', { + maxDiffPixelRatio: 0.01, +}); +``` + +This could allow a test to pass when, for example, up to 1% of all pixels differ. A pixel count, however, cannot distinguish a small broken area at the edge of an image from a small OS-induced difference. + +What I need to protect in this CSS theme includes spacing of just a few pixels, borders, and text positions. I therefore left `maxDiffPixels` and `maxDiffPixelRatio` unset instead of widening the tolerance merely to make the tests pass. + +This choice is not about being strict for its own sake; it gives an OSS project a shared standard for making decisions. Instead of folding differences that occur only on one person's Mac into the allowed tolerance, comparing every PR in the same Linux environment lets contributors and maintainers inspect the same Diff. + +I standardized the comparison environment instead. + +```yaml +jobs: + test: + runs-on: ubuntu-latest + container: + image: mcr.microsoft.com/playwright:v1.58.2-noble +``` + +Both PR tests and baseline updates run in the same Playwright Linux container. Pinning the container tag also limits the impact of browser updates on the runner. + +## Manage permission to change baselines within the PR + +`playwright test --update-snapshots` can be run locally, but contributors no longer commit images generated on their own computers as the baseline for this OSS project. + +When the baseline needs to change, a maintainer posts the following comment on a PR in the same repository. + +```text +/update-screenshots +``` + +GitHub Actions receives the `issue_comment` event and checks the commenter's repository permission. If they have `admin`, `maintain`, or `write`, the workflow adds an 👀 reaction to the comment. Updating the baseline requires authorization just as pushing code does. + +```yaml +on: + issue_comment: + types: [created] + +jobs: + pr-metadata: + if: >- + github.event.issue.pull_request && + contains(github.event.comment.body, '/update-screenshots') +``` + +The actual update runs in a read-only job inside the pinned Playwright container. + +1. Check out the PR head SHA that was current when the command was posted +2. Run `--update-snapshots` with Ionic 9 +3. Switch to Ionic 8 and run the normal tests against the same images +4. Save the updated images as an artifact + +I do not maintain separate baselines for Ionic 8 and 9. Ionic 9 updates the baseline, and the tests verify that Ionic 8 produces the same appearance. + +```yaml +- name: Update screenshots with Ionic 9 + run: npm run test:e2e:update + env: + IONIC_MAJOR: 9 + +- name: Verify Ionic 8 against the same screenshots + run: npm run test:e2e + env: + IONIC_MAJOR: 8 +``` + +Keeping separate images for the two versions would make both test suites easier to pass. The guarantee I want from this theme, however, is that “the same theme version looks the same on Ionic 8 and Ionic 9.” Sharing one set of baselines makes CI enforce that compatibility. + +## Keep externally supplied code and write permission out of the same job + +A new commit might be added to the PR while the screenshots are being generated. Images created from an old SHA must not be committed to a new head. + +I therefore separated the job that generates the images from the job that commits them. The job that checks out and builds PR code has no write access to the repository. + +```text +read-only job + PRのSHAをcheckout + Linuxで画像を更新 + artifactへ保存 + │ + ▼ +write job + PRの現在のhead SHAを再取得 + コメント時点のSHAと一致するか確認 + GitHub APIでbot commit +``` + +Before committing, the workflow confirms that the PR's repository, branch, and head SHA are unchanged from when image generation began. If any value has changed, it fails and requires another `/update-screenshots` command. + +It also validates filenames so the artifact cannot introduce unexpected files. The commit uses GitHub's Git Data API and updates the branch with `force: false`. + +The workflow never uses permissions from the upstream repository to commit automatically to a fork PR. Automatic image updates work only for branches in the same repository. For a fork, the contributor must update images using the same Playwright Linux container, or a maintainer must bring the change onto a branch in the upstream repository. This limitation is somewhat inconvenient, but I prioritized avoiding a path that writes to a third party's branch with upstream permissions. + +After a successful update, the workflow posts the following comment on the PR. + +```text +✅ Screenshots have been updated successfully! + +The new screenshots have been committed to this PR. +``` + +The PR now contains the entire process: inspect the difference, update the baseline, and revalidate the result with Ionic 8 and 9. + +## Let contributors and maintainers inspect the same Diff on the PR + +When a visual regression test fails, a long Actions log does not tell a contributor what changed. Review is not possible if only a maintainer's local environment can display the Diff. + +On failure, the workflow therefore saves Playwright's Expected, Actual, Diff, and trace files as artifacts. A summary of the Ionic 9 JSON report is posted as a PR comment, and the HTML report is published to a per-PR GitHub Pages directory. That report directory is removed after the PR is closed. + +CI uses a matrix for Ionic 8 and 9. + +```yaml +strategy: + fail-fast: false + matrix: + include: + - ionic-major: 8 + ionic-version: 8.8.19 + - ionic-major: 9 + ionic-version: 9.0.0 +``` + +With `fail-fast: false`, one failure does not prevent the other result from being recorded. The PR can show whether the issue affects only Ionic 8, only Ionic 9, or whether both versions correctly detected the difference. + +## OSS baselines are decided by review, not implementation + +`/update-screenshots` is useful, but it is not a command to run automatically whenever a test fails. + +A visual regression test tells us only one fact: the result differs from before. Contributors and maintainers must inspect Expected, Actual, and Diff to decide whether that difference is a fix or a regression. + +The correct order is: + +1. The visual regression test detects a difference +2. A person inspects Actual and Diff +3. If the change is intentional, run `/update-screenshots` +4. CI updates the baseline on Linux +5. Ionic 8 and 9 are revalidated against the same images + +Increasing the tolerance makes CI quieter. Quiet CI and correct visuals are not the same thing. + +An OSS visual regression test must share not only its images, but also the process used to decide what is correct. + +The differences between Mac and Linux were not a reason to make comparisons more permissive; they were a reason to choose one place where baselines are created. Update baselines only in CI. Share the Diff on the PR and have people review it. Keep externally supplied code separate from permission to write to the repository. + +With these boundaries in place, no particular maintainer's computer defines correctness. Whoever opens a PR can inspect the same images and protect differences measured in just a few pixels. + +See you next time.