Skip to content

fix(migration): don't apply wildcard auth files on the old nginx-proxy template - #58

Open
mrrobot47 wants to merge 3 commits into
EasyEngine:developfrom
mrrobot47:fix/auth-migration-old-proxy-template
Open

mrrobot47 wants to merge 3 commits into
EasyEngine:developfrom
mrrobot47:fix/auth-migration-old-proxy-template

Conversation

@mrrobot47

@mrrobot47 mrrobot47 commented Sep 25, 2026 •

Copy link
Copy Markdown
Member

Problem

The site auth files migration (20260924120000_auth-command_regenerate_site_auth_files.php, new in v1.3.0) runs before core's image migration and writes the new _wildcard.X and alias files straight into the proxy's mounts. On an upgrade from 4.12.0 the running proxy still has the old template, which applies htpasswd/_wildcard.X to every subdomain of X that has no file of its own. Its docker-gen re-renders that template on any container event on the host, e.g. the cron container restarting or any site restart, with or without a reload. So a sibling site such as shop.<multisite> asks for the multisite's password until the proxy is recreated with the new template, and indefinitely if the image pull fails, because down() was empty and the migration stays recorded.

Fix

  • Backup: up() first backs up htpasswd/ and vhost.d/*_acl to /opt/easyengine/.backup/auth-migration-<timestamp>/ (mode 0700, since the files hold password hashes; copies keep mode and mtime). If the backup fails, the migration fails before anything is written. The backup is kept after a successful upgrade, since a manual downgrade to 4.12.0 needs it.
  • Template check: up() checks whether the RUNNING proxy has the new template (grep -c 'define "acl"' /app/nginx.tmpl in the proxy; that block only exists in the template that applies _wildcard.X to *.X hosts only). A proxy that isn't running counts as old.
  • New template: everything is written and the proxy reloaded, as before.
  • Old template or proxy down:
    • The exact-host htpasswd and ACL files (site, alias domains) are written live; the old template handles them correctly.
    • The _wildcard.* htpasswd and _wildcard.*_acl files are staged in EE_BACKUP_DIR/auth-wildcard-staging/{htpasswd,vhost.d} (/opt/easyengine/.backup/auth-wildcard-staging/, mode 0700), outside the proxy's mounts. A live file of a staged name is removed.
    • The proxy isn't reloaded.
    • The old template never sees a _wildcard.* file.
  • Promotion: once the running proxy has the new template, the staged files are moved into place (temp file + rename), the proxy is reloaded, and the staging dir is removed.
    • It runs on core's after_docker_image_migration hook (fix(migration): revert container migrations when a later upgrade step fails easyengine#1936), right after the image migration.
    • It also self-heals on the next ee run (find_command_to_run_pre, once per process, only when the staging dir exists and no migration is pending), e.g. after ee was killed mid-upgrade.
    • A site whose live file no longer matches its staged copy has its files regenerated from the DB instead. That happens when auth changed in between, e.g. through the older ee after an interrupted upgrade, so a revoked password never reaches the subdomains. Staged files of deleted sites are dropped.
    • Errors only warn: the files stay staged and promotion is retried later.
  • Restore: down() discards the staging dir and makes htpasswd/ and the *_acl files of vhost.d/ identical to the backup (added files removed, restored files renamed into place). It reloads the proxy if it's running and removes the backup. Core calls it only when the image migration (or an earlier step) fails, not after the image migration has completed.
  • Skipped runs (fresh install, no sites) take no backup and stage nothing.

Known window: until promotion, subdomains of subdomain multisites and *.X alias domains are unprotected, which is the 4.12.0 behaviour. Promotion happens at the end of the image migration, about 8 s per WP/PHP site after the proxy is recreated (55 s with 6 sites). This fails closed and never leaks. Exact-host names stay protected throughout.

Tests

  • Isolated harness (fake docker emulating the old template, the new one or a stopped proxy; PHP 7.4 and 8.5, 65/65). It covers:

    • backup and exact restore;
    • the old template stages the subdomain and *.X alias files, with nothing wildcard live, the exact-host files live and the staging 0700 under the backup dir;
    • down() discards staging;
    • the hook promotes only on the new template, with files identical to the staged ones and one reload;
    • down() after promotion restores exactly;
    • the new template writes everything live and is idempotent;
    • a stopped proxy stages from the existing files;
    • stale auth is regenerated from the DB and a deleted site's files are dropped;
    • self-heal skips while a migration is pending or the old proxy runs, and runs once per process.
  • docker-gen render check with easyengine/nginx-proxy:v4.11.1 and :v4.13.0:

    • With the files staged, the old template never protects the sibling, even after container events. The same file placed live leaks to the sibling after one event.
    • After promotion on the new template, only the *.X host gets the wildcard auth.
  • Live upgrade from 4.12.0, with a phar of develop + fix(migration): revert container migrations when a later upgrade step fails easyengine#1936 + #1937 + this PR built the way CI builds the nightly.

    • Sites: a WP subdomain multisite with site auth and a whitelist, a sibling shop.<multisite> without auth, a site with alias and *.X aliases plus auth and a whitelist, and WP --cache, PHP with a DB, php 8.1, php 7.4 and html sites.
    • A probe curled the public IP every 0.5 s without credentials. Container events were forced every ~5 s during every run, and a poller checked the proxy mounts for _wildcard.* files every 0.2 s.
    Run Result
    Image pull fails (Docker Hub blocked) shop. never 401 (0/88), no _wildcard.* file in the mounts at any time, files byte-identical to before, staging and backup removed, row removed
    Retry, successful upgrade shop. never 401 (0/113); _wildcard.* files appear only after the new proxy runs, then subsites, deeper subsites, alias and *.X aliases answer 401 and accept the credentials; backup kept
    Failure after the image migration no revert: the files stay promoted, subsites and aliases stay 401
    ee killed mid-pull, password changed with the 4.12.0 phar, retry while killed: only staged, shop. never 401 (0/72); the retry regenerates the outdated files from the DB: the new password works on subsites, the old one is refused
    Staging left over on an upgraded host the next ee cli version or ee site list promotes it once
    Nightly re-runs no staging, no reload, nothing changed

    The TESTING-NOTES §1 matrix passed after the upgrade (site auth on subsites and aliases, whitelist, global auth, alias add/remove and validation, self-signed HTTPS, site delete cleanup), and so did create/delete of every site type.

  • Behat (@destructive @migration, features/auth-migration.feature on the test branch, on the earlier head): with the registry blocked, the upgrade fails, the alias/_wildcard files are absent again, the row is removed and the version stays old. After unblocking, they're written and the aliases answer 401. CI: https://github.com/mrrobot47/auth-command/actions/runs/36124341500.

  • php -l on PHP 7.4 and 8.5; no new PHPCS violations.

Release

Needs a patch tag, auth-command v1.3.1, and a core pin bump to it before core v4.13.0 is tagged. Ship it with EasyEngine/easyengine#1936 in v4.13.0.

…y template

The auth files migration runs before the image migration, and reloaded the proxy right after writing the new _wildcard.X files. The old nginx-proxy template applies _wildcard.X to every subdomain of X without its own file, so a sibling site such as shop.<multisite> asked for the multisite's password until the proxy was recreated, and indefinitely when the image pull failed.

The migration now backs up htpasswd/ and vhost.d/*_acl to /opt/easyengine/.backup/auth-migration-<ts>/ and reloads the proxy only when the running one has the new template (its nginx.tmpl defines the acl block); otherwise the image migration's recreated proxy renders the files. down() restores the backup exactly, including removing files it added, and reloads the proxy if it runs. The backup is kept after a successful upgrade for a manual downgrade.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Docker events can still make the old template consume newly written wildcard files before proxy replacement.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1 High severity

Open (1)
What changed in this PR

Prevents wildcard authentication files from being applied prematurely by legacy nginx-proxy templates during upgrades.

Changes:

  • Backs up and restores authentication/ACL files.
  • Detects the running proxy template before reloading.
  • Adds rollback support for failed upgrades.
File Description
migrations/​container/​20260924120000_auth-command_regenerate_site_auth_files.php Adds template detection and reversible auth-file migration.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +55 to +59
if ( $new_template ) {
\EE\Site\Utils\reload_global_nginx_proxy();
} else {
// The old template applies `_wildcard.X` to every subdomain of X; the image migration recreates the proxy with the new one.
EE::debug( 'Not reloading nginx-proxy: it runs the old template, the image migration recreates it.' );
Skipping the reload didn't close the leak: the old nginx-proxy's docker-gen re-renders its template on any container event on the host, e.g. the cron container restarting, and that template applies _wildcard.X to every subdomain of X without its own file. Once the migration had written _wildcard.* files into the proxy's mounts, a sibling site such as shop.<multisite> asked for the multisite's password until the proxy was recreated, until down() restored the files after a failed pull, or until the retry when ee was killed mid-pull.

When the running proxy has the old template, or isn't running, the migration now writes only the exact-host htpasswd and ACL files, which the old template handles correctly, and stages the _wildcard.* ones in /opt/easyengine/.staging/auth-wildcard/ (outside the mounts, mode 0700). They are moved into place (temp file + rename) and the proxy is reloaded by the after_docker_image_migration hook once the new proxy runs, or by the first ee run after the migration completed (find_command_to_run_pre), so a killed or failed upgrade heals on the retry. A site whose auth changed since staging, e.g. through the older ee after an interrupted upgrade, gets its files regenerated instead, and files of deleted sites are dropped. down() discards the staging dir. With the new template, nothing changes: everything is written and the proxy reloaded.

Until promotion, subdomains stay unprotected as in 4.12.0, and the old template never sees a _wildcard.* file.
The staged _wildcard.* files now live in EE_BACKUP_DIR/auth-wildcard-staging/{htpasswd,vhost.d} (/opt/easyengine/.backup/...) instead of a new /opt/easyengine/.staging dir. It's still outside the proxy's mounts and 0700. Site names always contain a dot, so no per-site .backup/<site>/ dir can clash with it. The migration, the promotion, the self-heal and down() all use get_wildcard_staging_dir().
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants