tests: an UpgradeFromPreviousReleaseIT boots the build on a deployment the previous release wrote (#7641)

[NicoleNG18]

Cause

Every IT in tests/tests-integrations boots on an empty database, so nothing exercised an upgrade. #7008, #7370, #7435, #7597 and #7049 were all failures that only an existing database shows, and each was found on a user's instance, not in CI.

Change

UpgradeFromPreviousReleaseIT (tests-integrations, @Tag("upgrade")) has two phases.

1. The previous release writes a deployment. The release named in tests/UPGRADE_FROM (14.74.0) runs as its published dirigiblelabs/dirigible image on a Testcontainers PostgreSQL. The test drives it over HTTP, the way the IDE does:

   • an intent project (UpgradeFromPreviousReleaseIT/app.intent) is generated and published;
   • it takes rows in every entity (Customer, Ticket, TicketDigest);
   • it parks one TicketReview process instance per ticket on a user task;
   • a 5-second schedule fires and generates the digests.

   The release is then stopped like an orchestrator stops it (SIGTERM plus a 60 s grace period), and its repository folder is copied out of the container. It isn't bind-mounted, because the image runs as root and the test JVM couldn't write the files on a Linux runner.

2. The build under test boots in-process on that database and folder, with the readiness bridge and DIRIGIBLE_READINESS_REQUIRE_CLEAN_BOOT=true. It must:

   • accept traffic, with isCleanBoot() true;
   • report zero failed artefacts in the health census;
   • have recorded every changeset of its system changelog that applies to PostgreSQL (the changelog's dbms filters are honoured);
   • keep every row exactly as the previous release returned it, and keep both running instances;
   • fire the job again and start the process again for a ticket created after the upgrade.

The previous release is driven by a new helper in tests-framework, package framework.upgrade:

• PreviousRelease resolves the tag (the system property dirigible.upgrade.from overrides the file);
• PreviousReleaseEnvironment runs the containers, does the graceful stop and copies the repository;
• PreviousReleaseClient makes the HTTP calls.

The existing in-process helpers (ProjectDeployer, BaseIntentTestProject / Selenide) can't target a release in a container, which is why the issue's BaseIntentTestProject isn't used.

CI:

• Exclusions: the api shards in build.yml and nightly.yml (both database legs) and the PR gate exclude upgrade. Dry runs confirm it: upgrade selects 1 class; the api expression (135 classes) and the PR gate (160) don't include the IT.
• The new upgrade-test job runs -Dit.groups=upgrade nightly and in release.yml, where release-docker-images now has needs: upgrade-test, so the release images are pushed only if the upgrade passes. It needs Docker but no database service.
• The tag bump: the release job writes the version it released into tests/UPGRADE_FROM in its "for development" commit. The release commit itself is still tested from N-1, and master from N afterwards.
• ci.md documents the tag and the job.

One thing the test environment needed, and a finding behind it

The two phases reach the same PostgreSQL by two names: the container uses its network alias postgres:5432, the test JVM uses the mapped port. A datasource keeps the URL that was resolved when its file was parsed (DataSourcesSynchronizer → Configuration.configureObject, stored in DIRIGIBLE_DATA_SOURCES), and an unchanged file is never parsed again. So the build under test first tried to connect to postgres and failed. The IT therefore rewrites the recorded URLs between the phases (readdressRecordedDataSources, which fails if it rewrites nothing). A real deployment keeps its database address across an upgrade, so this doesn't weaken the test.

The behaviour behind it is worth a separate issue, and I haven't changed it here: on an existing deployment, a changed DIRIGIBLE_DATASOURCE_DEFAULT_URL doesn't take effect, because the resolved URL persisted by an earlier boot wins.

Verified

• Green locally against 14.74.0: 66 s after the previous-release phase, about 2.5 min in total including that release's boot.
• Red on data loss, as the issue asks. A temporary edit (not committed) renamed Customer's name column in the copied project before the boot, the change a branch renaming that column would ship. The IT then failed with [customers] expected: [{"Id"=1.0, "Name"="Acme"}] but was: [{"Id"=1.0, "Name"=null}]. After the edit was reverted, the file is byte-identical to the green version.
• tests-framework unit tests: 7 green, including 3 new PreviousReleaseTest cases for the UPGRADE_FROM lookup.
• mvn -T 1C formatter:validate with the cache wiped: BUILD SUCCESS. The release javadoc build on tests-framework: BUILD SUCCESS.
• The workflow YAML parses, and the job graph is as described.

Not verified

• The new CI jobs haven't run yet. They only start nightly and on a release. The local runs used Docker Desktop on macOS with Testcontainers' Ryuk disabled, because Ryuk couldn't start against Docker Desktop's socket. CI's Linux runners use the standard socket.
• Upgrade ordering isn't asserted: the IT doesn't check the order in which Liquibase changesets ran, only that they're all recorded. Tenants beyond the default, and Quartz rows other than the generated schedule, aren't covered.

Fixes #7641

🤖 Generated with Claude Code