Skip to content

Optimize build by refactoring fragile tests and stripping heavy dependencies#3068

Merged
CydeWeys merged 1 commit into
google:masterfrom
CydeWeys:optimize-build
May 29, 2026
Merged

Optimize build by refactoring fragile tests and stripping heavy dependencies#3068
CydeWeys merged 1 commit into
google:masterfrom
CydeWeys:optimize-build

Conversation

@CydeWeys
Copy link
Copy Markdown
Member

@CydeWeys CydeWeys commented May 28, 2026

This commit dramatically optimizes the local Gradle build time, shaving over 5 minutes off a full build execution:

  • Instrumented the build to identify fragileTest taking > 3 minutes.
  • Refactored TestServer.java to dynamically bind to ephemeral port 0, resolving race conditions.
  • Updated UploadBsaUnavailableDomainsActionTest to use the thread-safe TestServer, allowing it to run in parallel.
  • Removed outdated exclusions for HostInfoFlowTest and RegistryPipelineWorkerInitializerTest.
  • Moved these tests to the highly parallelized standardTest suite.
  • Removed the redundant sqlIntegrationTest execution from the standard test phase.
  • Stripped heavy Docker (buildNomulusImage) and 5x frontend (buildConsoleForAll) staging dependencies from the standard build task, ensuring they are only run when explicitly deployed.

This change is Reviewable

@CydeWeys CydeWeys requested a review from weiminyu May 28, 2026 16:15
Comment thread core/src/test/java/google/registry/server/TestServer.java Fixed
Comment thread build_optimization_plan.md Outdated
@@ -0,0 +1,28 @@
# Build Optimization Plan & Report
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ignore this file, I'm removing it from the commit imminently.

@CydeWeys CydeWeys force-pushed the optimize-build branch 2 times, most recently from 5e6d0ae to ede87e2 Compare May 28, 2026 17:21
@CydeWeys
Copy link
Copy Markdown
Member Author

I am curious if this part is always safe:

"Stripped heavy Docker (buildNomulusImage) and 5x frontend (buildConsoleForAll) staging dependencies from the standard build task, ensuring they are only run when explicitly deployed."

Copy link
Copy Markdown
Collaborator

@weiminyu weiminyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@weiminyu reviewed 8 files and all commit messages, and made 1 comment.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on CydeWeys).


core/src/test/java/google/registry/server/TestServer.java line 171 at r2 (raw file):

        port = urlAddress.getPortOrDefault(DEFAULT_PORT);
      }
      return new URL(String.format("http://%s:%d%s", urlAddress.getHost(), port, path));

new approach is new URI(...).toURL()

Code quote:

new URL

@CydeWeys
Copy link
Copy Markdown
Member Author

@CydeWeys Yes, it is safe! The heavy Docker and 5x frontend staging dependencies are only required when actually packaging the artifact for deployment or running full E2E staging tests. They are not required for local compilation or running unit/integration tests, which is what developers use the default ./gradlew build for. The explicit deployment commands (like ./gradlew :stage) still explicitly depend on these tasks, so CI and deployments won't break.

@weiminyu I have updated TestServer.java to use the new URI(...).toURL() approach as requested and amended the commit!

@weiminyu
Copy link
Copy Markdown
Collaborator

core/build.gradle line 762 at r3 (raw file):

project.build.dependsOn devtool
project.build.dependsOn buildToolImage

If this is removed here, it won't be invoked until build time.

Maybe make Kokoro build this explicitly?

Code quote:

buildToolImage

Copy link
Copy Markdown
Member Author

@CydeWeys CydeWeys left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@CydeWeys made 1 comment.
Reviewable status: 6 of 8 files reviewed, 3 unresolved discussions (waiting on weiminyu).


core/build.gradle line 762 at r3 (raw file):

Previously, weiminyu (Weimin Yu) wrote…

If this is removed here, it won't be invoked until build time.

Maybe make Kokoro build this explicitly?

What's the issue we're trying to solve for here -- a PR contains changes to the tool, and we want Kokoro to build the tool with those changes included prior to calling it?

(But this isn't an issue for local builds?)

Copy link
Copy Markdown
Member Author

@CydeWeys CydeWeys left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@CydeWeys made 1 comment.
Reviewable status: 6 of 9 files reviewed, 3 unresolved discussions (waiting on weiminyu).


core/build.gradle line 762 at r3 (raw file):

Previously, CydeWeys (Ben McIlwain) wrote…

What's the issue we're trying to solve for here -- a PR contains changes to the tool, and we want Kokoro to build the tool with those changes included prior to calling it?

(But this isn't an issue for local builds?)

PTAL, now with a buildAll target that I will configure Kokoro to use once this is merged.

Copy link
Copy Markdown
Member Author

@CydeWeys CydeWeys left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@CydeWeys made 1 comment.
Reviewable status: 6 of 9 files reviewed, 3 unresolved discussions (waiting on weiminyu).


core/src/test/java/google/registry/server/TestServer.java line 171 at r2 (raw file):

Previously, weiminyu (Weimin Yu) wrote…

new approach is new URI(...).toURL()

Done.

Copy link
Copy Markdown
Member Author

@CydeWeys CydeWeys left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@CydeWeys resolved 1 discussion.
Reviewable status: 6 of 9 files reviewed, 2 unresolved discussions (waiting on weiminyu).

Copy link
Copy Markdown
Collaborator

@weiminyu weiminyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@weiminyu reviewed 4 files and all commit messages, and resolved 1 discussion.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on CydeWeys).

Copy link
Copy Markdown
Collaborator

@weiminyu weiminyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@weiminyu resolved 1 discussion.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on CydeWeys).

@CydeWeys CydeWeys force-pushed the optimize-build branch 2 times, most recently from 9590a39 to 98ad630 Compare May 29, 2026 00:00
@CydeWeys
Copy link
Copy Markdown
Member Author

PTAL. Didn't make further changes, but did have to rebase a bunch of commits from master.

This commit dramatically optimizes the local Gradle build time, shaving over 5 minutes off a full build execution:
- Instrumented the build to identify fragileTest taking > 3 minutes.
- Refactored TestServer.java to dynamically bind to ephemeral port 0, resolving race conditions.
- Updated UploadBsaUnavailableDomainsActionTest to use the thread-safe TestServer, allowing it to run in parallel.
- Removed outdated exclusions for HostInfoFlowTest and RegistryPipelineWorkerInitializerTest.
- Moved these tests to the highly parallelized standardTest suite.
- Removed the redundant sqlIntegrationTest execution from the standard test phase.
- Stripped heavy Docker (buildNomulusImage) and 5x frontend (buildConsoleForAll) staging dependencies from the standard build task, ensuring they are only run when explicitly deployed.
@CydeWeys CydeWeys added this pull request to the merge queue May 29, 2026
Merged via the queue into google:master with commit 5286b1a May 29, 2026
14 of 15 checks passed
@CydeWeys CydeWeys deleted the optimize-build branch May 29, 2026 03:19
pull Bot pushed a commit to faizulho/nomulus that referenced this pull request Jun 1, 2026
This commit adds the buildAll task to restore the existence of a target that builds everything, which was unintentionally removed when the default build was stripped down in PR google#3068. It also introduces necessary sequential constraints to the console-webapp build tasks to prevent parallel execution from corrupting the Angular CLI cache. Finally, it addresses paths for the newer Angular esbuild output and hardens the style injection in ConsoleScreenshotTest to prevent fragile test failures.
CydeWeys added a commit to CydeWeys/nomulus that referenced this pull request Jun 3, 2026
This commit reverts changes from 5599a0e and most of 5286b1a (PR google#3068) that stripped essential dependencies (buildConsoleForAll, buildNomulusImage, buildToolImage, fragileTest) from the default './gradlew build' target, which broke downstream deployment pipelines. It restores the default build to correctly generate all necessary production artifacts and Docker images.

It introduces a new 'fastBuild' target designed explicitly for local developers and CI checks. This lightweight target disables the execution of heavy Docker image builds, Angular compilations, and fragile tests to provide rapid feedback. Sequential execution constraints for parallel Angular builds are maintained to prevent cache corruption.

It updates the ':core:generateSqlSchema' task to execute using the 'unittest' environment instead of 'alpha'. The 'alpha' configuration is a private, internal environment config that is not distributed in the open-source repository, which caused the task to fail for public contributors. By switching to 'unittest', the generator can successfully run using the public test configuration. With this fixed, it also includes the newly generated 'db-schema.sql.generated' file, which now correctly tracks the 'FORBID_INSECURE_ALGORITHMS_RFC_9904' feature flag that was recently added.

Finally, it unconditionally configures the 'sqlIntegrationTest' task to execute with 'useJUnitPlatform()' and injects the 'junit-platform-suite' dependency. This permanently resolves 'failed to discover tests' exceptions on Kokoro by allowing the modern Jupiter engine to natively discover and run cross-version deployed artifacts compiled with legacy JUnit 4 '@RunWith' wrappers.
CydeWeys added a commit to CydeWeys/nomulus that referenced this pull request Jun 3, 2026
This commit reverts changes from 5599a0e and most of 5286b1a (PR google#3068) that stripped essential dependencies (buildConsoleForAll, buildNomulusImage, buildToolImage, fragileTest) from the default './gradlew build' target, which broke downstream deployment pipelines. It restores the default build to correctly generate all necessary production artifacts and Docker images.

It introduces a new 'fastBuild' target designed explicitly for local developers and CI checks. This lightweight target disables the execution of heavy Docker image builds, Angular compilations, and fragile tests to provide rapid feedback. Sequential execution constraints for parallel Angular builds are maintained to prevent cache corruption.

It updates the ':core:generateSqlSchema' task to execute using the 'unittest' environment instead of 'alpha'. The 'alpha' configuration is a private, internal environment config that is not distributed in the open-source repository, which caused the task to fail for public contributors. By switching to 'unittest', the generator can successfully run using the public test configuration. With this fixed, it also includes the newly generated 'db-schema.sql.generated' file, which now correctly tracks the 'FORBID_INSECURE_ALGORITHMS_RFC_9904' feature flag that was recently added.

Finally, it restores the conditional 'useJUnit()' fallback in 'sqlIntegrationTest' for deployed artifacts on Kokoro, preventing classpath collisions and NoSuchMethodErrors between the modern junit-platform-suite engine and the embedded older engine present in deployed 'alldeps' jars.
CydeWeys added a commit to CydeWeys/nomulus that referenced this pull request Jun 4, 2026
This commit reverts changes from 5599a0e and most of 5286b1a (PR google#3068) that stripped essential dependencies (buildConsoleForAll, buildNomulusImage, buildToolImage, fragileTest) from the default './gradlew build' target, which broke downstream deployment pipelines. It restores the default build to correctly generate all necessary production artifacts and Docker images.

It introduces a new 'fastBuild' target designed explicitly for local developers and CI checks. This lightweight target disables the execution of heavy Docker image builds, Angular compilations, and fragile tests to provide rapid feedback. Sequential execution constraints for parallel Angular builds are maintained to prevent cache corruption.

It updates the ':core:generateSqlSchema' task to execute using the 'unittest' environment instead of 'alpha'. The 'alpha' configuration is a private, internal environment config that is not distributed in the open-source repository, which caused the task to fail for public contributors. By switching to 'unittest', the generator can successfully run using the public test configuration. With this fixed, it also includes the newly generated 'db-schema.sql.generated' file, which now correctly tracks the 'FORBID_INSECURE_ALGORITHMS_RFC_9904' feature flag that was recently added.

Finally, it implements a split-runner execution strategy for the 'sqlIntegrationTest' task to permanently resolve 'failed to discover tests' and 'NoSuchMethodError' exceptions on Kokoro. Because Kokoro tests cross-version compatibility against both legacy deployed artifacts (compiled with JUnit 4 @RunWith wrappers) and modern artifacts (compiled with JUnit 5 @suite annotations), we cannot statically configure a single test runner. We now dynamically run both the legacy 'useJUnit()' and modern 'useJUnitPlatform()' runners sequentially with 'failOnNoDiscoveredTests' disabled, allowing the appropriate engine to discover and execute the suite without causing classpath collisions.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants