CI infra: apache-rat-plugin 0.18 check fails intermittently in parallel packaging-check build #20465

This issue was generated automatically by Claude Code (Anthropic's AI coding agent) running a scheduled CI-triage routine on behalf of @FrankChen021. Analysis and suggested fixes are AI-produced; please verify before acting on them.

Status: Fix PR #20527 open
Subject: apache-rat-plugin:0.18:check during .github/scripts/packaging-check.sh (Static Checks CI, packaging-check (25) / packaging-check-jdk25)
Failures: 3 · First seen: 2026-09-06 · Last seen: 2026-09-30

Root cause

apache-rat-plugin 0.18 declares rat:check as threadSafe = true, but it keeps per-run state in JVM-wide singletons. packaging-check.sh runs mvn install -Prat -T1C, so several modules run rat:check at the same time in one JVM and corrupt each other's configuration. #20000 (0.15 → 0.18) brought in this code; 0.16.1 doesn't contain it. There are two separate races:

1. Excludes silently dropped (2026-09-30, druid-cassandra-storage). Since 0.17, the mojo turns its POM config into CLI args and parses them with OptionCollection.parseCommands. Each Arg enum constant owns one static commons-cli OptionGroup, and Arg.getOptions() puts those same instances into every Options. DefaultParser.parse first calls setSelected(null) on every group, and Arg.processInputArgs later reads EXCLUDE.isSelected() / EXCLUDE_STD.isSelected() from those shared groups, not from the module's own CommandLine. If another module's parse runs in between, this module skips all its <excludes> (including **/target/**) and the MAVEN std collection. AbstractRatMojo.getConfiguration captures the excluder at that moment for the DirectoryWalker. The buildDirectory filter that RatCheckMojo adds afterwards never reaches the walker's cached matcher. So at verify, every generated file under target/ gets scanned, which matches the 36 UNAPPROVED files.
2. NPE in SCM-ignore parsing (2026-09-10, druid-consul-extensions). StandardCollection.GIT holds one shared GitIgnoreBuilder. AbstractFileProcessorBuilder.build() mutates and then clear()s its instance TreeMap levelBuilders, so concurrent builds wipe each other's entries and levelBuilders.get(key).asMatcherSet() NPEs. druid-consul-extensions is one of the few modules with its own .gitignore files, at two levels, so it takes that createMatcherSetList path. This is upstream RAT-553, which reports the identical stack.

The 2026-09-06 NPE (File.getParentFile() on a null file, druid-deltalake-extensions) fits race 1 hitting OUTPUT_FILE: Arg.processOutputArgs sees the shared group as selected and calls getParentFile() on a value that is null for this module. The log has no stack trace (no -e), so this one isn't confirmed. Upstream fixed both races in RAT-573 ("Allow parallel Maven builds of RAT": parseCommands synchronized, per-call SCM ignore builders), targeted at 1.0.0, which is unreleased; 0.18 is the latest release. None of the failing commits touched the build, and neighbouring master runs passed this job.

Suggested fix

Pin apache-rat-plugin back to 0.16.1 in the rat profile of the root pom.xml until a release containing RAT-573 ships, then upgrade. Alternatively, keep 0.18 but run the license check single-threaded: drop -Prat from the -T1C install in .github/scripts/packaging-check.sh and add a separate mvn -B -Prat apache-rat:check step without -T. Setting <parseSCMIgnoresAsExcludes>false</parseSCMIgnoresAsExcludes> only avoids race 2, not the dropped excludes.

Occurrences

Failed push-triggered master jobs only. The daily triage routine adds one row per new failed job.