[build] fluss-filesystems uber-jars ship unshaded jackson + guava that can shadow downstream dependencies

[Jackeyzhe]

Search before asking

• I searched in the issues and found nothing similar.

Fluss version

main (development)

Please describe the bug 🐞

While fixing the same class of shading leak in fluss-fs-s3 (#4073, follow-up to
#3884), I checked the rest of the fluss-filesystems family. I rebuilt every fs
plugin from current main and inspected the produced shaded uber-jar. All
seven ship unshaded com.fasterxml (jackson) and com.google (guava) that
will shadow any downstream application's own jackson / guava on the classpath,
causing NoSuchMethodError / NoClassDefFoundError / silent version skew — the
same failure mode reported in #3884.

Measured leak (top-level unshaded entries in the shaded uber-jar)

Plugin	jackson (com/fasterxml)	guava (com/google)	MRJ escaped*	notes
fluss-fs-hdfs	1,073	2,731	25	baseline
fluss-fs-s3	1,093	2,042	25	leaks on main; fix proposed in open PR #4073 (not yet merged)
fluss-fs-oss	1,073	2,230	25
fluss-fs-cos	1,073	2,246	25
fluss-fs-azure	1,073	2,042	25
fluss-fs-obs	1,161	2,042	25
fluss-fs-gs	1,073	7,351	25	worst: ~3.6× the guava of the others, and no relocated copy at all (see below)

* MRJ = META-INF/versions/N/com/fasterxml/jackson/core/io/doubleparser/**
entries that escape relocation — the exact #3884 bug class, present in all 7.

fluss-fs-hadoop-shaded itself is fine: it relocates
com.fasterxml → org.apache.fluss.fs.shaded.hadoop3.com.fasterxml and ships
hadoop's guava as org.apache.hadoop.thirdparty.com.google (already
third-partied by hadoop, not leaked). The leak is in the consumer fs plugins.

Root cause (confirmed from pom inspection)

Each fs plugin's own maven-shade-plugin config only relocates
org.apache.commons → org.apache.fluss.shaded.org.apache.commons. It does
not relocate com.fasterxml or com.google. Two consequences:

1. hdfs / s3 / oss / cos / azure / obs pull jackson + guava from their direct
   deps (hadoop-common, aws-sdk, azure/gcs libs). Those land unshaded at the jar
   top level → leak. (These modules also bundle a second, correctly-relocated
   jackson copy inherited from fluss-fs-hadoop-shaded, which is why a relocated
   org/apache/fluss/fs/shaded/hadoop3/com/fasterxml copy also appears — but the
   unshaded copy is what breaks downstream apps.)
2. fluss-fs-gs is the worst offender: its shade config relocates nothing
   (not even org.apache.commons), and it does not depend on
   fluss-fs-hadoop-shaded, so it has no relocated copy to lean on. The Google
   Cloud Storage libs drag in a huge com.google tree (guava + protobuf + gax),
   all unshaded → 7,351 leaked guava entries.

The fix proposed in #4073 (relocate com.fasterxml + com.google + add the MRJ
exclusion filter) is exactly what the other six need. Note: on main, s3 still
leaks like the rest (1,093 jackson + 2,042 guava) — #4073 is an open PR, not yet
merged, so s3's fix only lands when that PR merges. Until then, s3 sits in the same
"to-fix" bucket as the other six in this issue; once #4073 merges, s3 drops out and
the remaining six (gs worst) become the work.

Proposed fix

Two viable strategies:

• (a) Centralize in fluss-filesystems parent pom — add the relocation block
  (com.fasterxml, com.google, org.codehaus, com.ctc, com.google.re2j,
  org.apache.htrace) plus the MRJ exclusion filter once, so every fs plugin
  inherits it. One place to maintain; matches the pattern [build] Relocate shaded jackson and other internal deps in fluss-fs-s3 #4073 used for s3.
  This is the lowest-risk way to get all 7 consistent.
• (b) Per-plugin fix — mirror [build] Relocate shaded jackson and other internal deps in fluss-fs-s3 #4073 into each plugin's pom.xml
  (fluss-fs-gs first, then hdfs/oss/cos/azure/obs; s3 is already covered by
  [build] Relocate shaded jackson and other internal deps in fluss-fs-s3 #4073). More localized, easier to review per plugin, but repeats config 6×.

Per-plugin verification: 0 unshaded com/fasterxml and com/google entries
in the produced uber-jar (base path + MRJ), with constant-pool references
rewritten to the shaded path, and hadoop's own
org.apache.hadoop.thirdparty.com.google namespace left untouched.

Status

• fluss-fs-s3: verified leaked on main (1,093 jackson + 2,042 guava); fix proposed in open PR [build] Relocate shaded jackson and other internal deps in fluss-fs-s3 #4073 (not merged yet)
• fluss-fs-hdfs: verified leaked (1,073 + 2,731) ← natural starting point
• fluss-fs-oss / cos / azure / obs: verified leaked (~1,073 jackson + ~2,000–2,250 guava)
• fluss-fs-gs: verified leaked, worst (1,073 jackson + 7,351 guava)

Happy to drive this (centralized parent-pom fix, or per-plugin starting with gs)
once #4073 lands.

References

• [FLUSS-3553][build] Fix shaded Jackson classes leaking into uber-jars #3884 — fluss-client / flink uber-jars jackson MRJ leak (shipped)
• [build] Relocate shaded jackson and other internal deps in fluss-fs-s3 #4073 — fluss-fs-s3 jackson + guava relocation (open PR, not yet merged)
• fluss-fs-hadoop-shaded/pom.xml — existing correct relocation pattern to copy

Solution

No response

Are you willing to submit a PR?

• I'm willing to submit a PR!

[binary-signal]

I reproduced this while working through the same class of leak across the rest of the build, and the per-plugin jackson and guava numbers here match what I measured. One correction to the note about fluss-fs-hadoop-shaded, though.

The issue says:

fluss-fs-hadoop-shaded itself is fine: it relocates com.fasterxml → org.apache.fluss.fs.shaded.hadoop3.com.fasterxml and ships hadoop's guava as org.apache.hadoop.thirdparty.com.google (already third-partied by hadoop, not leaked).

That is right about jackson, but the module also ships an unshaded guava and an unshaded commons of its own. Built clean from its pre-existing pom:

  package                  leaked      mrj  relocated
  com/fasterxml                 0        0       1033
  org/codehaus                  0        0        125
  com/ctc                       0        0        205
  com/google/re2j               0        0         47
  org/apache/commons         2228        0          0  LEAK
  com/google/common          1954        0          0  LEAK

The jar does contain 2094 org/apache/hadoop/thirdparty/com/google entries, which is presumably what the note refers to — but that copy sits alongside 1954 unshaded com.google.common classes rather than instead of them. Since every filesystem plugin bundles this module, its leak propagates into all of them, which is part of why the guava counts in the table above are as high as they are.

Its <relocations> block covers com.google.re2j, org.apache.htrace, com.fasterxml, org.codehaus and com.ctc, and never mentions guava or commons.

I have written the leaks that fall outside this issue up as #4143, and #4144 implements them, including relocating guava and the eight bundled commons packages in fluss-fs-hadoop-shaded. It also covers the six sibling filesystem plugins measured here, so there is overlap with this issue — happy to rework the PR if you would rather own that part, since you filed first.

One thing worth carrying over into any fix here: the relocation patterns need to name the packages a jar actually bundles rather than the com.google or org.apache.commons prefix. The shade plugin rewrites every reference matching a pattern, including references to classes the jar does not contain, which turns a soft dependency into a coordinate nothing can provide. A broad org.apache.commons in this module breaks the NameNode outright:

java.lang.NoClassDefFoundError: org/apache/fluss/shaded/org/apache/commons/cli/ParseException
    at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1713)

fluss-fs-hadoop-shaded references but does not bundle commons-cli, commons-codec, commons-math3, commons-net, com.google.protobuf and com.google.gson. The same caution applies to #4073's com.google pattern, which I have commented on separately.