[build] fluss-fs-s3 uber-jar bundles 1,114 unshaded jackson classes that can shadow downstream jackson
- Dominant language
- Java
- Stars
- 2.1k
- Forks
- 625
- Avg merge
- 3d 14h
- Merged PRs (30d)
- 97
Description
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.
### Fluss version
main (development)
### Please describe the bug 🐞
### Describe the bug
The `fluss-fs-s3` uber-jar bundles `hadoop-common` and `aws-java-sdk-*`
dependencies **without any shade relocations**. As a result, 1,114 unshaded
jackson class files leak into the jar at `com/fasterxml/*` paths:
| Package | Approx. count | Source |
| --- | --- | --- |
| `com/fasterxml/jackson/databind/**` | ~800 | hadoop-common |
| `com/fasterxml/jackson/core/**` | ~200 | hadoop-common |
| `com/fasterxml/jackson/annotation/**` | ~20 | hadoop-common |
| `com/fasterxml/jackson/dataformat/cbor/**` | ~50 | aws-java-sdk-core |
| `META-INF/versions/{11,17,19}/com/fasterxml/**` (MRJ) | ~25 | jackson-core MRJ |
When a downstream application puts `fluss-fs-s3` on its classpath alongside its
own jackson-core (e.g. 2.16+), the leaked classes can shadow the application's
jackson and cause runtime failures such as `NoSuchMethodError` — the same
failure mode reported in #3553 for the MRJ entries, but here affecting the full
jackson-databind/core/annotations/cbor surface, not just MRJ entries.
In contrast, `fluss-fs-hadoop-shaded` already relocates these same packages
(`com.fasterxml`, `org.codehaus`, `com.ctc`, `com.google.re2j`,
`org.apache.htrace`) into `org.apache.fluss.fs.shaded.hadoop3.*`.
### To Reproduce
```bash
./mvnw -pl fluss-filesystems/fluss-fs-s3 -am -DskipTests package
unzip -l fluss-filesystems/fluss-fs-s3/target/fluss-fs-s3-*.jar | grep -c " com/fasterxml"
# -> 1114
```
### Expected behavior
All bundled third-party dependencies should be relocated under Fluss's shaded
namespace (mirroring the `fluss-fs-hadoop-shaded` pattern), so the uber-jar
never exposes `com/fasterxml/*` classes on the classpath.
### Additional context
- Same issue class as #3553 (MRJ entries escaping shade relocation), but a
larger surface: the entire jackson family is unshaded, not just MRJ entries.
- Other filesystem plugins (`fluss-fs-oss`, `fluss-fs-gs`, `fluss-fs-azure`,
`fluss-fs-obs`, `fluss-fs-cos`, `fluss-fs-hdfs`) may have the same gap; can
be checked in follow-ups.
### Solution
Mirror the `fluss-fs-hadoop-shaded` approach: add `maven-shade-plugin`
relocations for `com.fasterxml.jackson.*`, `com.fasterxml.jackson.databind.*`,
`com.fasterxml.jackson.dataformat.cbor.*` to `org.apache.fluss.fs.shaded.s3.*`,
and add a `` to exclude `META-INF/versions/*/com/fasterxml/**` (MRJ
entries are not relocated by the shade plugin). A draft PR is in progress.
### Are you willing to submit a PR?
- [x] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Build fluss-filesystems/fluss-fs-s3 with the Maven command in the issue and inspect the resulting uber-jar for com/fasterxml entries. Compare its packaging configuration with fluss-fs-hadoop-shaded, then verify that the S3 jar no longer exposes unrelocated Jackson classes or MRJ entries while downstream Jackson remains usable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- build-system
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100