apache / apache/fluss

[build] fluss-fs-s3 uber-jar bundles 1,114 unshaded jackson classes that can shadow downstream jackson

Open
#4,072 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
2.1k
Forks
625
Avg merge
3d 14h
Merged PRs (30d)
97

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.

### Fluss version

main (development)

### Please describe the bug 🐞

### Describe the bug

The `fluss-fs-s3` uber-jar bundles `hadoop-common` and `aws-java-sdk-*`
dependencies **without any shade relocations**. As a result, 1,114 unshaded
jackson class files leak into the jar at `com/fasterxml/*` paths:

| Package | Approx. count | Source |
| --- | --- | --- |
| `com/fasterxml/jackson/databind/**` | ~800 | hadoop-common |
| `com/fasterxml/jackson/core/**` | ~200 | hadoop-common |
| `com/fasterxml/jackson/annotation/**` | ~20 | hadoop-common |
| `com/fasterxml/jackson/dataformat/cbor/**` | ~50 | aws-java-sdk-core |
| `META-INF/versions/{11,17,19}/com/fasterxml/**` (MRJ) | ~25 | jackson-core MRJ |

When a downstream application puts `fluss-fs-s3` on its classpath alongside its
own jackson-core (e.g. 2.16+), the leaked classes can shadow the application's
jackson and cause runtime failures such as `NoSuchMethodError` — the same
failure mode reported in #3553 for the MRJ entries, but here affecting the full
jackson-databind/core/annotations/cbor surface, not just MRJ entries.

In contrast, `fluss-fs-hadoop-shaded` already relocates these same packages
(`com.fasterxml`, `org.codehaus`, `com.ctc`, `com.google.re2j`,
`org.apache.htrace`) into `org.apache.fluss.fs.shaded.hadoop3.*`.

### To Reproduce

```bash
./mvnw -pl fluss-filesystems/fluss-fs-s3 -am -DskipTests package
unzip -l fluss-filesystems/fluss-fs-s3/target/fluss-fs-s3-*.jar | grep -c " com/fasterxml"
# -> 1114
```

### Expected behavior

All bundled third-party dependencies should be relocated under Fluss's shaded
namespace (mirroring the `fluss-fs-hadoop-shaded` pattern), so the uber-jar
never exposes `com/fasterxml/*` classes on the classpath.

### Additional context

- Same issue class as #3553 (MRJ entries escaping shade relocation), but a
larger surface: the entire jackson family is unshaded, not just MRJ entries.
- Other filesystem plugins (`fluss-fs-oss`, `fluss-fs-gs`, `fluss-fs-azure`,
`fluss-fs-obs`, `fluss-fs-cos`, `fluss-fs-hdfs`) may have the same gap; can
be checked in follow-ups.

### Solution

Mirror the `fluss-fs-hadoop-shaded` approach: add `maven-shade-plugin`
relocations for `com.fasterxml.jackson.*`, `com.fasterxml.jackson.databind.*`,
`com.fasterxml.jackson.dataformat.cbor.*` to `org.apache.fluss.fs.shaded.s3.*`,
and add a `` to exclude `META-INF/versions/*/com/fasterxml/**` (MRJ
entries are not relocated by the shade plugin). A draft PR is in progress.

### Are you willing to submit a PR?

- [x] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Build fluss-filesystems/fluss-fs-s3 with the Maven command in the issue and inspect the resulting uber-jar for com/fasterxml entries. Compare its packaging configuration with fluss-fs-hadoop-shaded, then verify that the S3 jar no longer exposes unrelocated Jackson classes or MRJ entries while downstream Jackson remains usable.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
build-system
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.