[Improvement] Supply a script to check shaded jar which have relocated but not exist class
- Dominant language
- Java
- Stars
- 454
- Forks
- 172
- Avg merge
- 5d 17h
- Merged PRs (30d)
- 5
Description
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issues](https://github.com/apache/incubator-uniffle/issues?q=is%3Aissue) and found no similar issues.
### What would you like to be improved?
Now we cannot prevent developer using a dependency which have provided scope directly or indirectly, we may relocate a package name something like a.b.c to org.apache.uniffle.shaded.a.b.c, but we could use a class a.b.c.d.MyClass which is in a provided dependency, so it could be relocated to `org.apache.uniffle.shaded.a.b.c.d` too, but this is not what we want, the MyClass should be provided out of uniffle.
If we do not find and resolve this, we could encounter a ClassNotFound Exception, so developer have to do some efforts to defeat this issue, maybe we can add a `a/b/c/d/**/*` to the configuration of shaded plugin.
So this issue introduce a way to check and find the relocated but not exist class within shaded jar, developer can find the missing list and add it to exclude if check failed.
### How should we improve?
We should provide a high-quality shaded jar to user which rare to encounter ClassNotFound and MethodNotDef issue to user.
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Apporach
- script
```shell
#!/bin/bash
# Constants
PROCYON_JAR_URL="https://github.com/mstrobel/procyon/releases/download/v0.6.0/procyon-decompiler-0.6.0.jar"
PROCYON_JAR_PATH="/tmp/procyon-decompiler-0.6.0.jar"
DECOMPILE_OUTPUT_DIR="/tmp/uniffle-shaded-client-decompile"
SHADOWED_CLIENT_IMPORTS_LIST="$DECOMPILE_OUTPUT_DIR/shaded.list"
SHADOWED_PACKAGE="org.apache.uniffle.shaded"
# Function to check missing imports
check_missing_imports() {
if [ "$#" -ne 1 ]; then
echo "Usage: $0 "
exit 1
fi
INPUT_FILE="$1"
while IFS= read -r line
do
class_path=$(echo "$line" | sed -n 's/^import \(.*\);$/\1/p')
file_path="${DECOMPILE_OUTPUT_DIR}/${class_path//.//}.java"
if [ ! -f "$file_path" ]; then
echo "Missing: $class_path should be at $file_path"
fi
done < "$INPUT_FILE"
}
# Clean up previous runs
clean_up() {
for file in "$PROCYON_JAR_PATH" "$DECOMPILE_OUTPUT_DIR" "$SHADOWED_CLIENT_IMPORTS_LIST"; do
if [ -e "$file" ]; then
rm -rf "$file"
fi
done
}
# Download Procyon
wget -q "$PROCYON_JAR_URL" -O "$PROCYON_JAR_PATH"
# Decompile uniffle shaded client and find import lines
java -jar "$PROCYON_JAR_PATH" "$1" -o "$DECOMPILE_OUTPUT_DIR"
find "$DECOPILE_OUTPUT_DIR" -name "*.java" -print0 | xargs -0 grep -h "import $SHADOWED_PACKAGE.*" | sort | uniq > "$SHADOWED_CLIENT_IMPORTS_LIST"
# Start check
check_missing_imports "$SHADOWED_CLIENT_IMPORTS_LIST"
```
Contributor guide
Research direction
The issue provides a Bash script that decompiles the shaded client jar with Procyon and checks imports under org.apache.uniffle.shaded. Start by reviewing and running that approach against the shaded jar, then verify that it reports relocated classes missing from the decompiled output. Done means developers receive a reliable missing-class list they can use to adjust shade-plugin excludes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- bash, java
- Domain
- build-system, tooling
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100