apache / apache/hudi

Shading of aircompressor in hudi-io module

Open
#17,016 1 comment 0 reactions 0 assignees View on GitHub
from-jira priority:high type:devtask
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

h1. *Presto*

Shading of hudi-io might be required for Presto might be required.
h1. *Trino*

Shading of hudi-io is not required as the SPI specifications guarantees classloader isolation.

Reference:
[https://trino.io/docs/current/develop/spi-overview.html#building-plugins-via-maven]

 

Quoting the spec:
{code:java}
All other dependencies are based on what the plugin needs for its own implementation. Plugins are loaded in a separate class loader to provide isolation and to allow plugins to use a different version of a library that Trino uses internally.{code}
 

However, there might still be risks of class conflict and they are:
# Hudi's aircompressor classes "leak" out of the plugin

*
** Leaking should not happen too as hudi-io and its usages are not returning an instance of AirCompressor to the Trino's SPI

# Trino attempts to use Hudi's version
** This is unlikely given the classloader hierarchy and if this happens, it is a bug on Trino's end in which they will need to fix as this breaks their SPI contract/guarantees

 

 

 

## JIRA info

- Link: https://issues.apache.org/jira/browse/HUDI-9416
- Type: Task

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the hudi-io module and the linked Trino SPI overview, then trace whether AirCompressor classes cross the plugin boundary. The issue is done when the need for shading is resolved for the Presto/Trino cases and the classloader-risk conclusion is documented or reflected in the module configuration.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
build-system
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.