HUDI Experiences Significant Delays When Initializing ServiceabilityAgentSupport with Velox Enabled
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
The current version of HUDI, when Velox (a C++ component) is enabled, uses ObjectSizeCalculator.getObjectSize, which triggers the initialization of ServiceabilityAgentSupport (in singleton mode, initialization occurs on the first call). During this process, the execution of the needSudo method takes approximately 50 seconds. If we restart a pod with Spark each time, it results in a significant delay, which is a critical issue. Although it's possible to bypass this by setting the JVM parameter -Djol.skipHotspotSAAttach=true, this workaround requires additional configuration to restore normal operation. Is this still considered an unexpected behavior?
Here's the private constructor for ServiceabilityAgentSupport:
{code:java}
private ServiceabilityAgentSupport() {
processId = getCurrentProcId();
agentStyle = senseAgentStyle();
sudoRequired = needSudo(agentStyle);
}
{code}
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-7329
- Type: Bug
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with ObjectSizeCalculator.getObjectSize and the ServiceabilityAgentSupport constructor, especially needSudo, then reproduce the delay in a Spark pod with Velox enabled. Compare behavior with -Djol.skipHotspotSAAttach=true and determine whether the initialization delay is expected; done means the behavior and any required configuration are clearly established.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, java, spark
- Domain
- data-engineering, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 45/100