apache / apache/hudi

HUDI Experiences Significant Delays When Initializing ServiceabilityAgentSupport with Velox Enabled

Open
#16,377 1 comment 0 reactions 0 assignees View on GitHub
from-jira priority:high type:bug
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

The current version of HUDI, when Velox (a C++ component) is enabled, uses ObjectSizeCalculator.getObjectSize, which triggers the initialization of ServiceabilityAgentSupport (in singleton mode, initialization occurs on the first call). During this process, the execution of the needSudo method takes approximately 50 seconds. If we restart a pod with Spark each time, it results in a significant delay, which is a critical issue. Although it's possible to bypass this by setting the JVM parameter -Djol.skipHotspotSAAttach=true, this workaround requires additional configuration to restore normal operation. Is this still considered an unexpected behavior?

Here's the private constructor for ServiceabilityAgentSupport:

 
{code:java}
private ServiceabilityAgentSupport() {
processId = getCurrentProcId();
agentStyle = senseAgentStyle();
sudoRequired = needSudo(agentStyle);
}
{code}
 

 

## JIRA info

- Link: https://issues.apache.org/jira/browse/HUDI-7329
- Type: Bug

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with ObjectSizeCalculator.getObjectSize and the ServiceabilityAgentSupport constructor, especially needSudo, then reproduce the delay in a Spark pod with Velox enabled. Compare behavior with -Djol.skipHotspotSAAttach=true and determine whether the initialization delay is expected; done means the behavior and any required configuration are clearly established.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, java, spark
Domain
data-engineering, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.