alibaba / alibaba/DataX

作业使用多task之后 内存增长很快,远超过真实传输的数据大小,导致年轻代GC频繁,有人遇到过吗

Open
#2,222 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

This issue has no description.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing a DataX job that uses multiple tasks and observe memory growth, the amount of data transferred, and young-generation GC frequency. Identify which task-related behavior accounts for the excess memory, then verify the behavior with a representative multi-task job and confirm that memory usage is proportional to the workload.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.