concurrently processing parquet files will consume a lot of memory, which will cause Lightning OOM [restore for 48576]
Open
type/enhancement
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Enhancement
https://pingcap.feishu.cn/wiki/wikcnsWqAhawzcb2SJQ6d4LXRt3
Flow control based on memory limits
Contributor guide
Assessment
This issue has not been assessed yet.