Columnar: memory requirements when inserting into partitioned tables.
Open
columnar
enhancement
- Dominant language
- C
- Stars
- 12.8k
- Forks
- 794
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 31
Description
If a table has many columnar partitions, it could create many concurrent writeStates, which may consume a lot of memory. If possible, find a way to flush some of the write states to balance memory vs. stripe size.
Contributor guide
Assessment
This issue has not been assessed yet.