Better cost estimates for read_intermediate_result
Open
performance
- Dominant language
- C
- Stars
- 12.8k
- Forks
- 794
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 31
Description
We currently use some very basic guesstimates to determine the row count of `read_intermediate_result`, based on the file size and column types. We don't consider things like row selectivity and we consider all variable fields to be 32 bytes in length. This might lead to poor join plans when there are selective filters or the row size estimate is way off.
We could consider reading the first few rows of large files to get a better estimate for the row size as well as row selectivity, or some other form of sampling.
See also: Hadi's comments in #1869
Contributor guide
Assessment
This issue has not been assessed yet.