Respect physical data placement in parquet iterators
Open
storage
- Dominant language
- Go
- Stars
- 11.7k
- Forks
- 802
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 80
Description
`parquet.List` of structs colocates fields on the same page. This means that we should never create an individual iterator for each of the columns in such cases ([example](https://github.com/grafana/pyroscope/blob/main/pkg/phlaredb/sample_merge.go#L120)): in fact, we fetch same pages repeatedly.
In addition, parquet reader issues a read operation for every ReadBufferPage size (+ page/group bounds), which prevents efficient streaming of data ranges from object storage.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.