apache / apache/arrow

read_parquet super slow

Open
#13,720 9 comments 1 reaction 0 assignees View on GitHub
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

My computer has 64 GB RAM and the parquet files are not large at all. However, they take surprisingly long time to read. I don't know what is the potential cause. If you need more info, please feel free to let me know.

![image](https://user-images.githubusercontent.com/54338793/181128725-9e15567a-2754-4a90-8d34-3c53f5f95bab.png)

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the reported read_parquet timing with the parquet files described in the issue and inspect the attached performance evidence. No source file or test is identified, and completion would require isolating the cause of the unexpectedly slow read before defining a fix.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
data, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.