[Python] Error of wrong number of rows read from Parquet file
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 91
Description
on version greater than 6.0.1 fail to read tables saying expected length n, got n=1 rows
Tables can be read column by column fine, or with a fixed number of rows matching the meta data fine. Reads correctly in version 6.0.1
**Environment**: Python Info
3.10.7 (tags/v3.10.7:6cc6b13, Sep 5 2022, 14:08:36) [MSC v.1933 64 bit (AMD64)]
Pyarrow Info
6.0.1
Platform Info
Windows-10-10.0.19042-SP0
Windows
10
10.0.19042
19042
AMD64
**Reporter**: [Blake erickson](https://issues.apache.org/jira/browse/ARROW-18064)
#### Original Issue Attachments:
- [badplug.parquet](https://issues.apache.org/jira/secure/attachment/13050964/badplug.parquet)
- [readBadParquet.py](https://issues.apache.org/jira/secure/attachment/13050963/readBadParquet.py)
- [screenshot-1.png](https://issues.apache.org/jira/secure/attachment/13051179/screenshot-1.png)
**Note**: *This issue was originally created as [ARROW-18064](https://issues.apache.org/jira/browse/ARROW-18064). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*
Contributor guide
Research direction
Start by running the attached readBadParquet.py against badplug.parquet and compare the reported row count across the stated PyArrow versions. Inspect the attached reproduction and screenshot to establish the failing read path; done means reproducing the mismatch, identifying its cause, and adding a regression test for the affected Parquet read behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100