apache / apache/arrow

read_feather error with 32 GB file

Open
#11,665 14 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

arrow v.6.0.0.2 results in the following error when attempting read_feather on a 32 GB feather file:

Error: Invalid: Invalid read (offset = 7140512496, size = -956703880)

Contributor guide

Open the contributing guide

Research direction

Start by reproducing read_feather on a 32 GB Feather file with Arrow v.6.0.0.2 and record whether the reported offset and negative size recur. Trace the read path that handles these values and verify the same file can be read successfully without the Invalid read error.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.