apache / apache/arrow

[Python] pyarrow.lib.ArrowInvalid: Not a Feather V1 or Arrow IPC file

Open
#33,412 8 comments 0 reactions 0 assignees View on GitHub
Component: Python Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 18h
Merged PRs (30d)
91

Description

## feather.read_dataframe throws the error below.

Traceback (most recent call last):
  File "./cv.py", line 86, in
    get_training('HeLa-S3'),
  File "./cv.py", line 19, in get_training
    feather.read_dataframe(f'\{cell_line}-training.feather').set_index(['chr1', 'x1', 'x2', 'chr2', 'y1', 'y2'])
  File "/home/bade/.local/lib/python3.7/site-packages/pyarrow/feather.py", line 208, in read_feather
    return (read_table(source, columns=columns, memory_map=memory_map)
  File "/home/bade/.local/lib/python3.7/site-packages/pyarrow/feather.py", line 230, in read_table
    reader.open(source, use_memory_map=memory_map)
  File "pyarrow/feather.pxi", line 67, in pyarrow.lib.FeatherReader.open
  File "pyarrow/error.pxi", line 123, in pyarrow.lib.pyarrow_internal_check_status
  File "pyarrow/error.pxi", line 85, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: Not a Feather V1 or Arrow IPC file

**Environment**: ubuntu
**Reporter**: [bade tutuş](https://issues.apache.org/jira/browse/ARROW-18226)
#### Original Issue Attachments:
- [HeLa-S3-training.feather](https://issues.apache.org/jira/secure/attachment/13051719/HeLa-S3-training.feather)

**Note**: *This issue was originally created as [ARROW-18226](https://issues.apache.org/jira/browse/ARROW-18226). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

Contributor guide

Open the contributing guide

Research direction

Start by reproducing feather.read_dataframe with the attached HeLa-S3-training.feather file and inspect the call sites in cv.py at lines 19 and 86. Read the linked migration documentation and determine whether the attachment is a valid Feather or Arrow IPC file; done means the rejection cause and expected handling are established.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.