[Python] pyarrow.lib.ArrowInvalid: Not a Feather V1 or Arrow IPC file
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 91
Description
## feather.read_dataframe throws the error below.
Traceback (most recent call last):
File "./cv.py", line 86, in
get_training('HeLa-S3'),
File "./cv.py", line 19, in get_training
feather.read_dataframe(f'\{cell_line}-training.feather').set_index(['chr1', 'x1', 'x2', 'chr2', 'y1', 'y2'])
File "/home/bade/.local/lib/python3.7/site-packages/pyarrow/feather.py", line 208, in read_feather
return (read_table(source, columns=columns, memory_map=memory_map)
File "/home/bade/.local/lib/python3.7/site-packages/pyarrow/feather.py", line 230, in read_table
reader.open(source, use_memory_map=memory_map)
File "pyarrow/feather.pxi", line 67, in pyarrow.lib.FeatherReader.open
File "pyarrow/error.pxi", line 123, in pyarrow.lib.pyarrow_internal_check_status
File "pyarrow/error.pxi", line 85, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: Not a Feather V1 or Arrow IPC file
**Environment**: ubuntu
**Reporter**: [bade tutuş](https://issues.apache.org/jira/browse/ARROW-18226)
#### Original Issue Attachments:
- [HeLa-S3-training.feather](https://issues.apache.org/jira/secure/attachment/13051719/HeLa-S3-training.feather)
**Note**: *This issue was originally created as [ARROW-18226](https://issues.apache.org/jira/browse/ARROW-18226). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*
Contributor guide
Research direction
Start by reproducing feather.read_dataframe with the attached HeLa-S3-training.feather file and inspect the call sites in cv.py at lines 19 and 86. Read the linked migration documentation and determine whether the attachment is a valid Feather or Arrow IPC file; done means the rejection cause and expected handling are established.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100