apache / apache/arrow

NativeFile.download throws seekable error

Open
#33,165 0 comments 0 reactions 0 assignees View on GitHub
Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 18h
Merged PRs (30d)
91

Description

The following code doesn't work:
```java

f.download("some_path")
```
   where `f` is a `pyarrow.NativeFile`.

I get the error:

```

(_execute_read_task pid=126666)     f.download(temp_file)
(_execute_read_task pid=126666)   File "pyarrow/io.pxi", line 576, in pyarrow.lib.NativeFile.download
(_execute_read_task pid=126666)   File "pyarrow/io.pxi", line 291, in pyarrow.lib.NativeFile.seek
(_execute_read_task pid=126666)   File "pyarrow/io.pxi", line 190, in pyarrow.lib.NativeFile.get_random_access_file
(_execute_read_task pid=126666)   File "pyarrow/io.pxi", line 219, in pyarrow.lib.NativeFile._assert_seekable
(_execute_read_task pid=126666) OSError: only valid on seekable files

```

 

 

**Environment**: Ubuntu
**Reporter**: [Vedant Roy](https://issues.apache.org/jira/browse/ARROW-17957)

**Note**: *This issue was originally created as [ARROW-17957](https://issues.apache.org/jira/browse/ARROW-17957). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

Contributor guide

Open the contributing guide

Research direction

Start in pyarrow/io.pxi at NativeFile.download and follow the seek path through NativeFile.seek and _assert_seekable. Reproduce the shown download call with a non-seekable NativeFile, then run the relevant PyArrow tests if available. Done means NativeFile.download handles the reported file type without raising the seekable-files error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.