apache / apache/arrow

[Python] Misleading error reported when `pyarrow.fs.FileSystem.from_uri` has authentication error

Open
#47,214 0 comments 0 reactions 0 assignees View on GitHub
Component: Python Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the bug, including details regarding any error messages, version, and platform.

Affects versions <21.0.0. #44900 I believe fixed this (possibly inadvertently) for version 21.

When using constructing a `ParquetDataset`, for example with `pyarrow.parquet.read_table`, all ValueErrors are excepted on [these lines in `pyarrow.parquet.core`](https://github.com/apache/arrow/blob/apache-arrow-20.0.0/python/pyarrow/parquet/core.py#L1333-L1336)

If the filesystem exists but the user gets a permission error, for example, this is excepted. Then, a new `ArrowInvalid` error is going to be thrown from on [the next line](https://github.com/apache/arrow/blob/apache-arrow-20.0.0/python/pyarrow/parquet/core.py#L1338) because the path is interpreted as local. Typically:
```
ArrowInvalid: Expected a local file system path, got a URI
```
This is confusing as the original permission error is nowhere in the stacktrace. Could the new error be raised from the original `ValueError` in this case, or the except clause narrowed to avoid suppressing the 'true' error? Potentially partially backporting #44900 is all that is needed.

### Component(s)

Python

### Example

Version <21
```python
>>> import pyarrow.parquet as pq
>>> import pyarrow.fs
>>>
>>> pyarrow.fs.FileSystem.from_uri("abfss://a/b/c")
pyarrow.lib.ArrowInvalid: AzureOptions doesn't contain a valid account name
>>> pq.read_table("abfss://a/b/c")
pyarrow.lib.ArrowInvalid: Expected a local filesystem path, got a URI: 'abfss://a/b/c/'
```
Version >=21
```python
>>> import pyarrow.parquet as pq
>>> import pyarrow.fs
>>>
>>> pyarrow.fs.FileSystem.from_uri("abfss://a/b/c")
pyarrow.lib.ArrowInvalid: AzureOptions doesn't contain a valid account name
>>> pq.read_table("abfss://a/b/c")
pyarrow.lib.ArrowInvalid: AzureOptions doesn't contain a valid account name
```
I would be happy to contribute a fix if agreed and if these type of minor issues are generally fixed in older versions

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.