Suggestion: Exception messages should print the bucket/key
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1k
- Forks
- 305
- Avg merge
- 22h 37m
- Merged PRs (30d)
- 4
Description
Hello
I think it might make sense to alter exception output and on IO operations that fail output also the path. As of now, there's no indication of the bucket/key for which the error has occured, e.g.:
Traceback (most recent call last):
....
File "/var/runtime/botocore/client.py", line 576, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
....
File "/opt/python/s3fs/core.py", line 349, in _lsdir
raise translate_boto_error(e)
PermissionError: Access Denied
which makes it difficult to debug (e.g. figure out which permissions are missing), and my usual workaround is to re-raise the error, which just increases the bolier-plate code all over the project:
try:
df = pd.read_parquet(fname)
except Exception as e:
raise Exception(f'Error on read {fname}: {e}') from e
I understand though, that the counter-argument could be that it might not make sense due to security reasons (i.e. that the bucket/key will stay in logs and it could be sensitive). But outputting paths looks like being a standard behavior of python libs:
>> open('/var/bla-bla-bla', 'r')
FileNotFoundError: [Errno 2] No such file or directory: '/var/bla-bla-bla'
Best regards
Vladimir
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing s3fs/core.py at _lsdir, where the example raises translate_boto_error(e), and compare the underlying botocore client error shown in the report. Before changing behavior, determine which IO operations and path details are in scope and how sensitive bucket/key values should be handled; done means the agreed exception messages identify the failed path without creating an unsafe logging behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- cloud
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100