Does aioboto3 Support Authentication with EC2 IAM Roles?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1k
- Forks
- 305
- Avg merge
- 22h 37m
- Merged PRs (30d)
- 4
Description
Describe the bug
While using the boto3 library, I can authenticate and interact with AWS services using the IAM role assigned to my EC2 instance. However, when attempting to perform similar operations with s3fs, I encounter errors. Below is the code snippet demonstrating the behavior with boto3 and the corresponding error with s3fs.
Environment:
- Python Version: 3.10
- OS name and version: ubuntu
- s3fs-2024.3.1
Sample code
import boto3
# Create boto3 client
s3 = boto3.client('s3')
# List contents of a bucket
response = s3.list_objects_v2(Bucket='your-bucket-name')
print(response)
import s3fs
fs = s3fs.S3FileSystem(anon=False)
# Attempt to list the same bucket with s3fs, expecting to use the same IAM role
print(fs.ls('your-bucket-name'))
Error log
Traceback (most recent call last):
File "/home/ubuntu/.local/lib/python3.10/site-packages/s3fs/core.py", line 720, in _lsdir
async for c in self._iterdir(
File "/home/ubuntu/.local/lib/python3.10/site-packages/s3fs/core.py", line 770, in _iterdir
async for i in it:
File "/home/ubuntu/.local/lib/python3.10/site-packages/aiobotocore/paginate.py", line 30, in __anext__
response = await self._make_request(current_kwargs)
File "/home/ubuntu/.local/lib/python3.10/site-packages/aiobotocore/client.py", line 408, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the ListObjectsV2 operation: No AWSAccessKey was presented.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/ubuntu/.local/lib/python3.10/site-packages/fsspec/asyn.py", line 118, in wrapper
return sync(self.loop, func, *args, **kwargs)
File "/home/ubuntu/.local/lib/python3.10/site-packages/fsspec/asyn.py", line 103, in sync
raise return_result
File "/home/ubuntu/.local/lib/python3.10/site-packages/fsspec/asyn.py", line 56, in _runner
result[0] = await coro
File "/home/ubuntu/.local/lib/python3.10/site-packages/s3fs/core.py", line 1005, in _ls
files = await self._lsdir(path, refresh, versions=versions)
File "/home/ubuntu/.local/lib/python3.10/site-packages/s3fs/core.py", line 733, in _lsdir
raise translate_boto_error(e)
PermissionError: No AWSAccessKey was presented.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the reported calls in s3fs/core.py, especially _lsdir, _iterdir, and _ls, and compare S3FileSystem(anon=False) with the working boto3 client. Reproduce the EC2 IAM-role case using the provided sample and verify whether fs.ls can list the bucket without an access key; document or fix the behavior based on that result.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100