Handle ExpiredToken by generating a new one if possible
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1k
- Forks
- 305
- Avg merge
- 22h 37m
- Merged PRs (30d)
- 4
Description
Hello folks,
I did some reading on the code and s3fs is setting the token to the maximum allowed 12h. Let's say that I have a long-running job that keeps reading/writing to an S3 bucket for 13 hours that job will fail with an error similar to this:
botocore.exceptions.ClientError: An error occurred (ExpiredToken) when calling the UploadPart operation: The provided token has expired.
I'm not sure how easy it would be to do this. But it would be great if we could request a new token upon this exception.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing how s3fs obtains its maximum-duration token and how the UploadPart operation surfaces botocore's ExpiredToken exception. Check whether credentials can be refreshed after expiry; done means a long-running S3 read or write can continue past 12 hours when renewal is possible, with coverage for the expired-token case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- cloud
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100