apache / apache/arrow-rs-object-store
move s3 express session token fetching out of request path
- Dominant language
- Rust
- Stars
- 322
- Forks
- 212
- Avg merge
- 5d 2h
- Merged PRs (30d)
- 10
Description
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
requests to S3 Express require creating and refreshing a 'Session Token' every 5 minutes.
This `CreateSession` API calls takes > 300ms to complete.
Currently `CreateSession` is implictly called by object-store lib whenever a request is made and the token is out of date. This however greatly increases the latency of the request itself.
For example, if you were doing a GET request that you expected to complete in single digit miliseconds, due to the `CreateSession` call being bundled with it, you now get a response time of >300 ms.
**Describe the solution you'd like**
Session Tokens should be refreshed in the background, instead of synchronously as part of request to S3 Express. Current implementation can cause massive p99 increases by bundling the `CreateSession` call with any PUT/GET call that is expected to take single digit ms.
**Describe alternatives you've considered**
**Additional context**
Contributor guide
Research direction
Start by tracing the object-store library's S3 Express request path and its CreateSession token handling. Determine how token refresh can run in the background without delaying PUT or GET requests, then verify that requests no longer wait for the roughly 300 ms session call when a token expires.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, rust
- Domain
- cloud
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100