Further S3 Support Umbrella Issue
- Dominant language
- C++
- Stars
- 1.5k
- Forks
- 145
- PR merge metrics
- No merged PRs in 30d
Description
This issue tracks remaining issues from the TODO list here: https://github.com/google/tensorstore/pull/91#issue-1649705676
Related:
- https://github.com/google/tensorstore/issues/46
- https://github.com/google/tensorstore/pull/91
# KVStore
- [ ] Versioned Bucket Support. From a high-level perspective might involve combining the `x-amz-version-id` and `etag` headers in `StorageGenerations.`
- [ ] S3 specific rate-limiting (per endpoint?). Current S3 rate limiting is based on the GCS rate limiting code. Relevant documentation
- https://docs.aws.amazon.com/AmazonS3/latest/userguide/optimizing-performance.html
- https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/
- https://docs.aws.amazon.com/sdkref/latest/guide/feature-retry-behavior.html
# Testing
- [x] ~~Upgrade to localstack 2.3 (release targeted for [26th Sep 2023](https://github.com/localstack/localstack/milestone/4))~~
- ~~`tensorstore::internal::TestKeyValueReadWriteOps` currently succeeds locally on `localstack/localstack:latest` image (c.f. https://github.com/localstack/localstack/issues/9076#issuecomment-1711974643)~~
- [ ] ~~Install localstack in github actions~~
- [x] Moto as S3 testharness. https://github.com/google/tensorstore/commit/8a7f109b4b60b7371929a2c7467516e10671d8f3, https://github.com/google/tensorstore/commit/81aef761d36cff582169fe5b2ab452bcccb6079a
- [ ] Benchmarking See https://github.com/google/tensorstore/pull/91#discussion_r1248172227
# Authorisation
- [x] #119
- [x] #115.
- https://github.com/google/tensorstore/commit/b9b90a21e8c8f7aab92f7699746d70f58890e01f
# Logging
- Review use of [fancy logging](https://github.com/google/tensorstore/commit/00b4519ac8ebcc6f46e2bfaa4f4a3edc2b8faee1) within S3 in general
Contributor guide
Assessment
This issue has not been assessed yet.