google / google/tensorstore

Tensorstore key value store driver not Retrying on 429 Responses with S3 API

Open
#194 8 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
1.5k
Forks
145
PR merge metrics
No merged PRs in 30d

Description

I'm using Tensorstore's key value store driver with OCI object storage using their S3 compatibility layer and am seeing that when the OCI Object Storage service sends 429s Tensorstore fails immediately without any retries. This was tested with debug logging enabled which also didn't show any retries happening either with the default value of 32 or with `s3_request_retries` set to a custom number.
I also tried testing this @sjperkins 's PR [here](https://github.com/google/tensorstore/pull/149) to see if that may enable retries to work here however I'm hitting bazel build issues on a M1 Mac of:
```
configurable attribute "content" in @com_github_aws_c_common//:write_config_h doesn't match this configuration. Would a default condition help?
```

I was wondering if there was anything I could do to debug the lack of retries further or any modifications I could make to force retries.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.