cockroachdb / cockroachdb/cockroach
cloud/s3: audit memory usage during uploads
Open
A-disaster-recovery
C-investigation
O-support
P-3
T-disaster-recovery
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
In https://github.com/cockroachdb/cockroach/issues/115190 we observed 600+ mb of heap allocated memory due to the s3 sdk buffers. With a concurrency of 5 and chunk size of 8mb, this number did not initially make sense to us. We should dig in more here to see if we're configuring something wrong.
Additional context in thread leading up to https://cockroachlabs.slack.com/archives/C05P9KPRZFW/p1701175750732169?thread_ts=1701123237.549029&cid=C05P9KPRZFW
Jira issue: CRDB-33929
Contributor guide
Assessment
This issue has not been assessed yet.