bloomreach / bloomreach/s4cmd

reduced performance in version 2

Open
#72 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.4k
Forks
209
PR merge metrics
No merged PRs in 30d

Description

Hello, I noticed a significant performance decrease with version 2.0.1 (boto3 1.3.1) relative to version
1.5.20 (boto 2.40.0). With version 1.5, a list operation of a bucket takes 3 seconds on average, while
with version 2 it takes 5 seconds on average. Previous version was downloading ~70GB in 15 minutes,
current version downloads ~40GB. An automated task that executes periodically a set of different
operations across multiple buckets and regions, took 5 to 6 times longer. Let me know if I could assist
somehow. Thank you.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the reported bucket list and download operations with boto3 1.3.1 and boto 2.40.0 across the affected buckets and regions. Compare timings for the list operation and the automated multi-operation task; done means identifying a reproducible regression and documenting or testing the affected path.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python
Domain
cli, cloud, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.