GoogleCloudPlatform / GoogleCloudPlatform/gsutil

Perfdiag processes vs threads

Open
#1,710 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
918
Forks
335
PR merge metrics
No merged PRs in 30d

Description

I ran `gsutil perfdiag` from an instance on GCP to a bucket in GCP. First run was with 5 processes with 4 threads each and the second was 4 processes with 5 threads each, copying 20 largs objects (2GB).
Both in `fan` parallelism mode.
I get bad performance on the second experiment. Is it a bug in gsutil in process scope (lock? Connection pool?)

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the two `gsutil perfdiag` runs described against a GCP bucket: 5 processes with 4 threads and 4 processes with 5 threads, both using `fan` parallelism. Compare process-scope locking and connection-pool behavior, and document whether the performance difference is a confirmed bug and where it occurs.

Written by the indexing model from the issue text.

Assessment

Tech stack
gcp, python
Domain
cli, cloud, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.