GoogleCloudPlatform / GoogleCloudPlatform/PerfKitBenchmarker
Create an SCP benchmark
- Dominant language
- Python
- Stars
- 2k
- Forks
- 562
- Avg merge
- 4h 55m
- Merged PRs (30d)
- 69
Description
This task adds a SCP benchmark for PKB. The goal of the benchmark is to scp a file to another vm and measure the time taken.
It can be divided into a few different components:
Add the benchmarking framework
create scp_benchmark.py file in linux_benchmarks
create scp_benchmark.py in windows benchmarks
populate BENCHMARK_NAME and BENCHMARK_CONFIG constants so that the benchmark can be found by PKB
create GetConfig, Prepare, Run and Cleanup functions handlers with pass/return [] as function content.
At this point you can run your new benchmark in PKB (though it will not do anything yet).
Install SCP
scp comes by default in linux, but may need installation in a windows machine. You can refer here for winscp installation process using powershell. https://winscp.net/eng/docs/library_install#google_vignette
Set up remote disks
Update BENCHMARK_CONFIG so that it is similar to netperf_benchmark. https://github.com/GoogleCloudPlatform/PerfKitBenchmarker/blob/master/perfkitbenchmarker/linux_benchmarks/netperf_benchmark.py
Update the BENCHMARK_CONFIG such that it has a disk_spec. e.g. disk_spec: *default_500_gb At this point, your VMs will be created with a remote disk.
The remote disks defaults to /scratch directory. You can validate this by ssh into the VM and issuing lsblk (or Get-Disk on Windows) command.
Prepare for benchmark run
In each of the VMs, create a large file (10GB is a start) using commands such as fallocate -l 10G foobar.img
Using python threading, concurrently copy each VM's file over to the /scratch directory of the other VM over internet. Start a timer before the copy command and time when the copy finishes.
Retrieve the time from both VMs and report the max time as the total run time.
Contributor guide
Assessment
This issue has not been assessed yet.