GoogleCloudPlatform / GoogleCloudPlatform/PerfKitBenchmarker

Integrate x-mem for numa aware memory benchmarking

Open
#5,230 0 comments 0 reactions 0 assignees View on GitHub
good first issue
Dominant language
Python
Stars
2k
Forks
562
Avg merge
4h 55m
Merged PRs (30d)
69

Description

https://github.com/microsoft/X-Mem

This task adds X-Mem as a new benchmark for PKB. It can be divided into a few different components:

1) Add the benchmarking framework
- create xmem_benchmark.py file in linux_benchmarks
- create xmem_benchmark.py in windows benchmarks
- populate BENCHMARK_NAME and BENCHMARK_CONFIG constants so that the benchmark can be found by PKB
- create GetConfig, Prepare, Run and Cleanup functions handlers with pass/return [] as function content.
At this point you can run your new benchmark in PKB (though it will not do anything yet).
2) Install xmem
- Download the xmem installer (use the OS that you will prefer to add to PKB) and try installing on your machine.
- Create a new package called xmem.py in linux_packages/windows_packages.
- create a new function, Install, that takes a vm as an input and installs xmem on that vm. You should be able to wrap shell commands as vm.RemoteCommand
- Install for windows and ubuntu2404 are top priority, other linux OSes are nice to have.
- add unit tests to make sure the remote commands are issued using mock to mock the vm
3) Run xmem
- Download and run xmem locally, you don't need to use PKB to run xmem. This part is about parsing the output from xmem into some sensible format.
- Store the xmem output as a file in the data directory under data/xmem. This file will be your raw data for parsing and parsing unit tests.
Add a function in xmem.py that you added to linux_packages/windows_packages with a sensible name, e.g. ParseResults
Parse results should take a str as input and produce a list of PKB Samples as output. You goal is to parse the output into useful samples, where each sample as a metric name, metric value, metric unit, metric metadata. Each row of Geekbench's output should be a separate metric.
Test the parser function

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.