GC benchmarks won't run when setting COMPlus_GCCpuGroup
- Dominant language
- F#
- Stars
- 773
- Forks
- 301
- Avg merge
- 6d 9h
- Merged PRs (30d)
- 15
Description
I'm trying to run the GC benchmarks and enable CPU Groups since the machine I'm benchmarking has >64 processors ([reference](https://docs.microsoft.com/en-us/dotnet/core/run-time-config/garbage-collector#cpu-groups)). When I try to add the `COMPlus_GCCpuGroup` environment variable to the run file, the Python script throws an error.
Run file:
```yaml
vary: coreclr
test_executables:
defgcperfsim: C:\Users\alex\source\repos\performance\artifacts\bin\GCPerfSim\release\netcoreapp5.0\GCPerfSim.dll
coreclrs:
a:
core_root: C:\Users\alex\source\repos\runtime-master\artifacts\tests\coreclr\windows.x64.Release\Tests\Core_Root
options:
default_iteration_count: 2
default_max_seconds: 300
common_config:
complus_gcserver: true
complus_gcconcurrent: false
complus_gcheapcount: 254
complus_gccpugroup: 1
benchmarks:
0gb:
arguments:
tc: 254
tagb: 300
tlgb: 0
lohar: 0
pohar: 0
sohsr: 100-4000
lohsr: 102400-204800
pohsr: 100-204800
sohsi: 0
lohsi: 0
pohsi: 0
sohpi: 0
lohpi: 0
sohfi: 0
lohfi: 0
pohfi: 0
allocType: reference
testKind: time
min_seconds: 10
scores:
speed:
FirstToLastGCSeconds:
weight: 1
PauseDurationMSec_95P:
weight: 1
```
Steps to reproduce:
```shell
$ py . run .\bench\suite-test\normal_server.yaml --overwrite
Error in C:\Users\alex\source\repos\performance\src\benchmarks\gc\bench\suite-test\normal_server.yaml
Traceback (most recent call last):
File "C:\Users\alex\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "C:\Users\alex\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File ".\__main__.py", line 9, in
run_command(ALL_COMMANDS)
File ".\src\commonlib\command.py", line 87, in run_command
run_command_worker(commands, cmd_and_args)
File ".\src\commonlib\command.py", line 243, in run_command_worker
cast(Callable[[Any], None], command.fn)(args)
File ".\src\exec\run_tests.py", line 106, in run
run_test(args)
File ".\src\exec\run_tests.py", line 118, in run_test
bench = parse_bench_file(bench_file_path)
File ".\src\commonlib\bench_file.py", line 1215, in parse_bench_file
return BenchFileAndPath(load_yaml(BenchFile, path), path)
File ".\src\commonlib\parse_and_serialize.py", line 84, in load_yaml
cls, safe_load(f), path, PropPath.root(f"load_yaml {cls}"), all_optional
File ".\src\commonlib\parse_and_serialize.py", line 248, in _yaml_to_typed
return cast(T, unwrap(_try_yaml_to_typed(cls, o, yaml_file_path, desc, all_optional)))
File ".\src\commonlib\parse_and_serialize.py", line 237, in _try_yaml_to_typed
handle_dataclass=handle_dataclass,
File ".\src\commonlib\type_utils.py", line 235, in match_type
return default_handler(t) if handle_dataclass is None else handle_dataclass(t)
File ".\src\commonlib\parse_and_serialize.py", line 218, in handle_dataclass
all_non_err(_get_field(fld) for fld in flds),
File ".\src\commonlib\result_utils.py", line 14, in all_non_err
for x in xs:
File ".\src\commonlib\parse_and_serialize.py", line 218, in
all_non_err(_get_field(fld) for fld in flds),
File ".\src\commonlib\parse_and_serialize.py", line 205, in _get_field
fld.type, d[fld.name], yaml_file_path, child(fld.name)
File ".\src\commonlib\parse_and_serialize.py", line 237, in _try_yaml_to_typed
handle_dataclass=handle_dataclass,
File ".\src\commonlib\type_utils.py", line 216, in match_type
return default_handler(t) if handle_union is None else handle_union(union)
File ".\src\commonlib\parse_and_serialize.py", line 229, in
members, lambda t: _try_yaml_to_typed(t, o, yaml_file_path, desc, all_optional)
File ".\src\commonlib\parse_and_serialize.py", line 37, in try_for_each_union_member
res = try_get(member)
File ".\src\commonlib\parse_and_serialize.py", line 229, in
members, lambda t: _try_yaml_to_typed(t, o, yaml_file_path, desc, all_optional)
File ".\src\commonlib\parse_and_serialize.py", line 237, in _try_yaml_to_typed
handle_dataclass=handle_dataclass,
File ".\src\commonlib\type_utils.py", line 235, in match_type
return default_handler(t) if handle_dataclass is None else handle_dataclass(t)
File ".\src\commonlib\parse_and_serialize.py", line 218, in handle_dataclass
all_non_err(_get_field(fld) for fld in flds),
File ".\src\commonlib\result_utils.py", line 14, in all_non_err
for x in xs:
File ".\src\commonlib\parse_and_serialize.py", line 218, in
all_non_err(_get_field(fld) for fld in flds),
File ".\src\commonlib\parse_and_serialize.py", line 205, in _get_field
fld.type, d[fld.name], yaml_file_path, child(fld.name)
File ".\src\commonlib\parse_and_serialize.py", line 237, in _try_yaml_to_typed
handle_dataclass=handle_dataclass,
File ".\src\commonlib\type_utils.py", line 216, in match_type
return default_handler(t) if handle_union is None else handle_union(union)
File ".\src\commonlib\parse_and_serialize.py", line 229, in
members, lambda t: _try_yaml_to_typed(t, o, yaml_file_path, desc, all_optional)
File ".\src\commonlib\parse_and_serialize.py", line 37, in try_for_each_union_member
res = try_get(member)
File ".\src\commonlib\parse_and_serialize.py", line 229, in
members, lambda t: _try_yaml_to_typed(t, o, yaml_file_path, desc, all_optional)
File ".\src\commonlib\parse_and_serialize.py", line 237, in _try_yaml_to_typed
handle_dataclass=handle_dataclass,
File ".\src\commonlib\type_utils.py", line 233, in match_type
return default_handler(t) if handle_primitive is None else handle_primitive(t)
File ".\src\commonlib\parse_and_serialize.py", line 148, in handle_primitive
raise Exception(f"TODO: handle {p}")
Exception: TODO: handle
```
The benchmarks run fine if I remove `complus_gccpugroup: 1` from the run file, but I want to run the benchmarks with CPU groups enabled.
If I try and set the environment variable before running the Python script, I get the following error:
```shell
$ $env:COMPlus_GCCpuGroup = 1
$ py . run .\bench\suite-test\normal_server.yaml --overwrite
Traceback (most recent call last):
File "C:\Users\alex\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "C:\Users\alex\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File ".\__main__.py", line 9, in
run_command(ALL_COMMANDS)
File ".\src\commonlib\command.py", line 87, in run_command
run_command_worker(commands, cmd_and_args)
File ".\src\commonlib\command.py", line 243, in run_command_worker
cast(Callable[[Any], None], command.fn)(args)
File ".\src\exec\run_tests.py", line 106, in run
run_test(args)
File ".\src\exec\run_tests.py", line 145, in run_test
built, bench, args.skip_where_exists, args.max_iterations, out_dir, run_errors
File ".\src\exec\run_tests.py", line 166, in _run_all_benchmarks
default_env = check_env()
File ".\src\exec\run_single_test.py", line 207, in check_env
raise Exception(msg)
Exception: Environment variables should not be set: COMPLUS_GCCPUGROUP
```
Extra info for troubleshooting:
```shell
$ py --version
Python 3.7.4
$ dotnet --info
.NET SDK (reflecting any global.json):
Version: 6.0.100-preview.6.21355.2
Commit: 7f8e0d76c0
Runtime Environment:
OS Name: Windows
OS Version: 10.0.20348
OS Platform: Windows
RID: win10-x64
Base Path: C:\Program Files\dotnet\sdk\6.0.100-preview.6.21355.2\
Host (useful for support):
Version: 6.0.0-preview.6.21352.12
Commit: 770d630b28
.NET SDKs installed:
5.0.302 [C:\Program Files\dotnet\sdk]
5.0.400-preview.21328.4 [C:\Program Files\dotnet\sdk]
6.0.100-preview.6.21355.2 [C:\Program Files\dotnet\sdk]
.NET runtimes installed:
Microsoft.AspNetCore.App 3.1.15 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 5.0.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 5.0.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.0-preview.6.21355.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 3.1.15 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 5.0.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 5.0.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.0-preview.6.21352.12 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 3.1.15 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 5.0.6 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 5.0.8 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.0-preview.6.21353.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
To install additional .NET runtimes or SDKs:
https://aka.ms/dotnet-download
```
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.