kernel_tuning_net gtest is fragile due to violating blackbox
Nobody has claimed this yet.
- Dominant language
- Assembly
- Stars
- 431
- Forks
- 407
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 441
Description
[Migrated from original issue] ROCm/MIOpen#3708
Original issue author: @randyspauldingamd
Test failure observed after updating CK:
[2025-05-01T10:36:12.397Z] /home/jenkins/workspace/MLLIBS_MIOpen_PR-3707/test/gtest/kernel_tuning_net.cpp:188: Failure
[2025-05-01T10:36:12.397Z] Expected equality of these values:
[2025-05-01T10:36:12.397Z] perf_config.ToString()
[2025-05-01T10:36:12.397Z] Which is: "DeviceGroupedConvBwdDataMultipleD_Xdl_CShuffle_v1<128, 128, 32, 32, 8, 8, Default, 32, 32, 2, 1, 8, 8, 1, 1, BlkGemmPipelineScheduler: Intrawave, BlkGemmPipelineVersion: v1>"
[2025-05-01T10:36:12.397Z] test_case.expected_config
[2025-05-01T10:36:12.397Z] Which is: "DeviceGroupedConvBwdDataMultipleD_Xdl_CShuffle_v1<128, 128, 32, 32, 8, 8, Default, 32, 32, 2, 1, 8, 8, 1, 1>"
@msaudulhassan reports that this is expected behavior when CK and the ktn models get out of sync. This makes the test inherently fragile, and it should be rewritten to use mocks and canned "gold" model files to achieve blackbox. In other words, a proper test shall test only the code itself, and not involve the actual version of CK nor model files.
The test shall be disabled for now. The priority and timeline are uncertain at this time, so if anyone can suggest an acceptable workaround, please do!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with test/gtest/kernel_tuning_net.cpp, especially the failure at line 188, and review how CK and ktn model files are used. Determine the mock and canned “gold” model boundaries described in the issue; done means the test checks its own behavior without depending on the installed CK version or live model files, with the current fragile test disabled until then.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- testing-qa
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100