Check for SASS changes to `cub::DeviceTransform` in CI
- Dominant language
- C++
- Stars
- 2.5k
- Forks
- 486
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 295
Description
@MatthiasKohl, who is working on evaluating the use of `cub::DeviceTransform` in pytorch, suggested that pytorch would like to have more rigorous regression testing in case they would adopt `cub::DeviceTransform`, since they seem to have bitten by compiler codegen regressions in the past. What they would like to see is a CI test, that verifies that any change unrelated to `cub::DeviceTransform`, notably an upgrade of the compiler, leaves the SASS identical. This point is important, we want to compare across different compiler versions.
The way I imagine this to work is that we would take a SASS dump of either a `cub::DeviceTransform` unit test or benchmark (or both) and store it as a reference dump somewhere. Each CI run would produce a new SASS dump and compare that no changes happened. This comparison may need some intelligence or filtering, to narrow down on only the kernels produced by `cub::DeviceTransform` (unit tests and benchmarks contain a lot of other kernels as well).
Producing a SASS dump on the fly for the state of the repository before and after a PR is insufficient, since it does not guard as against code changes introduced by upgrading toolchains.
Any PR than actively changes `cub::DeviceTransform` would obviously fail this test, which is good. The PR would then need to update the reference SASS diffs as well, after inspection.
Contributor guide
Assessment
This issue has not been assessed yet.