E3SM-Project / E3SM-Project/scream
Allow building with Cuda UVM and run select kernels on host.
- Dominant language
- No language data
- Stars
- 79
- Forks
- 54
- PR merge metrics
- No merged PRs in 30d
Description
Logging ideas that sparked in a conversation with @ndkeen .
During debug/development on GPU, it might be helpful to run a handful of kernels on Host (rather than device), to allow bisecting where an error is generating. Currently, on GPU, we use the native Cuda memory space, which means data is not accessible on host. However, CudaUVM should allow that.
To allow running a particular kernel on host, we should allow to build scream with CudaUVM (if explicitly requested). CudaUVM might be slower than native Cuda space, but we would use this just to debug, and memory space should not have any impact on the actual numbers generated (that is, Cuda should be bfb with CudaUVM). Once we verify things work correctly, we can try to swap policies:
```
// This is the "normal" policy
// const auto policy = ekat::ExeSpaceUtils::get_default_team_policy(nj, nk_pack);
const auto policy = ekat::ExeSpaceUtils::get_default_team_policy(nj, nk_pack);
Kokkos::parallel_for (
"my_kernel",
policy,
KOKKOS_LAMBDA(const MemberType& team) {
...
});
```
Ideally, that should be enough to force `my_kernel` to run on CPU. This might be a view a tad too naive, but I think it should work...more or less. It is basically what YAKL does for BFB runs.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.