Packing broken when not using pinned memory or ATS
@corbett5 is already working on this.
Since Jan 22, 2021.
- Dominant language
- C++
- Stars
- 287
- Forks
- 109
- Avg merge
- 4d 41m
- Merged PRs (30d)
- 5
Description
When we're not using pinned memory our packing buffers are standard host allocations. They are written to directly from the GPU and this only works on Lassen because of the Address Translation Service which is not present on most GPU machines and is quite slow.
My suggested fix is that we stop using a std::vector and instead use a Array1D. When not using pinned memory this can use the standard array1d< buffer_unit_type >, which will move the memory between spaces as appropriate. For pinned memory we should create a new LvArray buffer type that allocates pinned memory (or more generally uses a specific Umpire allocator). This buffer type would no do any memory movement.
This will require changes to at least the device packing functions to take an array1d instead of a pointer but that should be pretty easy.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.