Large overhead due to device init code
- Dominant language
- C++
- Stars
- 885
- Forks
- 181
- PR merge metrics
- No merged PRs in 30d
Description
I was profiling the performance of multiple zfp compressions/decompressions and noticed that the newer (unreleased) zfp code contains a "device init" function that is called whenever the execution mode is set for a zfp stream.
https://github.com/llnl/zfp/blob/cccbb9d5e69a2998f63eb06153ca3b7b63901b56/src/cuda/device.cuh#L17-L53
called from here
https://github.com/llnl/zfp/blob/cccbb9d5e69a2998f63eb06153ca3b7b63901b56/src/cuda/interface.cu#L20-L42
Here is a list of events from a profile I took that shows two different compression calls:
For our use case, the data to be compressed is relatively small (8MB-32MB), so the compression call itself only runs for ~40us, while the other calls related to this device init code incur up to 100s of microseconds of latency. I was able to resolve this by only calling `zfp::cuda::internal::device_init` the first time `zfp_internal_cuda_init` is called. I think this is an acceptable approach, since I believe the device init function is just making sure that the application has correctly enabled the CUDA runtime and that the application is able to run the kernels. After device init is succeeds once, it is likely that it will succeed throughout the course of the application. Do I have a misconception about the purpose of this code or is my suggested code change valid?
Contributor guide
Research direction
Start with src/cuda/device.cuh lines 17-53 and src/cuda/interface.cu lines 20-42, then compare the initialization path with the reported profile. Determine whether device_init needs to run on every execution-mode change; done means repeated compression and decompression calls avoid the initialization overhead without changing CUDA behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 50/100