[BUG] Memory allocations from flushing L2 can lead to significant delays between benchmark executions
Nobody has claimed this yet.
- Dominant language
- Cuda
- Stars
- 927
- Forks
- 123
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 2
Description
Description
In an attempt to gather more accurate timings, nvbench will "flush" the L2 cache by querying the device's L2 cache size, allocating device memory of that size, memset that memory to zero, and then free it.
NVBench will do this between every cold iteration. This can be quite expensive if there are a large number of cold iterations or points in the benchmark axis space. @GregoryKimball reported that this can cause up to a 1.2s delay between each iteration as cudaMalloc/cudaFree can be quite expensive.
Possible Solutions
- Add option to disable flushing L2 cache
- Avoid allocating/freeing every time and instead make a single allocation per device and memset the same every allocation each time.
- Enable user to provide their own allocator to allocate the memory used for flushing.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with nvbench/detail/l2flush.cuh and trace where L2 flushing runs between cold iterations. Measure the allocation and free costs across benchmark points, then compare the listed approaches for preserving cache-flush behavior without repeated allocations. Done means the selected approach removes the reported delays while retaining accurate cold-iteration timings.
Written by the indexing model from the issue text.
Assessment
- Domain
- performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100