NVIDIA / NVIDIA/nvbench

[BUG] Memory allocations from flushing L2 can lead to significant delays between benchmark executions

Open
#100 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

P1: should have
Dominant language
Cuda
Stars
927
Forks
123
Avg merge
2d 10h
Merged PRs (30d)
2

Description

Description

In an attempt to gather more accurate timings, nvbench will "flush" the L2 cache by querying the device's L2 cache size, allocating device memory of that size, memset that memory to zero, and then free it.

NVBench will do this between every cold iteration. This can be quite expensive if there are a large number of cold iterations or points in the benchmark axis space. @GregoryKimball reported that this can cause up to a 1.2s delay between each iteration as cudaMalloc/cudaFree can be quite expensive.

Possible Solutions

  1. Add option to disable flushing L2 cache
  2. Avoid allocating/freeing every time and instead make a single allocation per device and memset the same every allocation each time.
  3. Enable user to provide their own allocator to allocate the memory used for flushing.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with nvbench/detail/l2flush.cuh and trace where L2 flushing runs between cold iterations. Measure the allocation and free costs across benchmark points, then compare the listed approaches for preserving cache-flush behavior without repeated allocations. Done means the selected approach removes the reported delays while retaining accurate cold-iteration timings.

Written by the indexing model from the issue text.

Assessment

Domain
performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.