NVIDIA / NVIDIA/nvbandwidth

./nvbandwidth -t device_to_device_memcpy_read_ce failed due to unknown issue

Open
#34 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
778
Forks
93
PR merge metrics
No merged PRs in 30d

Description

I execute ./nvbandwidth -t device_to_device_memcpy_read_ce

However, it output as follows:

$ ./nvbandwidth -t device_to_device_memcpy_read_ce  
nvbandwidth Version: v0.7
Built from Git version: v0.7

CUDA Runtime Version: 12040
CUDA Driver Version: 12040
Driver Version: 550.127.08

Device 0: NVIDIA A100-SXM4-80GB (00000000:34:00)
Device 1: NVIDIA A100-SXM4-80GB (00000000:35:00)
Device 2: NVIDIA A100-SXM4-80GB (00000000:36:00)
Device 3: NVIDIA A100-SXM4-80GB (00000000:37:00)
Device 4: NVIDIA A100-SXM4-80GB (00000000:9b:00)
Device 5: NVIDIA A100-SXM4-80GB (00000000:9c:00)
Device 6: NVIDIA A100-SXM4-80GB (00000000:9d:00)
Device 7: NVIDIA A100-SXM4-80GB (00000000:9e:00)

Running device_to_device_memcpy_read_ce.
 Invalid value when checking the pattern at 0x70a640000000
 Current offset : 11136 
ASSERT in expression h_errorFlag == 0 in void memcmpPatternHelper(CUstream, CUdeviceptr, long long unsigned int, unsigned int, CompareKernelFunc, std::shared_ptr<NodeHelper>)() : /mnt/project/cuda_related/nvbandwidth/memcpy.cpp:165

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing ./nvbandwidth -t device_to_device_memcpy_read_ce with the reported CUDA and A100 environment. Read memcpy.cpp around line 165 and trace the invalid pattern check at offset 11136; done means identifying the cause and resolving or documenting the failing assertion with a verified run.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.