failed to run sm read cases on l40s platform

I have a compute node with 8 L40s gpus. when I run nvbandwidth, the following cases failed/aborted:

• device_to_device_memcpy_read_sm
• device_to_device_bidirectional_memcpy_read_sm
• all_to_one_read_sm
• one_to_all_read_sm

All these cases aborted with message like:

Invalid value when checking the pattern

The following is one of them:

$ nvbandwidth -t  one_to_all_read_sm.
nvbandwidth Version: v0.4
Built from Git version: v0.4

NOTE: This tool reports current measured bandwidth on your system.
Additional system-specific tuning may be required to achieve maximal peak bandwidth.

CUDA Runtime Version: 12020
CUDA Driver Version: 12020
Driver Version: 535.154.05

Device 0: NVIDIA L40S
Device 1: NVIDIA L40S
Device 2: NVIDIA L40S
Device 3: NVIDIA L40S
Device 4: NVIDIA L40S
Device 5: NVIDIA L40S
Device 6: NVIDIA L40S
Device 7: NVIDIA L40S

Running one_to_all_read_sm.
 Invalid value when checking the pattern at <0x7fac6200e480>
 Current offset [ 58496/66306048]
Aborted (core dumped)

please help to check what is the problem, thanks.

Start by reproducing the reported failures with nvbandwidth on an 8-GPU L40S system, especially one_to_all_read_sm and the other named read_sm cases. Trace the handling of the reported pattern check and current offset, then verify that all four cases complete without aborting and retain valid bandwidth measurements.