RMAPI_GPU_LOCK_INTERNAL alloc requested without holding the RMAPI lock

NVIDIA Open GPU Kernel Modules Version

545.29.06

Please confirm this issue does not happen with the proprietary driver (of the same version). This issue tracker is only for bugs specific to the open kernel driver.

• I confirm that this does not happen with the proprietary driver package.

Operating System and Version

Arch Linux

Kernel Release

6.6.7

Please confirm you are running a stable release kernel (e.g. not a -rc). We do not accept bug reports for unreleased kernels.

• I am running on a stable kernel release.

Hardware: GPU

NVIDIA GeForce RTX 4060 (10de:28e0)

Describe the bug

dmesg spam with:

 NVRM rmapiAllocWithSecInfo: RMAPI_GPU_LOCK_INTERNAL alloc requested without holding the RMAPI lock

To Reproduce

Boot on such system, and check dmesg.

Bug Incidence

Always

nvidia-bug-report.log.gz

nvidia-bug-report.log.gz

More Info

I tried to debug the issue by appending a os_stack_trace() after where the message is printed, I got the following backtrace:

[  854.764480] NVRM rmapiAllocWithSecInfo: RMAPI_GPU_LOCK_INTERNAL alloc requested without holding the RMAPI lock
[  854.764483] CPU: 15 PID: 5542 Comm: kworker/15:0 Tainted: G           OE      6.6.7-arch1-1 #1 4505c4baa0b3d7c4037b0e8f5402626fa360717f
[  854.764486] Hardware name: ASUSTeK COMPUTER INC. ROG Zephyrus G14 GA402XV_GA402XV/GA402XV, BIOS GA402XV.313 08/10/2023
[  854.764487] Workqueue: pm pm_runtime_work
[  854.764490] Call Trace:
[  854.764491]  <TASK>
[  854.764492]  dump_stack_lvl+0x47/0x60
[  854.764496]  rmapiAllocWithSecInfo+0x306/0x410 [nvidia 17e5ed799529ff7ce72e190f3462b0a04291b9d9]
[  854.764574]  ? srso_alias_return_thunk+0x5/0x7f
[  854.764575]  ? __kmem_cache_alloc_node+0x1a6/0x340
[  854.764577]  ? os_alloc_mem+0xc8/0xe0 [nvidia 17e5ed799529ff7ce72e190f3462b0a04291b9d9]
[  854.764631]  ? os_alloc_mem+0xc8/0xe0 [nvidia 17e5ed799529ff7ce72e190f3462b0a04291b9d9]
[  854.764684]  ? srso_alias_return_thunk+0x5/0x7f
[  854.764686]  ? __kmalloc+0x50/0x150
[  854.764689]  rmapiAlloc+0x27/0x40 [nvidia 17e5ed799529ff7ce72e190f3462b0a04291b9d9]
[  854.764758]  memdescSendMemDescToGSP+0x171/0x2c0 [nvidia 17e5ed799529ff7ce72e190f3462b0a04291b9d9]
[  854.764837]  ? memdescSendMemDescToGSP+0x120/0x2c0 [nvidia 17e5ed799529ff7ce72e190f3462b0a04291b9d9]
[  854.764912]  fbsrCopyMemoryMemDesc_GM107+0x46a/0xe80 [nvidia 17e5ed799529ff7ce72e190f3462b0a04291b9d9]
[  854.764988]  ? _issueRpcAndWait+0x3c/0x210 [nvidia 17e5ed799529ff7ce72e190f3462b0a04291b9d9]
[  854.765074]  _memmgrWalkHeap+0x156/0x680 [nvidia 17e5ed799529ff7ce72e190f3462b0a04291b9d9]
[  854.765146]  memmgrSavePowerMgmtState_KERNEL+0x18b/0x320 [nvidia 17e5ed799529ff7ce72e190f3462b0a04291b9d9]
[  854.765218]  gpuPowerManagementEnter.constprop.0+0x6a/0x2e0 [nvidia 17e5ed799529ff7ce72e190f3462b0a04291b9d9]
[  854.765295]  ? srso_alias_return_thunk+0x5/0x7f
[  854.765298]  gpuEnterStandby_IMPL+0x109/0x280 [nvidia 17e5ed799529ff7ce72e190f3462b0a04291b9d9]
[  854.765369]  ? srso_alias_return_thunk+0x5/0x7f
[  854.765372]  RmPowerManagementInternal+0x113/0x1a0 [nvidia 17e5ed799529ff7ce72e190f3462b0a04291b9d9]
[  854.765451]  RmGcxPowerManagement+0x2fc/0x360 [nvidia 17e5ed799529ff7ce72e190f3462b0a04291b9d9]
[  854.765522]  ? rmGpuLocksAcquire+0xbb/0x130 [nvidia 17e5ed799529ff7ce72e190f3462b0a04291b9d9]
[  854.765601]  rm_transition_dynamic_power+0x83/0x122 [nvidia 17e5ed799529ff7ce72e190f3462b0a04291b9d9]
[  854.765671]  ? srso_alias_return_thunk+0x5/0x7f
[  854.765676]  nv_pmops_runtime_suspend+0x6f/0x100 [nvidia 17e5ed799529ff7ce72e190f3462b0a04291b9d9]
[  854.765726]  pci_pm_runtime_suspend+0x67/0x1e0
[  854.765728]  ? __pfx_pci_pm_runtime_suspend+0x10/0x10
[  854.765730]  __rpm_callback+0x41/0x170
[  854.765732]  ? __pfx_pci_pm_runtime_suspend+0x10/0x10
[  854.765734]  rpm_callback+0x5d/0x70
[  854.765736]  ? __pfx_pci_pm_runtime_suspend+0x10/0x10
[  854.765738]  rpm_suspend+0x120/0x6a0
[  854.765740]  ? __pfx_pci_pm_runtime_idle+0x10/0x10
[  854.765742]  pm_runtime_work+0x84/0xb0
[  854.765745]  process_one_work+0x171/0x340
[  854.765747]  worker_thread+0x27b/0x3a0
[  854.765749]  ? __pfx_worker_thread+0x10/0x10
[  854.765750]  kthread+0xe5/0x120
[  854.765752]  ? __pfx_kthread+0x10/0x10
[  854.765754]  ret_from_fork+0x31/0x50
[  854.765756]  ? __pfx_kthread+0x10/0x10
[  854.765758]  ret_from_fork_asm+0x1b/0x30
[  854.765762]  </TASK>

Backtraces are almost the same each time it printed.

Start with the reported call chain at rmapiAllocWithSecInfo and follow it through memdescSendMemDescToGSP, memmgrSavePowerMgmtState_KERNEL, and rm_transition_dynamic_power. Reproduce by booting the affected system and checking dmesg; done means the lock warning no longer appears during runtime power management.