NVIDIA / NVIDIA/open-gpu-kernel-modules
GSP: PFM_REQ_HNDLR_STATE_SYNC_CALLBACK missing from _kgspProcessRpcEvent bootup whitelist
Nobody has claimed this yet.
- Dominant language
- C
- Stars
- 17.4k
- Forks
- 1.9k
- PR merge metrics
- No merged PRs in 30d
Description
Summary
NV_VGPU_MSG_EVENT_PFM_REQ_HNDLR_STATE_SYNC_CALLBACK (0x101a) is not on the bootup-poll whitelist in _kgspProcessRpcEvent (src/nvidia/src/kernel/gpu/gsp/kernel_gsp.c, around line 1436). When this RPC arrives while CPU-RM is polling for GSP_INIT_DONE without holding the API lock, it falls through to the default: arm, emits a misleading "during bootup without API lock" error, hits NV_ASSERT(0) at kernel_gsp.c:1447, and the RPC is silently dropped, so the Platform Request Handler never sees that wake's state sync.
This is a log-noise / dropped-RPC report, not a crash. The driver continues to function, but the kernel log accumulates two NVRM error lines per affected wake.
Symptom
On a Blackwell laptop dGPU (RTX PRO 1000, driver 595.71.05, NVreg_DynamicPowerManagement=3), runtime-D3 wake-ups produce the following pair:
NVRM: _kgspProcessRpcEvent: Attempted to process RPC event from GPU0: 0x101a (PFM_REQ_HNDLR_STATE_SYNC_CALLBACK) during bootup without API lock
NVRM: nvAssertFailedNoLog: Assertion failed: 0 @ src/nvidia/src/kernel/gpu/gsp/kernel_gsp.c:1447
Roughly two-thirds of runtime-D3 wakes produce the pair; the rest are clean. Timing-dependent, with clusters of all-assert and all-clean consecutive wakes, consistent with the RPC being driven by the platform handler's own sync schedule rather than the wake itself. The "during bootup without API lock" wording is misleading: this is well past initial GSP boot, on a steady-state runtime-D3 wake.
Root cause
The whitelist immediately above the default: arm enumerates the RPCs that are explicitly safe (or expected) to arrive during the bootup poll window. PFM_REQ_HNDLR_STATE_SYNC_CALLBACK is simply absent from that list:
case NV_VGPU_MSG_EVENT_UCODE_LIBOS_PRINT:
case NV_VGPU_MSG_EVENT_GSP_LOCKDOWN_NOTICE:
case NV_VGPU_MSG_EVENT_GSP_POST_NOCAT_RECORD:
case NV_VGPU_MSG_EVENT_GSP_INIT_DONE:
case NV_VGPU_MSG_EVENT_OS_ERROR_LOG:
case NV_VGPU_MSG_EVENT_GSP_LOAD_EXEC_GENERIC_BOOTLOADER:
case NV_VGPU_MSG_EVENT_GSP_LOAD_EXEC_HS_BINARY:
break;
default:
NV_PRINTF(LEVEL_ERROR, "Attempted to process RPC event from GPU%d: 0x%x (%s) during bootup without API lock\n",
...);
NV_ASSERT(0);
Why this handler is safe to run in the bootup-poll context
_kgspRpcEventPlatformRequestHandlerStateSyncCallback decodes the RPC payload and dispatches via pfmreqhndlrStateSync_IMPL on the flags field:
- For
PMGR_LOADandTHERM_INIT, the prereq callbacks only callosQueueWorkItemand return; they touch no shared state inline and make no GSP RPC. The in-tree comment at those callbacks ("we need this to succeed so that we don't have a recursive GSP RPC") guarantees the no-RPC property explicitly. - For
SMBPBI_OP_SET/SMBPBI_OP_CLEAR, the synchronous portion writes Platform Request Handler counter fields. The polling thread holds the GPU subdevice lock for the duration of the bootup poll, so no concurrent writer to that state can race against the dispatch.
In short: no recursive GSP RPCs, no unprotected shared-state writes. The handler is well-behaved under the constraints the bootup-poll whitelist is enforcing.
Suggested fix
diff --git a/src/nvidia/src/kernel/gpu/gsp/kernel_gsp.c b/src/nvidia/src/kernel/gpu/gsp/kernel_gsp.c
index ab1c3695..b3c36405 100644
--- a/src/nvidia/src/kernel/gpu/gsp/kernel_gsp.c
+++ b/src/nvidia/src/kernel/gpu/gsp/kernel_gsp.c
@@ -1440,6 +1440,7 @@ _kgspProcessRpcEvent
case NV_VGPU_MSG_EVENT_OS_ERROR_LOG:
case NV_VGPU_MSG_EVENT_GSP_LOAD_EXEC_GENERIC_BOOTLOADER:
case NV_VGPU_MSG_EVENT_GSP_LOAD_EXEC_HS_BINARY:
+ case NV_VGPU_MSG_EVENT_PFM_REQ_HNDLR_STATE_SYNC_CALLBACK:
break;
default:
NV_PRINTF(LEVEL_ERROR, "Attempted to process RPC event from GPU%d: 0x%x (%s) during bootup without API lock\n",
Verification
Built 595.71.05 with the diff applied, ~3 hours of normal use including many runtime-D3 wake cycles. Zero kernel_gsp.c:1447 / 0x101a lines after the patch, vs roughly two-thirds of wakes producing the pair before. No regressions observed.
Relation to issue #1064
This fix is not a fix for #1064. #1064 is a DIFR pmlock deadlock that wedges Mutter on nvkms_lock and requires SysRq REISUB to recover; the deadlock reproduces with this patch applied. The two are unrelated: the only connection is that #1064 was the path by which the missing whitelist entry got noticed. Tracking that separately on the #1064 thread.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in src/nvidia/src/kernel/gpu/gsp/kernel_gsp.c around _kgspProcessRpcEvent at line 1436 and inspect the bootup-poll whitelist and the PFM state-sync callback handler. Build the driver with the callback included in the whitelist, then exercise runtime-D3 wake cycles and confirm the 0x101a and kernel_gsp.c:1447 messages no longer appear.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c
- Domain
- operating-systems
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100