NVIDIA / NVIDIA/open-gpu-kernel-modules
GB206M (RTX 5070 Laptop): GSP-RM init regression between 570.172.08 and 570.190
Nobody has claimed this yet.
- Dominant language
- C
- Stars
- 17.4k
- Forks
- 1.9k
- PR merge metrics
- No merged PRs in 30d
Description
NVIDIA Open GPU Kernel Modules Version
595.84 for the attached log. The bug is present in 570.190, 570.195.03, 570.211.01, 575.64, 580.173.02, 595.84, 610.43.03 and 610.57.04. It is absent in 570.169 and 570.172.08.
Please confirm this issue does not happen with the proprietary driver (of the same version)
N/A. GB206M requires the open kernel modules. The proprietary module loads, then refuses the device:
NVRM: The NVIDIA GPU 0000:01:00.0 (PCI ID: 10de:2d58)
NVRM: installed in this system requires use of the NVIDIA open kernel modules.
NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x22:0x56:1050)
Operating System and Version
NixOS 26.11 (Zokor)
Kernel Release
Linux 6.12.95 #1-NixOS SMP PREEMPT_DYNAMIC x86_64 for the bisect. The attached log is from 7.1.3. Both fail. The bisect ran entirely on 6.12.95, so the kernel is a controlled variable.
Please confirm you are running a stable release kernel
Yes. 6.12.95 and 7.1.3 are both stable releases.
Hardware: GPU
nvidia-smi -L reports No devices found on an affected driver. On 570.169 the same machine reports:
GPU 0: NVIDIA GeForce RTX 5070 Laptop GPU (UUID: GPU-b3c9b96b-77b1-c6cc-702f-5ab0437b719f)
lspci:
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GB206M
[GeForce RTX 5070 Max-Q / Mobile] [10de:2d58] (rev a1)
Host: ASUS ROG Strix G16 G615JPR, BIOS G615JPR.318 (2026-05-20), i9-14900HX. The Intel UHD iGPU drives the internal panel. The dGPU owns the HDMI port, so system firmware initialises it before the driver loads.
Describe the bug
The dGPU does not initialise. GSP-RM boots and then rejects its own initialisation:
NVRM: nvAssertOkFailedNoLog: Assertion failed: Invalid argument to call
[NV_ERR_INVALID_ARGUMENT] (0x1F) returned from RPC_HDR->rpc_result
@ kernel_gsp.c:4811
NVRM: ... returned from kgspWaitForRmInitDone(pGpu, pKernelGsp) @ kernel_gsp_gh100.c:928
NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x1f:1861)
This is a regression, and there is a known-good version. Ten driver versions were tested on this machine:
| driver | result |
|---|---|
| 570.169 | works |
| 570.172.08 | works |
| 570.190 | fails |
| 570.195.03 | fails |
| 570.211.01 (newest r570) | fails |
| 575.64 | fails, NV_ERR_INVALID_STATE 0x62:0x40:1941 |
| 580.173.02 | fails |
| 595.84 | fails |
| 610.43.03 | fails |
| 610.57.04 (newest shipped) | fails |
Three findings narrow this to the GSP-RM firmware image.
1. The host sends byte-identical data across the regression. Both drivers were instrumented to hexdump every structure they hand GSP-RM before GSP_INIT_DONE:
| payload | size | result |
|---|---|---|
GspSystemInfo |
936 B | identical except 2 per-boot values |
| packed registry table | 1383 B | byte-identical |
GSP_ARGUMENTS_CACHED |
72 B | identical except sharedMemPhysAddr |
GspFwWprMeta |
256 B | identical except DMA addresses and sizeOfRadix3Elf |
FSP NVDM_PAYLOAD_COT |
860 B | identical except 2 DMA addresses |
| libos region table | 6 entries | identical id8 tags and order |
Every difference is a per-boot physical address, or sizeOfRadix3Elf, which is derived from the firmware image. The per-boot values were confirmed as noise by a control: two RmInitAdapter attempts in one boot of one driver differ at exactly the same offsets.
2. Swapping only the firmware image fixes it. 570.190's driver with 570.172.08's gsp_ga10x.bin initialises the GPU. CUDA works. nvidia-smi lists the device. 570.211.01 also works with that image.
To do this, _kgspFwContainerVerifyVersion at kernel_gsp.c:4383 must be relaxed. It rejects the image with:
_kgspFwContainerVerifyVersion: GSP firmware image version mismatch:
got version 570.172.08, expected version 570.190
That check is a portStringCompare against NV_VERSION_STRING on the ELF .fwversion section.
3. The transplant does not cross a branch. 610.43.03 with the 570.172.08 image fails with 0x62:0x55:2119. GspSystemInfo gained fields between r570 and r610, so this looks like an RPC ABI mismatch.
What this does not claim. It does not show that the 570.190 image is defective on its own. It shows the image is the variable. A defect in the image, and a disagreement between two correct components, both fit. The test that separates them is not available from outside: nova-core cannot host NVIDIA-packaged firmware, and fails before GSP-RM runs on every triple tried. It also does not claim the driver is behaviourally identical. Six payloads were compared. Register writes, timing, and FSP handshake ordering were not.
This is not NVBug 5287221. That fix shipped in 570.169 and 575.64 (#876). Every driver tested here is newer than 570.169.
To Reproduce
- Use a laptop with a GB206M dGPU wired to a display output, so system firmware initialises it at power-on.
- Install nvidia-open 570.190 or later. Any release from 570.190 to 610.57.04 reproduces it.
- Boot. Load the full module stack:
nvidia,nvidia_modeset,nvidia_uvm,nvidia_drm. - Open the device, for example with
nvidia-smi -L.RmInitAdapterruns on first open, not at probe. - Read
dmesgfrom the top. Do not read the tail. The first attempt showsNV_ERR_INVALID_ARGUMENT. Later retries showunexpected WPR2 already up, which is an artifact of the first failure.NVreg_RegistryDwords=RmGspBootRetryAttempts=1removes the retries. - To confirm it is a regression, install 570.169 or 570.172.08. The GPU initialises.
No power cycle, reset, or state preparation is needed.
Bug Incidence
Always
nvidia-bug-report.log.gz
Captured on 595.84 / kernel 7.1.3 while the bug reproduces.
nvidia-bug-report-595.84-failing.log.gz
More Info
The request: bisect the GSP-RM image between 570.172.08 and 570.190 on GB206M. Both releases are public. The driver source across that window was also reverted in groups, with no effect: the 14-file RPC/GSP/crashcat cluster, the 7-file S0ix feature, and the RPC sequence match added in 570.190 were each reverted on 570.190 and the GPU still failed.
Two other machines report GB206M failing to initialise, with different signatures. #1328 fails earlier, at PlatformRequestHandler, so it should not be merged with this. forum 380605 reports 0x22:0x56:897 on an Acer with the same PCI ID.
Full write-up, including the byte dumps, the patch used for the transplant, and every negative result: https://gist.github.com/YuanYuYuan/1b3c7290a3dbbf3029a326e1ad7ff644
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with kernel_gsp.c, especially _kgspFwContainerVerifyVersion and the GSP initialization path around kgspWaitForRmInitDone. Reproduce on a GB206M using 570.172.08 and 570.190 or later, then bisect the GSP-RM firmware image between those releases while checking dmesg for the first initialization failure. Done means identifying the regression point or narrowing it to a specific firmware change.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, linux
- Domain
- operating-systems, testing-qa
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100