[555.42.02] D3cold on Turing Mobile not working with kernel 6.9.2. Works with closed driver.

NVIDIA Open GPU Kernel Modules Version

550.78

Please confirm this issue does not happen with the proprietary driver (of the same version). This issue tracker is only for bugs specific to the open kernel driver.

• I confirm that this does not happen with the proprietary driver package.

Operating System and Version

Gentoo Linux x86_64 6.7.9-gentoo

Kernel Release

6.7.9-gentoo, own config

Please confirm you are running a stable release kernel (e.g. not a -rc). We do not accept bug reports for unreleased kernels.

• I am running on a stable kernel release.

Hardware: GPU

NVIDIA GeForce RTX 2070 with Max-Q Design

Describe the bug

I noticed my laptop was slightly warmer than expected. This on 6.8.9-gentoo.
A number of reboots later, I can state that :

dagb@gillette:~ (20:32) $ cat /proc/driver/nvidia/gpus/0000\:01\:00.0/power 
Runtime D3 status:          Not supported
Video Memory:               Active

GPU Hardware Support:
 Video Memory Self Refresh: Not Supported
 Video Memory Off:          Supported

... is the result, if the nvidia-drivers package is built with the kernel-open flag in gentoo, running gentoo-sources-6.7.9.

If built with -kernel-open (leading '-' implies 'no') I have fine-grained control again.

HOWEVER, please also note: I also tried both variants (open/closed kernel driver) on 6.8.9, and there I get 'Not supported' in both cases'.

I have not bisected the issue to a particular kernel version. I just happened to have 6.7.9 on disk.

To Reproduce

• run gentoo
• install gentoo-sources-6.7.9
• build/install kernel
• build install nvidia-driver:

[ebuild   R    ] x11-drivers/nvidia-drivers-550.78:0/550::gentoo  USE="X kernel-open modules static-libs strip tools wayland -dist-kernel -modules-compress -modules-sign -persistenced -powerd" ABI_X86="(64) -32" 0 KiB
- driver options:

blacklist nouveau
options nvidia-drm modeset=0
options nvidia NVreg_DeviceFileGID=27
options nvidia NVreg_DeviceFileMode=432
options nvidia NVreg_DeviceFileUID=0
options nvidia NVreg_ModifyDeviceFiles=1
options nvidia NVreg_TemporaryFilePath=/var/tmp
options nvidia NVreg_PreserveVideoMemoryAllocations=0
options nvidia NVreg_DynamicPowerManagement=0x02
options nvidia NVreg_EnableS0ixPowerManagement=1
options nvidia NVreg_UsePageAttributeTable=1
alias char-major-195 nvidia
alias /dev/nvidiactl char-major-195
remove nvidia modprobe -r --ignore-remove nvidia-drm nvidia-modeset nvidia-uvm nvidia

- udev rules:

dagb@gillette:~ (20:45) $ cat /etc/udev/rules.d/80-nvidia-pm.rules

Remove NVIDIA USB xHCI Host Controller devices, if present

ACTION=="add", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x0c0330", ATTR{remove}="1"

Remove NVIDIA USB Type-C UCSI devices, if present

ACTION=="add", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x0c8000", ATTR{remove}="1"

Remove NVIDIA Audio devices, if present

ACTION=="add", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x040300", ATTR{remove}="1"

Enable runtime PM for NVIDIA VGA/3D controller devices on driver bind

ACTION=="bind", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x030000", TEST=="power/control", ATTR{power/control}="auto"
ACTION=="bind", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x030200", TEST=="power/control", ATTR{power/control}="auto"

Disable runtime PM for NVIDIA VGA/3D controller devices on driver unbind

ACTION=="unbind", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x030000", TEST=="power/control", ATTR{power/control}="on"
ACTION=="unbind", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x030200", TEST=="power/control", ATTR{power/control}="on"

### Bug Incidence

Always

### nvidia-bug-report.log.gz

[nvidia-bug-report.log.gz](https://github.com/NVIDIA/open-gpu-kernel-modules/files/15287343/nvidia-bug-report.log.gz)


### More Info

I *think* 6.6.30 works with both open and closed kernel driver. Will give it another spin to verify, and update this ticket with the result.

Start with the attached nvidia-bug-report.log.gz and the reported /proc/driver/nvidia/gpus/0000:01:00.0/power output, comparing kernel-open and proprietary-driver behavior across the listed 6.7.9, 6.8.9, and 6.9.2 kernels. Reproduce the runtime power-management state with the supplied Gentoo build and module options; done means D3cold is supported and the GPU reaches the expected low-power state with the open driver.