Manually setting memory clocks destroys NVENC's performance

NVIDIA Open GPU Kernel Modules Version

580.95.05, but happened ever since I got the laptop 2/3 months ago

Please confirm this issue does not happen with the proprietary driver (of the same version). This issue tracker is only for bugs specific to the open kernel driver.

• I confirm that this does not happen with the proprietary driver package.

Operating System and Version

Arch Linux

Kernel Release

6.17.6-arch1-1

Please confirm you are running a stable release kernel (e.g. not a -rc). We do not accept bug reports for unreleased kernels.

• I am running on a stable kernel release.

Hardware: GPU

GeForce RTX 5070ti laptop

Describe the bug

When recording using NVENC after manually setting the memory clocks with for instance nvidia-smi -lmc 405,14001 and regarless of the given range, NVENC starts to massively lag.
For reference, I had this issue cross-desktops (Hyprland and GNOME), cross-systems (my old Arch and my new installation), cross-softwares (OBS and GPU-screen-recorder) and cross capture methods (Pipewire portal and OBS-vkcapture).
The following test worked on all three encoders and settings combination ; in my reported below test I used AV1 CQP20 medium without BFrames, look-ahead or adaptive quantization.
Without changing the memory clocks, recording in 60FPS gets me 18% NVENC usage and scales nicely to ~60% when highering to 240FPS.
After setting the memory clocks (whatever range, even the default full one), the exact same settings gets NVENC to 26% at 60FPS and jumps straight to 100% at 240FPS (70% of dropped frames according to OBS logs)
Surprisingly, this only happens when setting memory clocks, touching GPU clocks does no change a thing here.

To Reproduce

• make sure your memory clocks are absolute defaults using nvidia-smi -rmc
• encode something and enjoy your good performance while you can
• use nvidia-smi -lmc to set the memory clocks range (even the full range will be problematic)
• encode at very slow speed
• cry and then do not waste months dwelling the internet for nothing trying to fix it by every imaginable mean as resetting your memory clocks should get back things to normal awaiting a fix

Bug Incidence

Always

nvidia-bug-report.log.gz

nvidia-bug-report.sh will now collect information about your
system and create the file 'nvidia-bug-report.log.gz' in the current
directory. It may take several seconds to run. In some
cases, it may hang trying to capture data generated dynamically
by the Linux kernel and/or the NVIDIA kernel module. While
the bug report log file will be incomplete if this happens, it
may still contain enough data to diagnose your problem.

If nvidia-bug-report.sh hangs, consider running with the --safe-mode
and --extra-system-data command line arguments.

Please include the 'nvidia-bug-report.log.gz' log file when reporting
your bug via the NVIDIA Linux forum (see forums.developer.nvidia.com)
or by sending email to 'linux-bugs@nvidia.com'.

By delivering 'nvidia-bug-report.log.gz' to NVIDIA, you acknowledge
and agree that personal information may inadvertently be included in
the output. Notwithstanding the foregoing, NVIDIA will use the
output only for the purpose of investigating your reported issue.

Running nvidia-bug-report.sh... complete.

Summary of Skipped Sections:

Skipped Component | Details

mst output | mst not found

nvlsm-bug-report.sh output | nvlsm-bug-report.sh not found

Summary of Errors:

Error Component | Details | Resolution

nvidia-bug-report.log.gz

More Info

No response

Reproduce with the reported GeForce RTX 5070 Ti laptop setup using nvidia-smi -rmc and nvidia-smi -lmc, then compare NVENC recording performance at 60 and 240 FPS. Start by reviewing the open GPU kernel module handling of memory-clock limits and the attached nvidia-bug-report.log.gz. Done means setting any memory-clock range no longer causes the reported NVENC lag or dropped frames.