Runtime D3 (RTD3) status Not supported on Ubuntu 24.04

[jobs-git]

NVIDIA Open GPU Kernel Modules Version

550/560/565

Please confirm this issue does not happen with the proprietary driver (of the same version). This issue tracker is only for bugs specific to the open kernel driver.

• I confirm that this does not happen with the proprietary driver package.

Operating System and Version

Ubuntu 24.04.1 LTS
Ubuntu 24.04.2 LTS

Kernel Release

Linux 6.8.4
Linux 6.8.0
Linux 6.8.12-060812-generic #202405300722 SMP PREEMPT_DYNAMIC Thu May 30 11:55:07 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Linux 6.8.0-53

Please confirm you are running a stable release kernel (e.g. not a -rc). We do not accept bug reports for unreleased kernels.

• I am running on a stable kernel release.

Hardware: GPU

RTX 4070 Super
Ryzen 5 7600
AM5 B650i

Describe the bug

As the title says

To Reproduce

Enable Hybrid GPU
Attach Monitor to iGPU

Bug Incidence

Always

nvidia-bug-report.log.gz

nvidia-bug-report.log.gz

More Info

No response

[Ristovski]

Can confirm the same on Asus B450M-A II with Ryzen 5700G and RTX 4070 Ti SUPER under Wayland, with 6.12.

When loading the nvidia driver with NVreg_RmMsg=":^*", it prints RTD3 ACPI support is not available, and the status is:

$ cat /proc/driver/nvidia/gpus/0000:01:00.0/power
Runtime D3 status:          Disabled by default
Video Memory:               Active

GPU Hardware Support:
 Video Memory Self Refresh: Not Supported
 Video Memory Off:          Supported

S0ix Power Management:
 Platform Support:          Not Supported
 Status:                    Disabled

Notebook Dynamic Boost:     Not Supported

I understand RTD3 is mainly for mobile devices, but does the nvidia driver not support suspending the card when not in use at all?

FWIW I too am running a configuration where there are no active video outputs connected to the GPU, and despite there being no clients actively using the GPU, it will still idle at either 3 or 10W at P8:

nvidia-smi
Sun Feb 23 15:27:47 2025       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 570.86.16              Driver Version: 570.86.16      CUDA Version: 12.8     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4070 ...    Off |   00000000:01:00.0 Off |                  N/A |
|  0%   44C    P8             10W /  150W |       2MiB /  16376MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
                                                                                         
+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

I also tested setting DynamicPowerManagement to 0x2/0x3, DynamicPowerManagementVideoMemoryThreshold to 0 and PreserveVideoMemoryAllocations to 0, with seemingly no effect (and with no clients using the GPU, these shouldn't have any effect anyway, afaik).

The 3 vs 10W idle power usage is also weird - it appears completely random but quite frequently it will get "stuck" at 10W and the difference in temperature (with fans off) becomes apparent as well - 10W idle running up to ~10C hotter for me. A S3/"deep" suspend/resume (or perhaps just a prolonged period of absolutely no use) usually fixes it and it will then usually drop back to 3W idle.

@aaronp24 / @mtijanic: Sorry for the ping, but seeing as there are multiple similar issues here and on various distro forums, can you confirm whether GCOFF (or w/e the nvidia terminology for suspending the card to a state where its using 0W) is supported on desktops? And if so, what are the exact requirements (in this case, no video outputs, no active clients still doesn't do it).

FWIW, booting to a TTY has no effect and the card will still remain "idle" at P8. I even tried disabling modesetting and fbdev, but this had absolutely no effect either.

TL;DR: Shutting off nvidia card on desktops so it uses 0W possible, or not?

[jobs-git]

The PreserveVideoMemoryAllocations should not be set to zero otherwise we CUDA and Rendering applications will lose access to GPU on system resume. So when RTD3 will be fixed it should also allow PreserveVideoMemoryAllocations=1, so shutting down of GPU power should only happen when GPU is not in used but it should be able to return back those vram to idle apps.