Repository navigation
questions about dynamic power state machine #731
Description
Thanks for your contributions to open source. We know that when there are no tasks on the GPU, it goes into an idle state, and during this time, if you check the GPU's bandwidth with lspci, it will show as Gen1. In the file dynamic_power.c, the mechanism of dynamic power states is thoroughly explained. Is this power management related to the idle state mentioned above?
Activity
mtijanic commentedon Nov 8, 2024
Hi there. There are two related but still distinct concepts when it comes to power management. The stuff in dynamic-power.c relates to RTD3/GC6 deep idle states. In these cases, the GPU is effectively powered off (except a few minor power islands). It certainly won't support anything above Gen1 in this state.
However, even if the GPU is not in a deep idle state, the PCIe link speed depends on the clocks/powerdraw of the GPU. For example, here is the difference when running the GPU in different pstates on my system:
(P0 is highest performance state, P8 is lowest on this GPU)
Hope this helps!
nju-zjx commentedon Nov 8, 2024
Thank you for the answer. I would like to ask if the dynamic adjustment of pstate is covered in the open-source driver code, such as P8/P0 's switch with the change of clocks/powerdraw. And if so, in which .c file can it be found?
mtijanic commentedon Nov 8, 2024
The actual pstates are changed by the PMU depending on the GPU load. This algorithm isn't part of these modules (nor the GSP code).
However, applications do have some tools available to instrument and modify these. This API is described here: https://github.com/NVIDIA/open-gpu-kernel-modules/blob/main/src/common/sdk/nvidia/inc/ctrl/ctrl2080/ctrl2080perf.h
Here is a very hacked up minimal example that uses the above API to set the pstate to P0 while the app is running;
https://gist.github.com/mtijanic/9c129900bfba774b39914ad11b0041f6
nju-zjx commentedon Nov 8, 2024
I will try it. Thanks a lot!
nju-zjx commentedon Nov 8, 2024
https://gist.github.com/mtijanic/9c129900bfba774b39914ad11b0041f6
Sorry, this websits can't open. Has it been transferred to another website
timur-tabi commentedon Nov 8, 2024
That URL is correct. You might need to log into github.com first.
nju-zjx commentedon Nov 11, 2024
Indeed, that's the case. I encountered the following issue during my attempt and look forward to an explanation!
The duration at P0 does not depend on NV2080_CTRL_PERF_BOOST_DURATION_MAX, but rather on the time of usleep, which is quite puzzling. Could you please explain the purpose of usleep and why the GPU reverts from P0 to P8 after the function ends?
mtijanic commentedon Nov 11, 2024
These perfboosts are tied to the 2080/subdevice objects under which they were made. Ignoring all the plumbing, in OOP pseudocode what that thing does is:
Once the
subdeviceobject gets destroyed, the perfboost request associated with it is cancelled. There are a few ways to destroy the subdevice object:NvRmFree(hSubdevice)- Explicitly tell the driver to delete this objectNvRmFree(hClient)(orNvRmFree(hDevice)) - Explicitly delete a parent object, which removes everything allocated underneath itclose(nvctl), which then implicitly does anNvRmFree()on all client objects that were allocated using that fd.What happens in the app is that when
usleep()finishes and the program terminates, the kernel will automatically close all open files, which triggers the (3) case above. You can use any other way to keep the program running, such as blocking on user input, polling on an fd, or even breaking into a debugger. So long as it doesn't terminate and the kernel doesn't reap the fd, it will run for the NV2080_CTRL_PERF_BOOST_DURATION_MAX.By the way, since this is not an actionable issue with the code, I will be converting this into a discussion thread.