We have set up a new headless server with ubuntu 18.04 no-GUI version.
We installed the latest nvidia driver via apt install nvidia-headless-510-server. After that we also installed nvidia-utils-510 so that we could monitor GPU usage via nvidia-smi.
But on this new server, the GPU fan speed is higher than usual, and the GPU utility is not zero despite no job is running.
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.47.03 Driver Version: 510.47.03 CUDA Version: 11.6 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce ... Off | 00000000:01:00.0 Off | N/A |
| 41% 57C P0 107W / 350W | 0MiB / 24576MiB | 3% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 1 NVIDIA TITAN Xp... Off | 00000000:21:00.0 Off | N/A |
| 25% 37C P0 59W / 250W | 0MiB / 12288MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+What did we do wrong? What is using the GPU constantly?
3 Reset to default
