hello people I want to know what
sudo nvidia-smi --gpu-reset -i 0
this command do is it just free upt he memory of GPU and do nothing?
CodePudding user response:
Resets GPU state. Can be used to clear double bit ECC errors or recover hung GPU. Requires -i switch to target specific device. Available on Linux only. https://developer.download.nvidia.com/compute/DCGM/docs/nvidia-smi-367.38.pdf
CodePudding user response:
From the nvidia-smi
help menu (man nvidia-smi
):
-r, --gpu-reset
Trigger a reset of one or more GPUs. Can be used to clear GPU HW and SW state in situations that would otherwise require a machine reboot. Typically useful if a double bit ECC
error has occurred. Optional -i switch can be used to target one or more specific devices. Without this option, all GPUs are reset. Requires root. There can't be any appli‐
cations using these devices (e.g. CUDA application, graphics application like X server, monitoring application like other instance of nvidia-smi). There also can't be any com‐
pute applications running on any other GPU in the system.
Starting with the NVIDIA Ampere architecture, GPUs with NVLink connections can be individually reset. On NVSwitch systems, Fabric Manager is required to facilitate reset.
If Fabric Manager is not running, or if any of the GPUs being reset are based on an architecture preceding the NVIDIA Ampere architecture, any GPUs with NVLink connections to a
GPU being reset must also be reset in the same command. This can be done either by omitting the -i switch, or using the -i switch to specify the GPUs to be reset. If the -i
option does not specify a complete set of NVLink GPUs to reset, this command will issue an error identifying the additional GPUs that must be included in the reset command.
GPU reset is not guaranteed to work in all cases. It is not recommended for production environments at this time. In some situations there may be HW components on the board
that fail to revert back to an initial state following the reset request. This is more likely to be seen on Fermi-generation products vs. Kepler, and more likely to be seen if
the reset is being performed on a hung GPU.
Following a reset, it is recommended that the health of each reset GPU be verified before further use. If any GPU is not healthy a complete reset should be instigated by power
cycling the node.
GPU reset operation will not be supported on MIG enabled vGPU guests.
Visit http://developer.nvidia.com/gpu-deployment-kit to download the GDK.