3.2_Devices_and_Initialization
3.2 Devices and Initialization
Devices correspond to physical GPUs. When CUDA is initialized (either explicitly by calling the driver API's cuInit() function or implicitly by calling a CUDA runtime function), the CUDA driver enumerates the available devices and creates a global data structure that contains their names and immutable capabilities such as the amount of device memory and maximum clock rate.
For some platforms, NVIDIA includes a tool that can set policies with respect to specific devices. The nvidia-smi tool sets the policy with respect to a
given GPU. For example, nvidia-smi can be used to enable and disable ECC (error correction) on a given GPU. nvidia-smi also can be used to control the number of CUDA contexts that can be created on a given device. These are the possible modes.
Default: Multiple CUDA contexts may be created on the device.
"Exclusive" mode: One CUDA context may be created on the device.
"Prohibited": No CUDA context may be created on the device.
If a device is enumerated but you are not able to create a context on that device, it is likely the device is in "prohibited" mode or in "exclusive" mode and another CUDA context already has been created on that device.