Options to run distributed shell and GPU
You can run the distributed shell by specifying resources other than memory and vCores.
Use the following command to run the distributed shell and GPU without a
Docker container:
yarn jar </opt/cloudera/parcels/<CDH-version>/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell.jar> \
-jar </opt/cloudera/parcels/<CDH-version>/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell.jar> \
-shell_command /usr/local/nvidia/bin/nvidia-smi \
-container_resources memory-mb=3072,vcores=1,yarn.io/gpu=2 \
-num_containers 2You
receive
output similar to the
following:
+-----------------------------------------------------------------------------+ | NVIDIA-SMI 375.66 Driver Version: 375.66 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | |===============================+======================+======================| | 0 Tesla P100-PCIE... Off | 0000:04:00.0 Off | 0 | | N/A 30C P0 24W / 250W | 0MiB / 12193MiB | 0% Default | +-------------------------------+----------------------+----------------------+ | 1 Tesla P100-PCIE... Off | 0000:82:00.0 Off | 0 | | N/A 34C P0 25W / 250W | 0MiB / 12193MiB | 0% Default | +-------------------------------+----------------------+----------------------+ +-----------------------------------------------------------------------------+ | Processes: GPU Memory | | GPU PID Type Process name Usage | |=============================================================================| | No running processes found | +-----------------------------------------------------------------------------+
