Using GPUs for Cloudera AI projects

A GPU is a specialized processor that can be used to accelerate highly parallelized computationally-intensive workloads. Because of their computational power, GPUs have been found to be particularly well-suited to deep learning workloads. Ideally, CPUs and GPUs are used in tandem for data engineering and data science workloads. A typical machine learning workflow involves data preparation, model training, model scoring, and model fitting. You can use existing general-purpose CPUs for each stage of the workflow, and optionally accelerate the math-intensive steps with the selective application of special-purpose GPUs. For example, GPUs allow you to accelerate model fitting using frameworks such as Tensorflow, PyTorch, and Keras.

By enabling GPU support, data scientists can share GPU resources available on Cloudera AI Workbenches. Users can request a specific number of GPU instances, up to the total number available, which are then allocated to the running session or job for the duration of the run.

Enabling GPUs on Cloudera AI Workbenches

If you are using ML Runtimes, you must use the ML Runtimes version for your Python library Nvidia GPU Edition.

If you are using a Legacy Engine, to enable GPU usage on Cloudera AI, select GPUs when you are provisioning the workbench. If your existing workbench does not have GPUs provisioned, contact your Cloudera AI administrator to provision a new one for you. For instructions, see Provisioning Cloudera AI Workbenches.