Cloudera Docs

Accelerating Cloudera Data Engineering Jobs and Sessions

You can accelerate your Spark jobs and Sessions using GPUs after you add GPU resources to the Cloudera Data Engineering service and virtual cluster.

Perform the following to use GPU resources:

  1. Create a Cloudera Data Engineering service with GPU quota.

  2. Create a virtual cluster in that Cloudera Data Engineering service with GPU quota.

  3. Enable GPU acceleration when creating Spark jobs using Cloudera Data Engineering UI or when running jobs using CDE CLI.

  4. Enable GPU acceleration when creating a session using Cloudera Data Engineering UI or CDE CLI.
  5. Check whether the Spark job or session ran using the GPU resources.