Installing CDS 3.1 for GPUs

CDS 3.1 for GPUs replaces the CDS 3 Powered by Apache Spark parcel and service. If you have already installed CDS 3 Powered by Apache Spark, remove it before continuing. For instructions on removing parcels, see Deactivate and Remove Parcels.

Follow these steps to install CDS 3.1 for GPUs:

1. Check that all the software prerequisites are satisfied. If not, you might need to upgrade or install other software components first.
2. Install the CDS 3.1 for GPUs service descriptor into Cloudera Manager.
   1. To download the CDS 3.1 for GPUs service descriptor, click the service descriptor link for the version you want to install.
   2. Log on to the Cloudera Manager Server host, and copy the CDS 3.1 for GPUs service descriptor to the location configured for service descriptor files (/opt/cloudera/csd by default).
   3. Set the file ownership of the service descriptor to cloudera-scm:cloudera-scm with permission 644.
   4. Restart the Cloudera Manager Server with the following command:

      systemctl restart cloudera-scm-server
      

3. In the Cloudera Manager Admin Console, add the CDS 3.1 for GPUs parcel repository to the Remote Parcel Repository URLs in Parcel Settings as described in Parcel Configuration Settings.
4. Download the CDS 3.1 for GPUs parcel, distribute the parcel to the hosts in your cluster, and activate the parcel. For instructions, see Managing Parcels.
5. Add the CDS 3.1 for GPUs service to your cluster, following the procedure documented in Adding a Service.
   1. In the Add a Service wizard, specify any optional dependencies, such as HBase and Hive, or select No Optional Dependencies.
   2. When customizing the role assignments, add a gateway role to every host.
   3. On the Review Changes page, you can enable TLS for the Spark History Server.
   4. Note that the History Server port is 18089 instead of the usual 18088.
   5. Complete the remaining steps in the wizard.
6. Return to the Home page by clicking the Cloudera Manager logo in the upper left corner.
7. Click the stale configuration icon to launch the Stale Configuration wizard and restart the necessary services.

CDS 3.1 for GPUs supports Apache Livy, but it cannot use the included Livy service, which is compatible with only Spark 2. To add and manage a Livy service compatible with Spark 3, you must install a service descriptor for the Livy for Spark 3 service.

1. Install the Livy for Spark 3 service descriptor into Cloudera Manager.
   1. To download the service descriptor, click the Livy for Spark 3 service descriptor link for the version you want to install.
   2. Log on to the Cloudera Manager Server host, and copy the Livy service descriptor to the location configured for service descriptor files (/opt/cloudera/csd by default).
   3. Set the file ownership of the service descriptor to cloudera-scm:cloudera-scm with permissions set to 644.
   4. Restart the Cloudera Manager Server with the following command:

      systemctl restart cloudera-scm-server
      

2. Add the Livy for Spark 3 service to your cluster, following the procedure documented in Adding a Service.
3. Return to the Home page by clicking the Cloudera Manager logo in the upper left corner.
4. Click the stale configuration icon to launch the Stale Configuration wizard and restart the necessary services.