Updating the Control Plane certificates in Cloudera Data Engineering Virtual Clusters

Learn about how to update the Control Plane certificates in Cloudera Data Engineering Virtual Clusters.

note

If a third-party certificate manager is installed, it validates and updates the CA certificate and the TLS Certificate page will be disabled. For more information about the third-party certificate manager, see Integrating third-party certificate manager.
Cloudera Data Engineering 1.5.5 SP4 and later requires single-purpose Server Authentication EKU certificates. Uploading any other certificate type displays the following error:
```
TLS certificate is not EKU compliant. Upload one with Server Authentication only.
```
(From Cloudera Data Engineering 1.5.5 SP1 onwards) To enable Spark Connect sessions in OpenShift Container Platform clusters, upload a separate certificate. You can use the API to upload the certificate.

When you create a Cloudera Data Engineering Virtual Cluster, Cloudera Data Engineering automatically installs a self-signed TLS certificate. Perform the following steps to update the TLS certificate for Cloudera Data Engineering virtual clusters:

Cloudera Data Engineering UI
Cloudera Data Engineering API

In the Cloudera console, click the Data Engineering tile. The Cloudera Data Engineering Home page displays.
Click Administration in the left navigation menu. The Administration page displays.
In the Services column, select the service containing the virtual cluster for which you want to update the TLS certificate.
In the Virtual Clusters column on the right, click the Cluster Details icon for the virtual cluster for which you want to update the TLS certificate.
Click Actions > Copy Jobs API URL to copy the URL to your clipboard.
note
Currently in the case of ECS, the URL copied to your clipboard starts with http://, not https://. To use the URL, you must manually change this to https://. In the case of OCP, the link must start with https://.
Paste the URL into a text editor to identify the endpoint host. For example, the URL is similar to the following:
```
http://dfdj6kgx.cde-2cdxw5x5.apps.ecs-demo.example.com/dex/api/v1
```
The endpoint host is dfdj6kgx.cde-2cdxw5x5.apps.ecs-demo.example.com.
Create your CA certificate and a private key. Make sure that your CA certificate includes the Subject Alternative Names (SAN) for all relevant endpoints.
note
If you want to use a wildcard certificate, then make sure that includes the Subject Alternative Names (SAN) for all relevant endpoints.
- CN (Common Name): *.cde-2cdxw5x5.apps.ecs-demo.example.com
  note
  - cde-2cdxw5x5 is the name of your Virtual Cluster environment and each new Virtual Cluster environment that is created requires a new certificate.
  - The certificate name must be less than or equal to 64 characters long.
- Domain name for the Virtual Cluster: dfdj6kgx.cde-2cdxw5x5.apps.ecs-demo.example.com
Sign in to Cloudera Data Engineering as VC Admin. The Cloudera Data Engineering Home page displays.
Click Administration in the left navigation menu. The Administration page displays.
In the Services column, select the service containing the virtual cluster for which you want to update the TLS certificate.
In the Virtual Clusters column on the right, click the Cluster Details icon for the virtual cluster for which you want to update the TLS certificate.
On the Configuration tab, click TLS Certificate.
Click Upload Certificate.
Under PEM file, click Select File and select your TLS certificate.
Under Key file, click Select File and select your key file.
(Optional) If your key file is encrypted with a password, then enter the password in the Passphrase field.
Click Install Certificate.
Repeat the preceding steps for all Virtual Clusters.

Prerequisites:

Perform the following steps to update the TLS certificate for Cloudera Data Engineering virtual clusters:

Make sure that you are using the VCAdmin role for the corresponding Virtual Cluster and rotate Virtual Cluster TLS certificate.

cdpcurl -X PUT -k [***CDP-ENDPOINT***]/dex/api/v1/cluster/[***CLUSTER-ID***]/instance/[***VIRTUAL-CLUSTER-ID***]/tls' \
  -d '{
         "certificate_details": {
            "certificate": "[***BASE64-encoded.pem-FILE-CONTENT***]",
            "key": "[***BASE64-ENCODED-FILE-CONTENT***]",
            "passphrase": "[***BASE64-ENCODED-PASSPHRASE-FOR-THE-KEY***]" (optional) 
      }
}' \
    --access_key [***ACCESS-KEY***] \
    --private_key [***PRIVATE-KEY***]

Make sure that you are using the VCUser role for the corresponding Virtual Cluster and view the Virtual Cluster level TLS Certificate information.

cdpcurl -X GET  '[***CDP-ENDPOINT***]/dex/api/v1/cluster/[***CLUSTER-ID***]/instance/[***VIRTUAL-CLUSTER-ID***]/tls' \
  -d '{}' \
    --access_key [***ACCESS-KEY***] \
    --private_key [***PRIVATE-KEY***]

(From Cloudera Data Engineering 1.5.5 SP1 onwards) For Spark Connect sessions in the OpenShift Container Platform cluster, perform the following steps:
1. In the Cloudera console, click the Data Engineering tile. The Cloudera Data Engineering Home page is displayed.
2. Click Administration in the left navigation menu. The Administration page is displayed.
3. In the Services column, select the service containing the virtual cluster for which you want to update the TLS certificate.
4. In the Virtual Clusters column on the right, click the Cluster Details icon for the virtual cluster for which you want to update the TLS certificate.
5. Click Actions > Copy Jobs API URL to copy the URL to your clipboard.
  note
  The link must start with https://.
6. Paste the URL into a text editor to identify the endpoint host. For example, the URL is similar to the following:
```
https://dfdj6kgx.cde-2cdxw5x5.apps.ocp-demo.example.com/dex/api/v1
```
  The endpoint host for the Spark Connect session is sc-dfdj6kgx.cde-2cdxw5x5.apps.ocp-demo.example.com. Spark Connect endpoints are prefixed with sc-.
7. Create your CA certificate and a private key. Make sure that your CA certificate includes the Subject Alternative Names (SAN) for all relevant endpoints for the Spark Connect Endpoint host.
  note
  For the Spark Connect certificate in OpenShift Container Platform, wildcard certificates are not recommended. Ensure that the Subject Alternative Names (SAN) contain the Spark Connect Domain.
  Domain name for the Spark Connect virtual cluster: sc-dfdj6kgx.cde-2cdxw5x5.apps.ocp-demo.example.com
8. Make sure that you are using the VCAdmin role for the corresponding Virtual Cluster and rotate the Virtual Cluster TLS certificate.
```
cdpcurl -X PUT -k [***CDP-ENDPOINT***]/dex/api/v1/cluster/[***CLUSTER-ID***]/instance/[***VIRTUAL-CLUSTER-ID***]/tls' \
  -d '{
         "certificate_details": {
            "certificate": "[***BASE64-encoded.pem-FILE-CONTENT***]",
            "key": "[***BASE64-ENCODED-FILE-CONTENT***]",
            "passphrase": "[***BASE64-ENCODED-PASSPHRASE-FOR-THE-KEY***]" (optional) 
            “service-type”: “spark-connect”
      }
}' \
    --access_key [***ACCESS-KEY***] \
    --private_key [***PRIVATE-KEY***]
```
  important
  - To generate an API access key, see Generating an API access key.
  - For more information about the cdpcurl command, see the cdpcurl page.
  - Use -k in the command only if your Cloudera Data Services on premises certificate is not trusted on your machine.
9. Make sure that you are using the VCUser role for the corresponding Virtual Cluster and view the Virtual Cluster-level TLS Certificate information.
```
cdpcurl -X GET  '[***CDP-ENDPOINT***]/dex/api/v1/cluster/[***CLUSTER-ID***]/instance/[***VIRTUAL-CLUSTER-ID***]/tls?service-type=spark-connect' \
  -d '{}' \
    --access_key [***ACCESS-KEY***] \
    --private_key [***PRIVATE-KEY***]
```