Adding a Cloudera Data Engineering service

Before you can use the Cloudera Data Engineering (CDE) service, you must add the service to an environment that you want to use CDE on.

Make sure that you have a working environment for which you want to enable the CDE service. For more information about environments, see Environments.

Cloudera recommends adding only one CDE service per environment. For different workload requirements, you can create as many virtual clusters as you need within the same CDE service. Each virtual cluster can have different resource allocations to address different workload requirements.

For example, you can create one virtual cluster with a large amount of CPU cores and less memory for CPU-intensive workloads, while another virtual cluster is allocated more memory for I/O-bound workloads.

Rather than creating multiple CDE services for these differing workload profiles, Cloudera recommends using a single CDE service, and controlling the resource allocation at the virtual cluster level.

  1. Navigate to the Cloudera Data Engineering Overview page by clicking the Data Engineering tile in the Cloudera Data Platform (CDP) management console.
  2. In the Environments column, click the plus icon at the top or the Enable new CDE link at the bottom to enable CDE for an environment.
  3. Enter a Name for the CDE service you are creating. In the Environment text box, select or type the name of the environment that you want to enable CDE for. The displayed list dynamically updates to show environment names matching your input. When you see the correct environment, click on it to select it.
  4. Click Enable.
The CDE Overview page displays the status of the CDE service initialization. You can view logs for the service by clicking on the service vertical ellipsis (three dots) menu, and then clicking View Logs.