Cloudera Docs

Creating a cluster from a definition on AWS

You can quickly create clusters from default or custom cluster definitions within an existing AWS environment.

To create a Cloudera Data Hub cluster on AWS, you must have an existing AWS environment. Also, you should make sure that the Runtime version of the Data Lake cluster matches the Cloudera Runtime version of the Cloudera Data Hub cluster that you are about to create; If these versions don't match, you may encounter warnings and/or errors.
Required role: DataHubCreator, EnvironmentAdmin at the scope of the environment where the Cloudera Data Hub cluster is running, or Owner of the environment
  1. Log in to the Cloudera web interface.
  2. Navigate to the Cloudera Management Console > Environments > click on an environment where you would like to create a cluster > click Create Data Hub. The following page is displayed:
  3. Select Cluster Definition.
  4. From the Cluster Definition dropdown, select the cluster definition that you would like to use for your cluster.

    The cluster template referenced in the selected cluster definition determines which services are included in the cluster. The list of services is automatically shown below the selected cluster definition name.

    You are able to select templates that run on ARM-64 processors. The ARM architecture is indicated in the name of the cluster definition, for example 7.3.1 - Data Engineering HA - Spark3 (ARM) for AWS.
  5. Specify General Settings for your cluster:
    Parameter Description
    Cluster Name Enter a name for your cluster. The name must be between 5 and 40 characters, must start with a letter, and must only include lowercase letters, numbers, and hyphens.
    Tags (Optional) Add tags that Data Hub should use to tag your AWS resources. Click Add to add a tag, and then enter a key and value for each tag. Repeat the steps if you would like to add more tags. For more information about tags, refer to Tags.
  6. Optionally, click on Advanced Options to modify advanced cluster settings. For more information on these options, refer to Advanced cluster options.
  7. On AWS and Azure only: Optionally, when you have finished providing the cluster settings, you can click the Show CLI Command button at the bottom of the page to review or copy the CDP CLI command used to create the cluster. You can copy the command from the pop-up window that appears, either to provision the cluster later or for use in scripts.
  8. You also have the option to review or copy the cluster template that is generated and will be used in cluster creation. Click the Show Generated Cluster Template button at the bottom of the page.
  9. To proceed with cluster provisioning immediately, click on Provision Cluster.
  10. You will be redirected to the Cloudera Data Hub cluster dashboard, and a new tile representing your cluster will appear at the top of the page.

    The following messages are written to the event history as your cluster is being created:

    Cluster built; Cluster manager ip:10.97.82.237
8/2/2019, 6:10:44 PM
Updating Cluster Proxy service with gateway configuration
8/2/2019, 6:10:43 PM
Building cluster; Cluster manager ip:10.97.82.237
8/2/2019, 5:45:12 PM
Starting cluster services
8/2/2019, 5:42:33 PM
Mounting attached disks
8/2/2019, 5:42:18 PM
Setting up infrastructure metadata
8/2/2019, 5:42:14 PM
Bootstrapping infrastructure cluster
8/2/2019, 5:41:59 PM
Registering cluster with Cluster Proxy service
8/2/2019, 5:41:57 PM
Infrastructure successfully provisioned
8/2/2019, 5:41:56 PM
Billing started, Infrastructure successfully provisioned
8/2/2019, 5:41:56 PM
Infrastructure metadata collection finished
8/2/2019, 5:41:55 PM
Infrastructure creation took 96 seconds
8/2/2019, 5:41:52 PM
Creating infrastructure
8/2/2019, 5:40:16 PM
Setting up CDP image
8/2/2019, 5:40:15 PM
  11. When your cluster is ready, its status will change to Running.
You can access links to Cloudera Manager, and other cluster UIs and endpoints from cluster details.