Cloudera Docs

Create a custom cluster on AWS

Create a custom Data Hub cluster within an existing AWS environment.

To create a Data Hub cluster on AWS, you must have an existing AWS environment. Also, you should make sure that the Runtime version of the Data Lake cluster matches the Runtime version of the Data Hub cluster that you are about to create; If these versions don't match, you may encounter warnings and/or errors.

  1. Log in to the CDP web interface.
  2. Navigate to the Management Console > Environments > click on an environment > click Create Data Hub.
  3. Under Selected Environment, confirm that the selected environment is the one where you would like to create your cluster.
  4. Select Custom.
  5. Under Platform Version, current Cloudera Runtime version is pre-selected.
  6. Under Cluster template, select the cluster template that you would like to use.
    The selected cluster template determines which services are included in the cluster. Select the cluster template and the list of services is automatically shown below it:
    For more information about cluster templates, refer to Cluster templates.
  7. Specify General Settings for your cluster:
    Parameter Description
    Cluster Name Enter a name for your cluster. The name must be between 5 and 40 characters, must start with a letter, and must only include lowercase letters, numbers, and hyphens.
    Tags (Optional) Add tags that Data Hub should use to tag your AWS resources. Click Add to add a tag, and then enter a key and value for each tag. Repeat the steps if you would like to add more tags. For more information about tags, refer to Tags.
  8. Optionally, click on Advanced Options to modify advanced cluster settings. For more information on these options, refer to LINK.
  9. Once done, click on Provision Cluster.
  10. You will be redirected to the Data Hub cluster dashboard, and a new tile representing your cluster will appear at the top of the page.

    The following messages are written to the event history as your cluster is being created:

    Cluster built; Cluster manager ip:10.97.82.237
8/2/2019, 6:10:44 PM
Updating Cluster Proxy service with gateway configuration
8/2/2019, 6:10:43 PM
Building cluster; Cluster manager ip:10.97.82.237
8/2/2019, 5:45:12 PM
Starting cluster services
8/2/2019, 5:42:33 PM
Mounting attached disks
8/2/2019, 5:42:18 PM
Setting up infrastructure metadata
8/2/2019, 5:42:14 PM
Bootstrapping infrastructure cluster
8/2/2019, 5:41:59 PM
Registering cluster with Cluster Proxy service
8/2/2019, 5:41:57 PM
Infrastructure successfully provisioned
8/2/2019, 5:41:56 PM
Billing started, Infrastructure successfully provisioned
8/2/2019, 5:41:56 PM
Infrastructure metadata collection finished
8/2/2019, 5:41:55 PM
Infrastructure creation took 96 seconds
8/2/2019, 5:41:52 PM
Creating infrastructure
8/2/2019, 5:40:16 PM
Setting up CDP image
8/2/2019, 5:40:15 PM
  11. When your cluster is ready, its status will change to Running.
You can access links to Cloudera Manager, and other cluster UIs and endpoints from cluster details.