Advanced Setup: Installing Cloudera Manager and CDH on AWS

This section describes how to create a cluster using the advanced setup procedure in the Altus Director UI. For the simple setup procedure, see Simple Setup: Creating a Cluster on AWS with Default Settings.

To install a Cloudera Manager deployment and launch a CDH cluster:

Add an Altus Director environment, as described in Adding an Altus Director Environment on AWS.
Open a web browser and go to the private IP address of the instance you created in Launching an EC2 Instance for Altus Director. Include port 7189 in the address. For example:
```
http://192.0.2.0:7189
```
In the Altus Director login screen, enter admin in both the Username and the Password fields.
In the Altus Director Welcome screen, click Let's get started.
This opens a wizard for adding an environment, Cloudera Manager, and a CDH cluster.
Click Continue to add Cloudera Manager.
In the Add Cloudera Manager screen:
1. Enter a name for this deployment of Cloudera Manager in the Cloudera Manager name field.
2. In the Instance Template field, click Select a Template if you already have one that you want to use, otherwise, click Create New Instance Template.
  The Create New Instance Template modal screen displays.
In the Create New Instance Template modal screen:
1. In the Instance Template name field, enter a name for the template.
2. In the Instance type field, select m4.large or m4.xlarge.
3. In the Image (AMI) ID field, enter the ID for the Amazon machine image (AMI) you chose in Launching an EC2 Instance for Altus Director, or find another AMI with a supported operating system.
  Note: To reduce cluster bootstrap times, you can preload the AMIs you use with Cloudera Manager and CDH. For more information, see Using Custom Repositories with Cloudera Manager and CDH.
4. In the Tags field, add one or more tags to associate with the instance.
  Note: Altus Director uses the Name field on AWS. By default, Altus Director puts the following values in the Name field: Altus Director id, Altus Director template name. If you want to create a custom key value for this field in place of "Name," see Configuring Altus Director to Use Custom Tag Names on AWS.
5. In the Security group IDs field, enter the security group ID you set up in Creating a New Security Group.
6. In the VPC subnet ID field, enter the ID of the VPC subnet that was created during VPC setup.
7. Click Advanced Options if you want to specify additional values for optional features, such as EBS, IAM, Spot instances, Auto Scaling groups, and AWS user data.
  Note: With Altus Director 2.4 and higher, you can enter opaque data or scripts in the User data field of the instance template, and this data will be passed to the instances as they are launched, using the EC2 user data mechanism. Note that AWS requires user data to be base64-encoded. You must perform the base64-encoding manually. For more information, including examples, see Instance Metadata and User Data in the AWS documentation.
8. Click Save changes.
In the Desired License Type field, select one of the following license types:
- Cloudera Enterprise: includes the core CDH services (HDFS, Hive, Hue, MapReduce, Oozie, Sqoop 1, YARN, and ZooKeeper) and, depending on the license edition, one or more additional services (Accumulo, HBase, Impala, Navigator, Solr, Spark). For more information on Cloudera Enterprise licenses, see Managing Licenses in the Cloudera Manager documentation.
- Cloudera Enterprise Trial: a 60-day trial license that includes all CDH services.
- Cloudera Express: no license required.
To enable usage-based billing, you must have a Cloudera Enterprise license and a billing ID provided by Cloudera. Perform these steps in the Add Cloudera Manager screen:
1. In the Desired License Type field, select Cloudera Enterprise.
2. In the License Key field, either select a Cloudera Enterprise license file to upload or select Direct Input and input the license file text directly into the text area.
3. To enable usage-based billing, enter the billing ID provided to you by Cloudera in the Billing ID field.
In the Database Server section, the default selection is Embedded Database. This installs an embedded PostgreSQL database for Cloudera Manager. The embedded PostgreSQL database should be used only when creating a demonstration or proof-of-concept deployment. It is not recommended for production. If desired, select Create Database Server Instance or Register Existing Database Server from the dropdown list, instead of Embedded Database:

For information about using an external database for Altus Director server, see the following pages:
- Using MySQL for Altus Director Server
- Using MariaDB for Altus Director Server
In the Add Cloudera Manager screen, click Cloudera Manager Configurations.
In the Cloudera Manager Configurations modal screen, set the heap size:
1. In the Scope field, select Host Monitor and add firehose_heapsize and 1073741824 in the respective Name and Value fields.
2. Click +.
3. In the Scope field, select Service Monitor and add firehose_heapsize and 1073741824 in the respective Name and Value fields.
4. Click Save Changes.

By default, the version of Cloudera Manager installed depends on the version of Altus Director you are using:

Altus Director version	Cloudera Manager version installed
Altus Director 2.0	Latest released version of Cloudera Manager 5.5
Altus Director 2.1	Latest released version of Cloudera Manager 5.7
Altus Director 2.2	Latest released version of Cloudera Manager 5.8
Altus Director 2.3	Latest released version of Cloudera Manager 5.10
Altus Director 2.4	Latest released version of Cloudera Manager 5.11
Altus Director 2.5	Latest released version of Cloudera Manager 5.12
Altus Director 2.6	Latest released version of Cloudera Manager 5.13
Altus Director 2.7	Latest released version of Cloudera Manager 5.14
Altus Director 2.8	Latest released version of Cloudera Manager 5.15
Altus Director 6.0	Latest released version of Cloudera Manager 6.0

To install a version of Cloudera Manager higher or lower than the default version, perform the following steps:

In the Configurations section, check Override default Cloudera Manager repository.
In the Repository URL field, enter the repository URL for the version of Cloudera Manager to install. Repository URLs for versions of Cloudera Manager 5 have the form https://archive.cloudera.com/cm5/ followed by the operating system, operating system major version, processor architecture, cm (for Cloudera Manager), and the Cloudera Manager major, minor, and (if applicable) maintenance release number. For example, the repository URL for Cloudera Manager 5.5.4 on any supported version of RHEL 7 is https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.5.4/.
Note: The Cloudera Manager minor version must the same as or higher than the CDH minor version. For example, Cloudera Manager 5.5 cannot be used to launch or manage a CDH 5.7 cluster, but Cloudera Manager 5.7 can be used with a CDH 5.7 (or lower) cluster.
In the Repository Key URL field, enter the URL for the repository key. Repository key URLs have the same form as repository URLs except they end with the name of the key file instead of the Cloudera Manager version. For example, the repository key URL for any version of Cloudera Manager 5 on any supported version of RHEL 7 is https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/RPM-GPG-KEY-cloudera.

In the Add Cloudera Manager screen, click Continue.
At the Confirmation prompt, click OK to begin adding a cluster.

On the Add Cluster screen:

Enter a name for the cluster in the Cluster name field.

Enter the version of CDH to deploy in the Version field or leave the default value. By default, the version of CDH installed depends on the version of Altus Director you are using:

Altus Director version	CDH version installed
Altus Director 2.0	Latest released version of CDH 5.5
Altus Director 2.1	Latest released version of CDH 5.7
Altus Director 2.2	Latest released version of CDH 5.9
Altus Director 2.3	Latest released version of CDH 5.10
Altus Director 2.4	Latest released version of CDH 5.11
Altus Director 2.5	Latest released version of CDH 5.12
Altus Director 2.6	Latest released version of CDH 5.13
Altus Director 2.7	Latest released version of CDH 5.14
Altus Director 2.8	Latest released version of CDH 5.15
Altus Director 6.0	Latest released version of CDH 6.0

To install a version of CDH higher or lower than the default version, perform the following steps:

Enter the desired CDH version in the Version field of the Products section. For example, for CDH 5.4.8 enter 5.4.8.
Scroll down to Configurations (optional) and expand the section.
Click Override default parcel repositories.
Enter the repository parcel URL for the version of CDH you want to install. Parcel URLs for versions of CDH 5 have the form https://archive.cloudera.com/cdh5/parcels/, followed by the major, minor, and (if applicable) maintenance release number. For example, the URL for CDH 5.4.8 is https://archive.cloudera.com/cdh5/parcels/5.4.8.
Note: The CDH minor version must not be higher than the Cloudera Manager minor version. For example, CDH 5.7 will not work with Cloudera Manager 5.5, but CDH 5.7 (or lower) will work with Cloudera Manager 5.7.

In the Services section, select the services you want to install.
In the Instance groups area, choose an existing instance template or create a new one, either for the all instance groups in the cluster, or for each group. For each instance group, indicate the number of instances you want.
Note: To reduce cluster bootstrap times, you can preload the AMIs you use for your cluster instance groups. For more information, see Using Custom Repositories with Cloudera Manager and CDH.
If you want to use Spot instances for your workers group:
1. In the Create New Instance Template modal screen, click Advanced Options.
2. In the Spot bid (USD/hr) field, enter your Spot bid price.
3. Click the Use Spot instances checkbox.
4. Click Save Changes.
  For more information about using Spot instances with Altus Director, see Using Spot Instances.

Click Continue.
At the Confirmation prompt, click OK to deploy the cluster. Altus Director displays a status screen.
When the cluster is ready, click Continue.
You are finished with the deployment tasks.

Simple Setup: Creating a Cluster on AWS with Default Settings

Pausing a Cluster on AWS