Provisioning a Cluster on AWS

The configuration file contains information Cloudera Director needs to operate and settings that define your cluster. The Cloudera Director configuration file is in HOCON format. For information on HOCON, see the documentation at https://github.com/typesafehub/config/blob/master/HOCON.md.

Sample configuration files are found either in /usr/lib64/cloudera-director/client or /usr/lib/cloudera-director/client, depending on the operating system you are using. Copy the sample files to your home directory before editing them.

To modify the configuration file:

  1. Rename the aws.simple.conf file to cluster.conf. For advanced cluster configuration, use aws.reference.conf.
  2. Open cluster.conf with a text editor.
  3. Configure the basic settings:
    • name - change to something that makes the cluster easy to identify.
    • id - leave this set to aws.
    • accessKeyId - AWS access key ID. Make sure the value is enclosed in double quotes.
    • secretAccessKey - AWS secret access key. Make sure the value is enclosed in double quotes.
    • region - specify the region (for example, us-west-2).
    • keyName - specify the name of the key pair used to start the cluster launcher. Key pairs are region-specific. For example, if you create a key pair (or import one you have created) in US-West-2, it will not be available in US-West-1. For information on creating key pairs in Amazon EC2 or importing existing key pairs, see Amazon EC2 Key Pairs.
    • subnetId - ID of the subnet that you noted earlier.
    • securityGroupsIds - ID of the security group that you noted earlier. Use the ID of the group, not the name (for example, sg-b139d3d3, not default).
    • instanceNamePrefix - enter the prefix to prepend to each instance's name.
    • image - specifies the AMI to use. Cloudera recommends Red Hat Enterprise Linux 6.4 (64bit). To find the correct AMI for the selected region, visit the Red Hat AWS Partner page.
  4. Configure the following cluster settings:
    1. You can only use Cloudera Manager 5. No changes are needed for repository and repository key URLs and you must set the parcel repositories to match the CDH and Impala versions you plan to install.
    2. Specify services to start on the cluster. For a complete list of allowed values, see the Cloudera Manager API Service Types.
    3. Specify the number of instances in the cluster.
  5. Save the file and exit.