This is the documentation for Cloudera Manager 5.1.x. Documentation for other versions is available at Cloudera Documentation.

Installing Cloudera Manager and CDH on EC2

The following procedure leads you through setting up Cloudera Manager and CDH on a cluster of Amazon Web Services (AWS) EC2 instances.
  • The Cloudera Manager installation wizard launches the EC2 version of the wizard when Cloudera Manager is started on EC2.
  • The resulting installation uses an embedded PostgreSQL database; there is no option for setting up other databases.
  • This wizard installs and starts all the latest Cloudera Manager-managed CDH services.
  Note:
  • The EC2 version of the wizard does not support Amazon Virtual Private Cloud (Amazon VPC).
  • This setup is not recommended for production use.

Continue reading:

Step 1: Set up an AWS EC2 instance for the Cloudera Manager Server.

  Note: The instance on which you install the Cloudera Manager Server must conform to the requirements described in Networking and Security Requirements. In particular, SELinux and iptables must be disabled.
  1. Log into the AWS console.
  2. Go to EC2.
  3. Create a security group:
    1. In the left pane, click Security Groups.
    2. Click Create Security Group.
    3. When prompted, enter a name and description, and click OK.
    4. Select the group you created in the list of groups.
    5. In the bottom panel, go to the Inbound tab.
    6. Authorize TCP ports 22, 7180, 7182, 7183, and 7432.
    7. Authorize ICMP Echo Reply.
  4. Create (or import) an SSH key pair:
    1. In the left pane, click Key Pairs.
    2. Click Create Key Pair.
    3. When prompted, enter a key pair name and click OK
    4. Your private key keypair-name.pem will be downloaded automatically. AWS does not store the private keys – if you lose this file, you won't be able to SSH into instances you provision with this key pair.
  5. Launch an EC2 instance:
    1. In the left pane, click Instances.
    2. Click Launch Instance.
    3. Select the Ubuntu 12.04 AMI 64-bit or other operating system supported by Cloudera Manager. See Cloudera Manager Requirements.
    4. Choose the Instance Type. Cloudera recommends using at least General purpose > m1.large instances.
    5. In the Configure Security Group tab, use the security group and key pair you prepared in the previous steps.
    6. Look at the instance details, and copy the public hostname.
    7. SSH into the instance: 
       $ ssh -i private-key-file username@ec2-xx-xx-xx-xx.compute-1.amazonaws.com

      The username is usually "ubuntu" on Ubuntu systems, and "ec2-user" on most other Linux images on EC2.

    8. Download the Cloudera Manager installer: 
      $ wget http://archive.cloudera.com/cm5/installer/latest/cloudera-manager-installer.bin
    9. Execute the installer: 
      $ sudo su
$ chmod +x cloudera-manager-installer.bin
$ ./cloudera-manager-installer.bin
  6. When the installer finishes, navigate to http://public hostname:7180 and log into the Cloudera Manager Admin console.
  7. (optional) Configure TLS encryption. (See Configuring TLS Security for Cloudera Manager).
  Note:
  • You must upload your AWS account credentials to launch the EC2 instances in the installation wizard, and Cloudera strongly recommends configuring TLS connection.
  • If you encounter any problems, consult Troubleshooting Installation and Upgrade Problems.

Step 2: Use the Cloud Wizard to provision cloud instances and install Cloudera Manager and CDH.

  1. Log into the Cloudera Manager Admin Console on your EC2 instance: <public hostname>:7180. The initial user name and password are admin.
  2. Choose which edition to install:
    • Cloudera Express, which does not require a license, but provides a somewhat limited set of features.
    • Cloudera Enterprise Data Hub Edition Trial, which does not require a license, but expires after 60 days and cannot be renewed
    • Cloudera Enterprise with one of the following license types:
      • Basic Edition
      • Flex Edition
      • Data Hub Edition
    If you choose Cloudera Express or Cloudera Enterprise Data Hub Edition Trial, you can elect to upgrade the license at a later time. See Managing Licenses.
  3. The Welcome Page appears.
      Warning: Instances provisioned on AWS EC2 by this wizard are instance store-based, so all data will be lost when an instance is stopped or terminated.
    Click Continue.
  4. Provide the instance specifications:
    1. Choose the OS.
    2. Alternatively, you may use a custom AMI:
      • Make sure the AMI is in the same region as Cloudera Manager Server.
      • Specify the username Cloudera Manager should use to SSH in. This is usually "ubuntu" on Ubuntu systems, and "ec2-user" on most other Linux images on EC2.
    3. Choose the type of EC2 instances you want to provision. Instances not matching the minimum requirements are deliberately removed from the list. For CDH 5 hosts, select General purpose > m1.large or larger instances.
    4. Specify the number of instances you wish to provision.
    5. Specify the group name (string). This string will be included in the name of your instances and the security group and key pair, which will be created by Cloudera Manager.
  5. Provide credentials:
    1. Enter the AWS access and secret key. To create new ones, follow these instructions:
      1. Go to https://console.aws.amazon.com/iam/.
      2. Click Users.
      3. Check the box next to the desired user, scroll down and click Manage Access Keys.
      4. Copy the new keys and paste them to the inputs.
    2. Choose the instance authentication method:
      1. Let Cloudera Manager create a new SSH key pair for your instances. You will be able to download the private key later to SSH into the new instances.
      2. Import and upload your own key:
        1. In the console, go to Key Pairs.
        2. Click Import Key.
        3. Select your private key file, specify the name and click Yes, Import.
  6. Review the Installation settings:
    1. You may go back if you want to correct any information you provided in the previous steps.
    2. Once the instances are provisioned, you must terminate them if you need to modify the installation settings.
    3. Click Start Installation.
  7. Provision new instances. Once instances are provisioned:
    1. Download the private SSH key if you chose to create one.
  8. The wizard leads you through the installation steps:
    1. Install Cloudera Manager and CDH.
    2. Run the Host Inspector.
    3. Start all services.
  9. When you are finished, terminate the instances through the AWS EC2 console.

Terminating EC2 Instances

  Warning: Cloudera Manager will only terminate instances if the installation fails. You must terminate the instances manually when you are done using the CDH cluster.
  1. Sign into the AWS EC2 console.
  2. In the left pane, select Instances.
  3. Select the instances you want to terminate. You may use the string you entered as "group name" to filter the instances provisioned by Cloudera Manager.
  4. From Actions select Terminate.
Page generated September 3, 2015.