Installing Cloudera Manager and CDH on EC2
The following procedure leads you through setting up Cloudera Manager and CDH on a cluster of Amazon Web Services (AWS) EC2 instances.
- The Cloudera Manager installation wizard launches the EC2 version of the wizard when Cloudera Manager is started on EC2.
- The resulting installation uses an embedded PostgreSQL database; there is no option for setting up other databases.
- This wizard installs and starts all the latest Cloudera Manager-managed CDH services.
Note:
- The EC2 version of the wizard does not support Amazon Virtual Private Cloud (Amazon VPC).
- This setup is not recommended for production use.
Step 1: Set up an AWS EC2 instance for the Cloudera Manager Server.
Note: The instance on which you install the Cloudera Manager Server must conform to the requirements described in Networking and Security Requirements. In particular, SELinux and iptables must be disabled.
- Log into the AWS console.
- Go to EC2.
- Create a security group:
- In the left pane, click Security Groups.
- Click Create Security Group.
- When prompted, enter a name and description, and click OK.
- Select the group you created in the list of groups.
- In the bottom panel, go to the Inbound tab.
- Authorize TCP ports 22, 7180, 7182, 7183, and 7432.
- Authorize ICMP Echo Reply.
- Create (or import) an SSH key pair:
- In the left pane, click Key Pairs.
- Click Create Key Pair.
- When prompted, enter a key pair name and click OK
- Your private key <keypair-name >.pem will be downloaded automatically. AWS does not store the private keys – if you lose this file, you won't be able to SSH into instances you provision with this key pair.
- Launch an EC2 instance:
- In the left pane, click Instances.
- Click Launch Instance.
- Select the Ubuntu 12.04 AMI 64-bit or other operating system supported by Cloudera Manager. See Cloudera Manager Requirements.
- Choose the Instance Type. Cloudera recommends using at least instances.
- In the Configure Security Group tab, use the security group and key pair you prepared in the previous steps.
- Look at the instance details, and copy the public hostname.
- SSH into the instance:
$ ssh -i <private-key-file> <username>@ec2-xx-xx-xx-xx.compute-1.amazonaws.com
The <username> is usually "ubuntu" on Ubuntu systems, and "ec2-user" on most other Linux images on EC2.
- Download the Cloudera Manager installer:
$ wget http://archive.cloudera.com/cm5/installer/latest/cloudera-manager-installer.bin
- Execute the installer:
$ sudo su $ chmod +x cloudera-manager-installer.bin $ ./cloudera-manager-installer.bin
- When the installer finishes, navigate to http://<public hostname>:7180 and log into the Cloudera Manager Admin console.
- (optional) Configure TLS encryption. (See Configuring TLS Security for Cloudera Manager).
Note:
- You must upload your AWS account credentials to launch the EC2 instances in the installation wizard, and Cloudera strongly recommends configuring TLS connection.
- If you encounter any problems, consult Troubleshooting Installation and Upgrade Problems.
Step 2: Use the Cloud Wizard to provision cloud instances and install Cloudera Manager and CDH.
- Log into the Cloudera Manager Admin Console on your EC2 instance: <public hostname>:7180. The initial user name and password are admin.
- Choose which edition to install:
- Cloudera Express, does not require a license, but provides a somewhat limited set of features.
- Cloudera Enterprise Data Hub Edition Trial, does not require a license, but expires after 60 days and cannot be renewed.
- Cloudera Enterprise with one of the following license
types:
- Basic Edition
- Flex Edition
- Data Hub Edition
- The Welcome Page appears. WarningClick Continue.
: Instances provisioned on AWS EC2 by this wizard are instance store-based, so all data will be lost when an instance is stopped or terminated. - Provide the instance specifications:
- Choose the OS.
- Alternatively, you may use a custom AMI:
- Make sure the AMI is in the same region as Cloudera Manager Server.
- Specify the username Cloudera Manager should use to SSH in. This is usually "ubuntu" on Ubuntu systems, and "ec2-user" on most other Linux images on EC2.
- Choose the type of EC2 instances you want to provision. Instances not matching the minimum requirements are deliberately removed from the list. For CDH 5 hosts, select or larger instances.
- Specify the number of instances you wish to provision.
- Specify the group name (string). This string will be included in the name of your instances and the security group and key pair, which will be created by Cloudera Manager.
- Provide credentials:
- Enter the AWS access and secret key. To create new ones, follow these instructions:
- Go to https://console.aws.amazon.com/iam/.
- Click Users.
- Check the box next to the desired user, scroll down and click Manage Access Keys.
- Copy the new keys and paste them to the inputs.
- Choose the instance authentication method:
- Let Cloudera Manager create a new SSH key pair for your instances. You will be able to download the private key later to SSH into the new instances.
- Import and upload your own key:
- In the console, go to Key Pairs.
- Click Import Key.
- Select your private key file, specify the name and click Yes, Import.
- Enter the AWS access and secret key. To create new ones, follow these instructions:
- Review the Installation settings:
- You may go back if you want to correct any information you provided in the previous steps.
- Once the instances are provisioned, you must terminate them if you need to modify the installation settings.
- Click Start Installation.
- Provision new instances. Once instances are provisioned:
- Download the private SSH key if you chose to create one.
- The wizard leads you through the installation steps:
- Install Cloudera Manager and CDH.
- Run the Host Inspector.
- Start all services.
- When you are finished, terminate the instances through the AWS EC2 console.
Terminating EC2 Instances
Warning: Cloudera Manager will only terminate instances if the installation fails. You must terminate the instances manually when you are done using the CDH cluster.
- Sign into the AWS EC2 console.
- In the left pane, select Instances.
- Select the instances you want to terminate. You may use the string you entered as "group name" to filter the instances provisioned by Cloudera Manager.
- From Actions select Terminate.
<< Creating and Using a Package Repository | Using Whirr to Launch Cloudera Manager >> | |