Setting up the AWS Environment

Whether you are using the Cloudera Director client or server, you must first set up the environment.

Setting Up VPC

Cloudera Director requires Amazon Virtual Private Cloud (Amazon VPC) to implement its virtual environment. You cannot use EC2-Classic. For more information about the differences between EC2-VPC and EC2-Classic, go to Amazon EC2 and Amazon Virtual Private Cloud.

To set up a VPC, follow these steps:

  1. Log in to web console at https://aws.amazon.com/console.
  2. In the upper right of the AWS Console, select a region.
  3. Select VPC from the Services navigation list box.
  4. Click Start VPC Wizard.
  5. On the Select a VPC Configuration page, specify IP address settings, a VPC name, and any other preferences. The easiest way to get started is to select VPC with a Single Public Subnet. For more information, see the VPC documentation.
  6. Click Create VPC.
  7. In the left pane, click Subnets.
  8. Click Create Subnet.
  9. Configure a subnet of the VPC you created and click Yes, Create.
  10. In the left pane, click Security Groups.
  11. Click Create Security Group.
  12. Enter a name and description. Make sure to select the VPC you created from the VPC list box.
  13. Click Yes, Create.
  14. Select the newly created security group and add the following rules:
    1. Add the All traffic, all protocols, all ports, and id of this security group inbound rules. Then add the SSH(22), TCP(6), 22, 0.0.0.0/0 inbound rules. You can secure this further later.



    2. Add the All traffic, all protocols, all ports, 0.0.0.0/0 outbound rule.

    For more information about security group rules, see the AWS documentation: Security Groups for Your VPC.

Creating a Key Pair

To interact with the cluster launcher and other instances, you must create an SSH key pair.

If you do not have a key pair, follow these steps:

  1. Select EC2 from the Services navigation list box.
  2. In the left pane, click Key Pairs.
  3. Click Create Key Pair. In the Create Key Pair dialog box, enter a name for the key pair and click Create.
  4. Note the key pair name. Move the automatically downloaded keyfile (with .pem extension) to a secure location and note the location.

Creating AWS Identity and Access Management (IAM) Policies

In AWS, you use IAM files to create policies that control access to resources in a VPC. Use the AWS Policy Generator to create the IAM file, keeping in mind the following requirements:
  • For EC2, Cloudera Director requires permissions for the following methods:
    • createTags
    • describeAvailabilityZones
    • describeImages
    • describeInstanceStatus
    • describeInstances
    • describeKeyPairs
    • describePlacementGroups
    • describeRegions
    • describeSecurityGroups
    • describeSubnets
    • runInstances
    • terminateInstances
  • To create RDS database servers for persistence on demand, Cloudera Director requires permissions for the following methods:
    • createDBInstance
    • deleteDBInstance
    • describeDBInstances

The following example IAM policy shows the format to use with Cloudera Director. Your Amazon Resource Name (ARN) will be different.

{
  "Statement": [
    {
      "Sid": "Stmt1423159499758",
      "Action": [
        "ec2:CreateTags",
        "ec2:DescribeAvailabilityZones",
        "ec2:DescribeImages",
        "ec2:DescribeInstanceStatus",
        "ec2:DescribeInstances",
        "ec2:DescribeKeyPairs",
        "ec2:DescribePlacementGroups",
        "ec2:DescribeRegions",
        "ec2:DescribeSecurityGroups",
        "ec2:DescribeSubnets",
        "ec2:RunInstances",
        "ec2:TerminateInstances"
      ],
      "Effect": "Allow",
      "Resource": "arn:aws:ec2:<region>:<account>:<resourceType>/<resourcePath>"
    },
    {
      "Sid": "Stmt1423159567748",
      "Action": [
        "rds:CreateDBInstance",
        "rds:DeleteDBInstance",
        "rds:DescribeDBInstances"
      ],
      "Effect": "Allow",
      "Resource": "*"
    }
  ]
}