Prerequisites on AWS

Before launching Cloudbreak on AWS, you must meet the following prerequisites.

AWS account

In order to launch Cloudbreak on AWS, you must log in to your AWS account. If you don't have an account, you can create one at https://aws.amazon.com/.

AWS region

Decide in which AWS region you would like to launch Cloudbreak. The following AWS regions are supported:

Region name Region
EU (Ireland) eu-west-1
EU (Frankfurt) eu-central-1
US East (N. Virginia) us-east-1
US West (N. California) us-west-1
US West (Oregon) us-west-2
South America (São Paulo) sa-east-1
Asia Pacific (Tokyo) ap-northeast-1
Asia Pacific (Singapore) ap-southeast-1
Asia Pacific (Sydney) ap-southeast-2

Clusters created via Cloudbreak can be in the same or different region as Cloudbreak; when you launch a cluster, you select the region in which to launch it.

Related links
AWS regions and endpoints (External)

SSH key pair

Import an existing key pair or generate a new key pair in the AWS region which you are planning to use for launching Cloudbreak and clusters. You can do this using the following steps.

Steps

  1. Navigate to the Amazon EC2 console at https://console.aws.amazon.com/ec2/.
  2. Check the region listed in the top right corner to make sure that you are in the correct region.
  3. In the left pane, find NETWORK AND SECURITY and click Key Pairs.
  4. Do one of the following:
    • Click Create Key Pair to create a new key pair. Your private key file will be automatically downloaded onto your computer. Make sure to save it in a secure location. You will need it to SSH to the cluster nodes. You may want to change access settings for the file using chmod 400 my-key-pair.pem.
    • Click Import Key Pair to upload an existing public key and then select it and click Import. Make sure that you have access to its corresponding private key.

You need this SSH key pair to SSH to the Cloudbreak instance and start Cloudbreak.

Related links
Creating a key pair using Amazon EC2 (External)

Virtual network

You must have a virtual network configured on your cloud provider.

Security group

Ports 22 (SSH), 80 (HTTPS), and 443 (HTTPS) must be open on the security group

Authentication

Before you can start using Cloudbreak for provisioning clusters, you must select a way for Cloudbreak to authenticate with your AWS account and create resources on your behalf. There are two ways to do this:

(Option 1) Use key-based authentication

If you are using key-based authentication for Cloudbreak on AWS, you must be able to provide your AWS access key and secret key pair. Cloudbreak will use these keys to launch the resources. You must provide the access and secret keys later in the Cloudbreak web UI later when creating a credential.

If you choose this option, all you need to do at this point is check your AWS account and make sure that you can access this key pair. You can generate new access and secret keys from the IAM Console > Users. Next, select a user and click on the Security credentials tab:

The minimum set of permissions required by Cloudbreak are the same as for the CredentialRole.

If you choose this option, you can proceed to launch Cloudbreak.

(Option 2) Configure role-based authentication

If you are using role-based authentication for Cloudbreak on AWS, you must create two IAM roles: one to grant Cloudbreak access to allow Cloudbreak to assume AWS roles (using the "AssumeRole" policy) and the second one to provide Cloudbreak with the capabilities required for cluster creation (using the "cb-policy" policy).

The following table provides contextual information about the two roles required:

Role Purpose Overview of steps Configuration
CloudbreakRole Allows Cloudbreak to assume other IAM roles - specifically the CredentialRole. Create a role called "CloudbreakRole" and attach the "AssumeRole" policy. The "AssumeRole" policy definition and steps for creating the CloudbreakRole are provided below.

When launching Cloudbreak, you will attach the "CloudbreakRole" IAM role to the VM.

If you are using hosted Cloudbreak, you do not need to perform this step.
CredentialRole Allows Cloudbreak to create AWS resources required for clusters.

Create a new IAM role called "CredentialRole" and attach the "cb-policy" policy to it. The "cb-policy" policy definition and steps for creating the CredentialRole are provided below.

When creating this role using the AWS Console, make sure that that it is a role for cross-account access and that the trust-relation is set up as follows: 'Account ID' is your own 12-digit AWS account ID and 'External ID' is “provision-ambari”. See steps below.

 Once you log in to the Cloudbreak UI and are ready to create clusters, you will use this role to create the Cloudbreak credential.

These role and policy names are just examples. You may use different names when creating your resources.

CloudbreakRole: Alternatively, instead of attaching the "CloudbreakRole" role during the VM launch, you can assign the "CloudbreakRole" to an IAM user and then add the access and secret key of that user to your 'Profile'.

CredentialRole: Alternatively you can generate the "CredentialRole" role later once your Cloudbreak VM is running by SSHing to the Cloudbreak VM and running the cbd aws generate-role command. This command creates a role with the name "cbreak-deployer" (equivalent to the "CredentialRole"). To customize the name of the role, add export AWS_ROLE_NAME=my-cloudbreak-role-name (where "my-cloudbreak-role-name" is your custom role name) as a new line to your Profile. If you choose this option, you must make sure that the "CloudbreakRole" or the IAM user have a permission not only to assume a role but also to create a role.

You can create these roles in the IAM console, on the Roles page via the Create Role option. Detailed steps are provided below.

Related links
Using instance profiles (External)
Using an IAM role to grant permissions to applications (External)

Create CloudbreakRole

Use these steps to create CloudbreakRole.

Use the following "AssumeRole" policy definition:

{
  "Version": "2012-10-17",
  "Statement": {
    "Sid": "Stmt1400068149000",
    "Effect": "Allow",
    "Action": ["sts:AssumeRole"],
    "Resource": "*"
  }
}

Steps

  1. Navigate to the IAM console > Roles and click Create Role.

  2. In the "Create Role" wizard, select AWS service role type and then select any service.

  3. When done, click Next: Permissions to navigate to the next page in the wizard.

  4. Click Create policy.

  5. Click Select next to "Create Your Own Policy".

  6. In the Policy Name field, enter "AssumeRole" and in the Policy Document paste the policy definition. You can either copy it from the section preceding these steps or download and copy it from here.

  7. When done, click Create Policy.

  8. Click Refresh. Next, find the "AssumeRole" policy that you just created and select it by checking the box.

  9. When done, click Next: Review.

  10. In the Roles name field, enter role name, for example "CloudbreakRole".

  11. When done, click Create role to finish the role creation process.

Related links
CloudbreakRole

Attach the CloudbreakRole to the VM

After creating the "CloudbreakRole", attach it to your VM. You can do this during the VM launch process or once your VM is running.

Attach CloudbreakRole when launching the VM

The option to attach an IAM role is available from the instance launch wizard Step 3: Configure Instance:

Attach CloudbreakRole to an existing VM

Perform the following steps from the EC2 console on AWS:

  1. Select the EC2 instance that you are planning to use for Cloudbreak and from the Actions menu select Instance Settings > Attach/Replace IAM Role:

  2. Next, under IAM role, select the CloudbreakRole that you created earlier and click Apply to apply the configuration:

    I

Create CredentialRole

Use these steps to create CredentialRole.

Use the following "cb-policy" policy definition:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "cloudformation:CreateStack",
        "cloudformation:DeleteStack",
        "cloudformation:DescribeStackEvents",
        "cloudformation:DescribeStackResource",
        "cloudformation:DescribeStacks"
      ],
      "Resource": [
        "*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "ec2:AllocateAddress",
        "ec2:AssociateAddress",
        "ec2:AssociateRouteTable",
        "ec2:AuthorizeSecurityGroupIngress",
        "ec2:DescribeRegions",
        "ec2:DescribeAvailabilityZones",
        "ec2:CreateRoute",
        "ec2:CreateRouteTable",
        "ec2:CreateSecurityGroup",
        "ec2:CreateSubnet",
        "ec2:CreateTags",
        "ec2:CreateVpc",
        "ec2:ModifyVpcAttribute",
        "ec2:DeleteSubnet",
        "ec2:CreateInternetGateway",
        "ec2:CreateKeyPair",
        "ec2:DeleteKeyPair",
        "ec2:DisassociateAddress",
        "ec2:DisassociateRouteTable",
        "ec2:ModifySubnetAttribute",
        "ec2:ReleaseAddress",
        "ec2:DescribeAddresses",
        "ec2:DescribeImages",
        "ec2:DescribeInstanceStatus",
        "ec2:DescribeInstances",
        "ec2:DescribeInternetGateways",
        "ec2:DescribeKeyPairs",
        "ec2:DescribeRouteTables",
        "ec2:DescribeSecurityGroups",
        "ec2:DescribeSubnets",
        "ec2:DescribeVpcs",
        "ec2:DescribeSpotInstanceRequests",
        "ec2:DescribeVpcAttribute",
        "ec2:ImportKeyPair",
        "ec2:AttachInternetGateway",
        "ec2:DeleteVpc",
        "ec2:DeleteSecurityGroup",
        "ec2:DeleteRouteTable",
        "ec2:DeleteInternetGateway",
        "ec2:DeleteRouteTable",
        "ec2:DeleteRoute",
        "ec2:DetachInternetGateway",
        "ec2:RunInstances",
        "ec2:StartInstances",
        "ec2:StopInstances",
        "ec2:TerminateInstances"
      ],
      "Resource": [
        "*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "iam:ListRolePolicies",
        "iam:GetRolePolicy",
        "iam:ListAttachedRolePolicies",
        "iam:ListInstanceProfiles",
        "iam:PutRolePolicy",
        "iam:PassRole",
        "iam:GetRole"
      ],
      "Resource": [
        "*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "autoscaling:CreateAutoScalingGroup",
        "autoscaling:CreateLaunchConfiguration",
        "autoscaling:DeleteAutoScalingGroup",
        "autoscaling:DeleteLaunchConfiguration",
        "autoscaling:DescribeAutoScalingGroups",
        "autoscaling:DescribeLaunchConfigurations",
        "autoscaling:DescribeScalingActivities",
        "autoscaling:DetachInstances",
        "autoscaling:ResumeProcesses",
        "autoscaling:SuspendProcesses",
        "autoscaling:UpdateAutoScalingGroup"
      ],
      "Resource": [
        "*"
      ]
    }
  ]
}

Steps

  1. Navigate to the IAM console > Roles and click Create Role.

  2. In the "Create Role" wizard, select Another AWS account role type. Next, provide the following:

    • In the Account ID field, enter your AWS account ID.
    • Under Options, check Require external ID.
    • In the External ID, enter "provision-ambari".

  3. When done, click Next: Permissions to navigate to the next page in the wizard.

  4. Click Create policy.

  5. Click Select next to "Create Your Own Policy".

  6. In the Policy Name field, enter "cb-policy" and in the Policy Document paste the policy definition. You can either copy it from the section preceding these steps or download and copy it from here.

  7. When done, click Create Policy.

  8. Click Refresh. Next, find the "cb-policy" that you just created and select it by checking the box.

  9. When done, click Next: Review.

  10. In the Roles name field, enter role name, for example "CredentialRole".

  11. When done, click Create role to finish the role creation process.

Once you are done, you can proceed to launch Cloudbreak.

Related links
CredentialRole