Cloudera Docs

Cross-account access IAM role

To allow CDP to create resources in your AWS account, you create a cross-account access IAM role in your AWS account and grant CDP access to the role as a trusted principal by specifying a specific AWS account and an external ID.

The policy for the cross-account access IAM role must have the permissions enumerated in the documentation linked below. In addition, the IAM role must reference the specific AWS account ID and external ID provided in the Management Console.

IAM policy definitions

When creating an IAM policy for the provisioning credential, use the following IAM policy definitions.

You have two options:

Option Description Policies
Default policy This policy is appropriate for registering a CDP environment and provisioning all CDP services. This is the same IAM policy as the Default policy provided in the Management Console web interface.
Reduced access policies Instead of the extensive default policy, you can use minimal reduced access policies. Note that the first policy is required for provisioning a CDP environment. The remaining policies are optional and are only needed if you would like to use the specific CDP services.

Create a cross-account IAM role

In order to use role-based authentication, you must create an IAM role on AWS.

Prior to creating a cross-account IAM role on AWS, log in to the CDP web interface and obtain the parameters that need to provide for the IAM role:
  1. In the Management Console, navigate to Shared Resources > Credentials > Create Credential:
  2. Note or copy the Cross-account Access Policy, Account ID, and External ID listed in the UI:

    You will need them to complete the following steps.

  1. Log in to the AWS Management Console.
  2. Navigate to the IAM console > Policies and click Create policy:
    1. Select the JSON tab and paste the Cross-account Access Policy to the Policy editor.
    2. Click Next.
    3. Enter a name for your policy in the Name field of the Review policy page:
    4. Click Create policy when finished.
  3. Navigate to the IAM console > Roles and click Create role:
  4. Select AWS account as Trusted Entity type.
  5. Check Another AWS account and provide the following information:
    • Copy and paste your Account ID provided in CDP to the Accound ID field.
    • Check Require external ID under Options.
    • Copy and paste the External ID from CDP to the External ID field.
  6. Click Next.
  7. Search for the previously created policy, and add the permission to the role by checking the box.
  8. Click Next.
  9. Enter a role name to the Role name field:
  10. Click Create role to finish the role creation process.
  11. Search for the created role, and obtain the IAM Role ARN. You will need it to create a role-based credential:

Once you are done creating the IAM role on AWS, create a role-based credential in CDP.

Reduced access policy definition

The following reduced policy definition can optionally be used to register a CDP environment and create Data Hubs and Operational Databases in it.

The IAM role used for the provisioning credential can use the reduced access policy instead of the default policy.

You should:

  1. Copy this policy.

  2. Update it as described below.

  3. Once done, you can create the cross-account role using this policy.

Customize the scope of iam:PassRole

CDP utilizes the iam:PassRole as part of the cloud identity federation setup process via IDBroker on AWS. The default policy uses “*” as the resource scope for this action: 
{
     "Effect": "Allow
      "Action": [
        "iam:PassRole"
      ],
      "Resource": "*"
}

However the scope can be reduced to just a few roles. To do this, you need to edit the following block and replace the “*” with the ARNs of the following roles used for the environment:

  • IDBROKER_ROLE
  • LOG_ROLE
  • RANGER_AUDIT_ROLE
  • DATALAKE_ADMIN_ROLE

You choose the name for these roles when you set up CDP as part of the Minimal Setup for Cloud Storage.

After updating according to the minimal setup for cloud storage, the iam:PassRole policy excerpt should like similar to the following: 

{
      "Effect": "Allow
      "Action": [
        "iam:PassRole"
      ],
      "Resource": [
               "arn:aws:iam::$ACCOUNTID:role/IDBROKER_ROLE",
               "arn:aws:iam::$ACCOUNTID:role/LOG_ROLE",
               “arn:aws:iam::$ACCOUNTID:role/RANGER_AUDIT_ROLE",
               "arn:aws:iam::$ACCOUNTID:role/DATALAKE_ADMIN_ROLE"
           ]
}

If you have already created these roles and you don’t know the name of these roles, you can obtain them using the following steps:

IDBROKER_ROLE

  1. Log in to CDP web interface and navigate to the Management Console service > Data Lakes.
  2. Scroll down to where you see Event History and select the Hardware tab.
  3. Locate the EC2 instance for the idbroker node (if you are running Medium Duty Data Lake, you will see two IDBroker nodes, you can use either):
  4. Click on the instance id. You will be redirected to the EC2 console.
  5. In the Instance Details section for this node, you can see the name of the IAM role being used by this instance. Click on this role to get to the details.
  6. Copy the ARN of the instance profile. It follows the following naming convention: arn:aws:iam::<12-digit-AWS-account-id>:instance-profile/<name-of-idbroker-role>

LOG_ROLE

  1. Navigate to Environments .
  2. Click on the existing environment
  3. Click on the Summary tab.
  4. Scroll to Logs Storage and Audits and you will see the Instance Profile entry.
  5. Copy the ARN of the Instance Profile:

RANGER_AUDIT_ROLE and DATALAKE_ADMIN_ROLE

  1. Navigate to Environments.
  2. Click on the existing environment.
  3. From the Actions menu, select Manage Access.
  4. Click on the IDBroker Mappings tab.
  5. Under Current Mappings you will see the mappings for the Data Access Role and Ranger Audit Role.
  6. Copy the ARNs of the Data Access Role and Ranger Audit Role:

Further permissions reduction (Optional)

You can reduce these policies even further by pre-creating some resources:

  • If you set up your own VPC and subnets, then the following actions can be removed from the policy:
    ec2:CreateVpc
ec2:CreateNatGateway
ec2:CreateRouteTable
ec2:CreateSubnet
ec2:CreateVpcEndpoint
ec2:CreateInternetGateway
ec2:DeleteSubnet
ec2:DeleteInternetGateway
ec2:AttachInternetGateway
ec2:DetachInternetGateway
ec2:DescribePrefixLists
ec2:AllocateAddress
ec2:AssociateRouteTable
ec2:CreateRoute
ec2:DeleteRouteTable
ec2:DeleteVpcEndpoints
ec2:DisassociateRouteTable
ec2:ReleaseAddress
ec2:DeleteRoute
ec2:DeleteNatGateway
ec2:DeleteVpc
  • If you use your own security groups, then the following actions can be removed from the policy:
    ec2:CreateSecurityGroup
ec2:AuthorizeSecurityGroupIngress
ec2:RevokeSecurityGroupEgress
ec2:AuthorizeSecurityGroupEgress
  • If you use private IPs or set up CCM to communicate with the CDP Control Plane, then the following actions can be removed from the policy:
    ec2:CreateSecurityGroup
ec2:AuthorizeSecurityGroupIngress
ec2:RevokeSecurityGroupEgress
ec2:AuthorizeSecurityGroupEgress
  • If you use private IPs or set up CCM to communicate with the CDP Control Plane, then the following actions can be removed from the policy:
    ec2:AllocateAddress
ec2:ReleaseAddress