Adding Cloudera Data Warehouse cluster access to external S3 buckets in the same AWS account
This topic explains how to add read/write access to external S3 buckets that reside in the same AWS account as the Cloudera Data Warehouse (CDW) service cluster.
Required role: DWAdmin
If you want to configure access to external S3 buckets that reside in the same AWS account as your CDW cluster, edit the AWS instance profile. Before you edit the instance profile, get the cluster ID from the Environments tile in the CDW service UI:
In the AWS Console, navigate to
<cluster-ID>-dwx-stack. For example, in the above image, the cluster ID is
env-6cwwggso the CloudFormation stack name for this cluster is
env-6cwwgg-dwx-stack. and locate the corresponding stack using the cluster ID you obtained from the CDW Environments tile, and click on its name. The CloudFormation stack name is based on the form:
In the CloudFormation stack details page, click the
Resources tab, locate the
NodeInstanceRole in the Logical ID column, and then click the
adjacent hyperlink in the Physical ID column:
This launches the Identity and Access Management (IAM) console.
- In the IAM console, locate the s3-read-write-own-buckets policy. Click Show…More if you do not see it.
Expand the row for the s3-read-write-own-buckets policy by
clicking the triangle icon to the left of the policy, and then click Edit
In the Edit s3-read-write-own-buckets editor page, click the
JSON tab and append information about the additional external
bucket in the "Resource" section of the JSON file. For example, if you wanted to add
access to the
more-sales-databucket, you append it at the end of the "Resource" section as shown in the following example where these lines are bolded:
"Resource":[ "arn:aws:s3:::cdw-sales-hj9s-dwx-managed", "arn:aws:s3:::cdw-sales-hj9s-dwx-managed/*", "arn:aws:s3:::cdw-sales-hj9s-dwx-external", "arn:aws:s3:::cdw-sales-hj9s-dwx-external/*", "arn:aws:s3:::more-sales-data", "arn:aws:s3:::more-sales-data/*" ],
- Click Review policy in the lower right corner of the page, and then click Save changes. You can access the new bucket from your CDW service cluster now. For example, you can create external Hive tables that point to the newly added bucket.