Authentication methods to use AWS credentials in replication policies
You can choose long-term AWS cloud credentials or temporary AWS session credentials when you want to replicate HDFS data, Hive external tables, and HBase data from Cloudera Private Cloud Base clusters to S3 buckets on Cloudera Public Cloud.
Long-term cloud credentials
You can use long-term credentials to replicate data to the cloud using
replication policies. To use long-term cloud credentials in a replication policy, you must:
- have an AWS account, and access key and secret key for it.
- register an external account in Cloudera Manager using AWS access key and AWS secret
key.
You can add an external account on the Cloudera Private Cloud Base clusters to cloud.
page. The external account serves as an authentication method during data replication, using replication policies, from - add the cloud credential in Cloudera Public Cloud Replication Manager.
The following use cases illustrate scenarios where you can use long-term AWS
credentials:
- Environments where you have multiple users and multi-tenancy – In this instance, you can add an Add Access Key Credentials external account in Cloudera Manager for Cloudera Private Cloud Base cluster, add the cloud credentials in the Cloudera Replication Manager, and then create a replication policy.
- Single user cluster, or where all the users of the cluster have the same privileges to the data in Amazon S3 – In this instance, you can add IAM role-based authentication in Cloudera Manager for Cloudera Private Cloud Base cluster, add the cloud credentials in the Cloudera Replication Manager, and then create a replication policy.
Temporary AWS session credentials
You can use temporary AWS session credentials to provide just-in-time, minimum
required access to replicate data using replication policies. Before you use temporary AWS
session credentials in a replication policy, you must:
- have an AWS account with an IAM role that has the required permissions to access the target S3 bucket and has the necessary trust relationships set up.
- install and configure IDBroker on the Cloudera Private Cloud Base cluster.
- add the cloud credential in Cloudera Replication Manager.
Alternatively, you can add an external account for the IDBroker topology in Cloudera Manager.