Cloudera Docs

Configuring IDBroker to use in replication policies

After you add a role instance to install IDBroker, you configure the required properties for it in Cloudera Manager for the Cloudera Private Cloud Base cluster.

  1. Go to the source Cloudera Manager > Clusters > Knox service > Instances page.
  2. Click the role instance on the Instances tab to open the Knox IDBroker service.
  3. Click the Configuration tab.
  4. Click Continue Editing Role Instance if an alert appears.
  5. Add the Kerberos username and ARN in the [***USERNAME***]=[***ARN***] format in the Knox IDBroker AWS User Mapping property. This maps the Kerberos user to the AWS role.
    If you have multiple usernames, you can map it as shown in the following sample Knox IDBroker AWS User Mapping value. The sample shows Kerberos users repl, hdfs, and hbase mapped to an AWS IAM role:

    repl=arn:aws:iam::134232123254:role/cldr-demo-role;hdfs=arn:aws:iam::134232123254:role/cldr-demo-role;hbase=arn:aws:iam::134232123254:role/cldr-demo-role

  6. Add the following in the Kerberos Proxy Block property for HDFS replication policies. This allows the hdfs user to impersonate the Kerberos user during the replication policy run:
    • “hadoop.proxyuser.hdfs.groups”: “[***KERBEROS USER***]
    • “hadoop.proxyuser.hdfs.hosts”: “*”
  7. Perform the following steps for HBase replication policies:
    1. Add the following details in the Kerberos Proxy Block property. This allows the hbase users to impersonate the Kerberos users during the replication policy run.
      • “hadoop.proxyuser.hbase.groups”: “[***KERBEROS USER***]
      • “hadoop.proxyuser.hbase.hosts”: “*”
    2. Add the following details in the Cluster-wide Advanced Configuration Snippet (Safety Valve) for core-site.xml in HDFS configuration. This allows the hbase users to impersonate the Kerberos users when exporting the initial snapshot from HDFS.
      • “hadoop.proxyuser.hbase.groups”: “[***KERBEROS USER***]
      • “hadoop.proxyuser.hbase.hosts”: “*”
  8. Configure the IDBroker Knox Token TTL property to ensure that the configured Knox session token time is greater than the time required to complete a replication policy run, and Save Changes.
  9. Use the default aws-cab topology, or create a custom topology, if required, using the Knox IDBroker Advanced Configuration Snippet (Safety Valve) for conf/cdp-resources.xml property. You can also create multiple topologies depending on your use case requirements.
    The following sample code shows a custom topology added to the Knox IDBroker Advanced Configuration Snippet (Safety Valve) for conf/cdp-resources.xml property:
    <property>
    <name>[***TOPOLOGY1***]</name>
    <value>
         providerConfigRef=cab-providers#IDBROKER:cloud.policy.config.provider=default#IDBROKER:cloud.client.provider=AWS
    </value>
</property>
  10. Complete the following steps to create the aws.credentials.key and aws.credentials.secret aliases in the topology.
    1. Search for the Save Alias Command Input property:
    2. Enter [***TOPOLOGY1***].aws.credentials.secret=[***SECRET***], and click Save Changes.
    3. Click Actions > Save Alias - IDBroker.
    4. Enter [***TOPOLOGY1***].aws.credentials.key=[***ACCESS KEY***], and click Save Changes.
    5. Click Actions > Save Alias - IDBroker.
  11. Optional: Add the following credential details to use the default AWS topology in IDBroker if all the required IAM roles are assumed by a single set of long-term AWS keys. IDBroker uses these credentials to authenticate and to request session tokens from AWS Session Token Service (AWS STS). These credentials are used by IDBroker only to request session tokens and are not used during replication.
    • Knox IDBroker AWS Credentials Key
    • Knox IDBroker AWS Credentials Secret
  12. Save the changes.
  13. Restart Stale Services, if any.
You can add the cloud credential in Cloudera Replication Manager. Alternatively, you can add an external account for the IDBroker topology in Cloudera Manager to use in replication policies.