Cloudera Docs

Before migrating

Before you commence to migrate HDFS data from HDP clusters, you must note about some of the underlying requirements.

To run the Hadoop DistCp tool to copy the HDFS data from the source HDP cluster to the Cloudera S3 bucket, the following additional details must be met:

  • The access key, secret key pair of the specified S3 bucket must be available.
    • Before you run the DistCp tool on your on-premises cluster, you must have valid cloud credentials to access and use AWS. Create a Support JIRA that states the time period for the migration.
    • The AWS access secret key will be disabled when the migration time expires.

  • SSM-KMS is enabled on the S3 bucket, the KMS key is also required.

  • Perform the appropriate kinit to the Hadoop DistCp tool for authentication by the Kerberos system.

    • For hdfs user, use the kinit command with hdfs user’s keytab file. (Look for keytab in /etc/security/keytabs)