Migrating Data to Cloudera on CloudPDF version

Migrating Hive data from HDP 2.x or HDP 3.x to

The recommended way to migrate Hive data from HDP to depends on the types of tables you are migrating: external, legacy-managed, or ACID (managed) tables.

The default behavior in has changed so that all newly created managed tables are transactional tables by default. To minimize impact to applications during upgrade and keep legacy behavior, all non-acid legacy tables (both managed and external) will be converted to external tables in during migration. Legacy ACID tables will be migrated to managed ACID tables in CDP.
All managed and external Non-ACID tables will be migrated to external tables in using the following components:
  • HMS-Mirror: using the SCHEMA_ONLY mode to transfer metadata
  • DistCP
In this scenario, HMS-Mirror and distcp are triggered from the source cluster (LEFT):

If direct Hive connectivity from on-prem to is not available, you can dump of the schema to export it, and then preform a re-run manually in the target cluster.

All managed ACID source tables (applicable to HDP) are migrated to ACID tables in using HMS Mirror in the HYBRID migrate-acid mode with the intermediate-storage option. This approach migrates both data and metadata using Hive queries, and stages the data in an intermediate-storage location on S3. This approach avoids the need to link the target cloud environment with the on-prem source.

In this scenario, HMS-Mirror is triggered from the source cluster (LEFT), as shown in the following diagram:

We want your opinion

How can we improve this page?

What kind of feedback do you have?