Cloudera Docs

Sidecar migration from HDP to CDP

Sidecar migrations allow you to migrate your HDP deployment to CDP on new hardware,

“Side-car” migration allows you to migrate an existing HDP deployment to a new cluster deployed on new (or different) hardware running the latest version of CDP Private Cloud Base. Migrations completed using this process have several advantages, including minimizing downtime during the migration process. You also will have the ability to test your workloads at production scale on the new cluster while keeping your production system running until you determine that the new cluster is ready.

Cloudera provides a variety of tools and documentation to help you move your data, service and cluster configurations, security configurations and policies to the new cluster. The exact procedures to follow may depend on the mix of services deployed on your cluster. This document can help you to learn how to plan and sequence your upgrade. You may need to engage Cloudera Professional Services to plan and execute some parts of the upgrade that could require special procedures not covered in this document.

This document will also provide you with links to useful information about what you need to know about changes in Cloudera Runtime components and how those might affect your workloads. Cloudera recommends that you review those documents closely to avoid unexpected changes in behavior that may delay your migration.

This method of upgrading your cluster has several advantages:

  • Service-level agreements for individual workloads are easier to meet because the legacy versions and the CDP versions can run in parallel without impacting each other.
  • Single, well-contained tenants can move one at a time without requiring a single coordinated event across all tenants.
  • Rollback requires coordination only at the workload or tenant level rather than at the whole cluster level.

  • Enables moving directly to CDP from any HDP 2 or HDP 3 version.

  • Sidecar migration provides the ability to test your workloads at production scale on the new cluster without impacting your live production environment.

You can also use a “rolling side-car migration”, where you decommission some of the existing cluster hosts, install CDP on the decommissioned hosts, and then migrate workloads one at a time. As workloads are migrated, you again decommission more hosts and repeat the process.

For an overview of other upgrade and migration options, see The Four Upgrade and Migration Paths to CDP from Legacy Distributions.

Process Overview

A side-car migration consists of the following phases, which are discussed in detail in other topics.

1. Pre-migration and planning

In this phase, you analyze your workloads and datasets to determine the type and quantity of hosts required for the migration. You also need to consider changes and upgrades to various components used in the source cluster. Some components are no longer available in CDP and you will need to understand how to migrate to services that provide similar functionality.

After determining the specifications for the destination cluster, you can begin to provision your hosts with a supported operating system and install the CDP Private Cloud Base software, which includes Cloudera Manager and Cloudera Runtime.

2. Migration

In this phase, you migrate your data and workloads to the destination cluster and begin testing for functionality and performance. After migrating your data, you can configure the replication process to use “snapshots” so that you can keep the data on the destination cluster synchronized with the source cluster as you complete your workload migration and testing.

If the cluster is using HDFS Transparent Data Encryption, you may also need to migrate KMS ACLs to the new CDP Cluster.

3. Post-migration

Some components require additional steps that must be performed after the data and workload migration are completed. Review the sections for your components in this publication to learn more about these requirements. When these steps are complete and you have completed your testing, you can move your new cluster(s) into production and de-commission the workloads running on the source cluster.