Upgrade and migration paths

To use the latest enhancements and capabilities of NiFi, Cloudera recommends deploying it on the Cloudera Data Platform (CDP). You can seamlessly integrate NiFi with CDP by installing the Cloudera Flow Management (CFM) parcel on your CDP infrastructure.

This guide provides information on upgrading or migrating from various NiFi deployment scenarios, including HDF, CFM on CDH, or earlier versions of CFM on CDP Private Cloud Base. Review the upgrade and migration details to ensure that you understand the specific requirements relevant to your chosen upgrade or migration scenario, aligning them with your unique use case.

Upgrade and migration paths overview

From CFM 2.x on CDP Private Cloud Base

If you are using CFM 2.x on CDP Private Cloud Base (CDP PvC Base) version 7.1.1 – 7.1.8, and you intend to upgrade to a higher CFM version, there are specific upgrade paths to follow. Check the Upgrade paths diagram and refer to the step-by-step procedure outlined in the following table:

Table 1. Upgrade procedure from CFM 2.x
Upgrade
Step 1 Upgrade CFM to 2.0.4 if you are using CFM 2.0.1 on CDP PvC Base 7.1.1
Step 2 Upgrade CDP Private Cloud Base to 7.1.7, 7.1.7 SP1, or 7.1.7 SP2
Step 3 Upgrade CFM to a higher version
Step 4 Upgrade CDP Private Cloud Base to 7.1.8

From CFM 1.x on CDH

If you are using CFM 1.x on CDH 5 or CDH 6, or as standalone CFM, and you want to upgrade to a higher CFM version, there are specific upgrade paths to follow. Check the Upgrade paths diagram and refer to the step-by-step procedure outlined in the following table:

Table 2. Upgrade/migration options from CFM 1.x
Migration Upgrade from CFM 1.x on CDH 6 Upgrade from CFM 1.x on CDH 5 Upgrade from standalone CFM 1.x
Step 1 Upgrade CFM 1.0.0 to 1.1.0 Upgrade CFM 1.0.1 to 1.1.0 Upgrade CFM 1.0.0 to 1.1.0 Upgrade CFM 1.0.1 to 1.1.0 Upgrade CFM 1.0.0 to 1.1.0 Upgrade CFM 1.0.1 to 1.1.0 Upgrade CFM 1.0.0 to 1.1.0 Upgrade CFM 1.0.1 to 1.1.0
Step 2 Install CFM 2.x on CDP Private Cloud Base 7.1.1 – 7.1.8 Upgrade CDH 6 to CDP Private Cloud Base 7.1.7, 7.1.7 SP1, or 7.1.7 SP2 Upgrade CDH 5.0-5.12 to CDH/CM 5.13 or higher

For instructions, see the Cloudera Enterprise Upgrade Guide.

Upgrade CFM 1.1.0 to CFM 2.x
Step 3 Migrate data from CFM 1.x to CFM 2.x Upgrade from CFM 1.1.0 to CFM 2.x Upgrade CDH 5.13 or higher to CDP Private Cloud Base 7.1.1 or 7.1.7
Step 4 Upgrade CFM 1.1.0 to CFM 2.x

For a better understanding of the differences between upgrade and migration, see CFM upgrade and migration options.

From HDF

If you are using HDF and you want to upgrade to CFM, there are specific upgrade paths to follow. Check the Upgrade paths diagram and refer to the step-by-step procedure outlined in the following table:

Table 3. Upgrade/migration options from HDF
Migration In-place upgrade
Step 1 Upgrade HDF to 3.5.x

See also the HDF 3.5.2 Release Notes.

Upgrade HDF to 3.5.2

See also the HDF 3.5.2 Release Notes.

Step 2 Install CFM 2.x on CDP Private Cloud Base 7.1.1 – 7.1.8 In-place upgrade HDF 3.5.2 to CFM 2.1.1 on CDP PvC Base 7.1.6 or CFM 2.1.2 – 2.1.4 on CDP PvC Base 7.1.7.
Step 3 Migrate data from HDF to CFM

For a better understanding of the differences between upgrade and migration, see CFM upgrade and migration options.

CFM upgrade and migration options

There are three methods available for transitioning from an older version to CFM 2.x:

Upgrade
Upgrade refers to a full upgrade of CFM on CDP Private Cloud Base without the necessity for migrating the data between clusters.
Possible upgrade paths include:
Migration
Data migration involves the transfer of existing HDF or CFM 1.x cluster workloads to a fresh installation of CDP Private Cloud Base. You can migrate your NiFi dataflows and NiFi Registry versioned flows from an HDF 3.5.x or CFM 1.1.0 (standalone, or on CDH 5 / CDH 6) cluster to a CFM 2.x cluster on CDP PvC Base. This method requires that you have both your HDF or CFM 1.1.0 cluster and a CFM 2.x cluster running at the same time.
For detailed instructions on performing migration, see Migration from HDF 3.5.x or CFM 1.x.
Possible migration paths include:
  • HDF 3.5.x > CFM 2.x
  • CFM 1.1.0 > CFM 2.x
In-place upgrade
This option allows you to install Cloudera Manager when using HDF and then enable Cloudera Manager to take over the management of your services. You can transform your existing Ambari blueprint into a Cloudera Manager Deployment template using the AM2CM tool.
If you have a version of HDF lower than 3.5.2.0, first upgrade to HDF 3.5.2.0 and then upgrade HDF to CFM 2.1.1 – 2.1.4 on CDP PvC Base 7.1.6 or 7.1.7. For detailed instructions on performing an in-place upgrade, see In-place upgrade from HDF 3.5.x.
Possible in-place upgrade paths include:
  • HDF 3.5.2.0 > CFM 2.1.1 on CDP 7.1.6
  • HDF 3.5.2.0 > CFM 2.1.1-2.1.5 on CDP 7.1.7