Upgrade and migration paths

To use the latest enhancements and capabilities of NiFi, Cloudera recommends deploying it on the Cloudera Data Platform (CDP). You can seamlessly integrate NiFi with CDP by installing the Cloudera Flow Management (CFM) parcel on your CDP infrastructure.

This guide provides information on upgrading or migrating from various NiFi deployment scenarios, including HDF, CFM on CDH, or earlier versions of CFM on CDP Private Cloud Base. Review the upgrade and migration details to ensure that you understand the specific requirements relevant to your chosen upgrade or migration scenario, aligning them with your unique use case.

Upgrade and migration paths overview

From CFM 2.x on CDP Private Cloud Base

If you are using CFM 2.x on CDP Private Cloud Base (CDP PvC Base) version 7.1.1 – 7.1.8 and you intend to upgrade to a higher CFM version, there are specific upgrade paths to follow. Check the Upgrade paths diagram and refer to the step-by-step procedure outlined in the following table:

Table 1. Upgrade procedure from CFM 2.x
Upgrade
Step 1 Upgrade CFM to 2.0.4 if you are using CFM 2.0.1 on CDP PvC Base 7.1.1
Step 2 Upgrade CDP PvC Base to 7.1.7, 7.1.7 SP1, or 7.1.7 SP2
Step 3 Upgrade CFM to a higher version

Check the Upgrade paths diagram to determine the sequence of intermediate upgrades required to transition from your current CFM version to the desired one.

Step 4 Optional: Upgrade CDP PvC Base to 7.1.9

From CFM 1.x on CDH

If you are using CFM 1.x on CDH 5 or CDH 6, or as standalone CFM, and you intend to upgrade to a higher CFM version, there are specific upgrade paths to follow. Check the Upgrade paths diagram and refer to the step-by-step procedure outlined in the following table:

Table 2. Upgrade/migration options from CFM 1.x
Migration Upgrade from CFM 1.x on CDH 6 Upgrade from CFM 1.x on CDH 5 Upgrade from standalone CFM 1.x
Step 1 Upgrade CFM 1.0.0 to 1.1.0 Upgrade CFM 1.0.1 to 1.1.0 Upgrade CFM 1.0.0 to 1.1.0 Upgrade CFM 1.0.1 to 1.1.0 Upgrade CFM 1.0.0 to 1.1.0 Upgrade CFM 1.0.1 to 1.1.0 Upgrade CFM 1.0.0 to 1.1.0 Upgrade CFM 1.0.1 to 1.1.0
Step 2 Install CFM 2.x on CDP PvC Base 7.1.1 – 7.1.9 Upgrade CDH 6 to CDP PvC Base 7.1.7, 7.1.7 SP1 or 7.1.7 SP2 Upgrade CDH 5.0 - 5.12 to CDH 5.13 or higher

For instructions, see the Cloudera Enterprise Upgrade Guide.

Upgrade CFM 1.1.0 to CFM 2.x
Step 3 Migrate data from CFM 1.x to CFM 2.x Upgrade from CFM 1.1.0 to CFM 2.x Upgrade CDH 5.13 or higher to CDP PvC Base
Step 4 Upgrade CFM 1.1.0 to CFM 2.x

For a better understanding of the differences between upgrade and migration, see CFM upgrade and migration options.

From HDF

If you are using HDF and you intend to upgrade to CFM, there are specific upgrade paths to follow. Check the Upgrade paths diagram and refer to the step-by-step migration and in-place upgrade procedures outlined in the following table:

Table 3. Upgrade/migration options from HDF
Migration In-place upgrade
Step 1 Upgrade HDF to 3.5.x.

See also the HDF 3.5.2 Release Notes.

Upgrade HDF to 3.5.2

See also the HDF 3.5.2 Release Notes.

Step 2 Install CFM 2.x on CDP PvC Base 7.1.1 – 7.1.8 In-place upgrade HDF 3.5.2 to CFM 2.1.1 on CDP PvC Base 7.1.6 or to CFM 2.1.2 – 2.1.4 on CDP PvC Base 7.1.7
Step 3 Migrate data from HDF to CFM

For a better understanding of the differences between upgrade and migration, see CFM upgrade and migration options.

CFM upgrade and migration options

There are three methods available for transitioning from an older version to CFM 2.x:

Upgrade
Upgrade refers to a full upgrade of CFM on CDP PvC Base without the necessity for migrating the data between clusters.
Possible upgrade paths include:
  • CFM 1.1.0 > CFM 2.x
  • CFM 2.x > CFM 2.x
Migration
Data migration involves the transfer of existing HDF or CFM 1.x cluster workloads to a fresh installation of CDP PvC Base. You can migrate your NiFi dataflows and NiFi Registry versioned flows from an HDF 3.5.x or CFM 1.1.0 (standalone, or on CDH 5 / CDH 6) cluster to a CFM 2.x cluster on CDP PvC Base. This method requires that you have both your HDF or CFM 1.1.0 cluster and a CFM 2.x cluster running at the same time.
Possible migration paths include:
  • HDF 3.5.x > CFM 2.x
  • CFM 1.1.0 > CFM 2.x
In-place upgrade
This option allows you to install Cloudera Manager when using HDF and then enable Cloudera Manager to take over the management of your services. You can transform your existing Ambari blueprint into a Cloudera Manager Deployment template using the AM2CM tool. If your HDF version is lower than 3.5.2.0, first upgrade to HDF 3.5.2.0 and then proceed to upgrade HDF to CFM 2.1.1 – 2.1.4 on CDP PvC Base 7.1.6 or 7.1.7.
Possible in-place upgrade paths include:
  • HDF 3.5.2.0 > CFM 2.1.1 on CDP 7.1.6
  • HDF 3.5.2.0 > CFM 2.1.1-2.1.5 on CDP 7.1.7