Cloudera Docs

Cloudera Data Flow NiFi 2 migrations

Cloudera Data Flow enables semi-automatic migrations of flows from NiFi 1.x to NiFi 2.x, using predefined transformation rules and logic. [Technical Preview]. Learn about the migration process.

You can migrate NiFi 1.x flow definitions from the Cloudera Data Flow Catalog. If the flow definition you want to migrate is not there, you must add it first.

  1. You select a NiFi 1.x flow definition from the Cloudera Data Flow Catalog and start the migration, providing the required inputs to the Migration Tool.

    Migrating a flow does not modify the source flow or the associated NiFi instances. The source flow is referenced as the starting point (NiFi 1.x) of a migration and a copy of that flow is migrated to NiFi 2.x. The source flow remains in the catalog as is.

    Migration always occurs between two specific NiFi 1 and NiFi 2 versions. The Migration Tool calculates the minimum compatible NiFi version where the created flow draft can be opened in Flow Designer and only allows you to select target workspaces that meet this requirement. If you have no access to any such workspace, you cannot proceed with the migration.

  2. The Migration Tool checks the NiFi 1.x flow to identify the requirements that need to be addressed in order for it to run in NiFi 2.x. The Tool then automatically addresses any variances or anomalies that it can handle on its own. In the event that it is unable to automate a change, the Tool flags it as something that requires manual review or manual intervention.

    The tool proceeds to create a draft of the migrated flow in Flow Designer. Using this draft, you will be able to make any necessary manual changes to the flow that the tool was not able to automatically complete on its own.

    Some of the major transformations are:

    • Parameter providers are converted to parameters.

    • All controller services are moved to the root process group. Names of the moved controller services are appended with the name of their original process group and a random hash.
    • All parameters are moved to the root process group. If there are several parameters with identical names, they are renamed in the parameterContextName.parameter format, where parameterContextName is the name of the original parameter context in the source flow definition.
    • Variables, not supported in NiFi 2.x, are converted to parameters.
    • Sensitive parameter values are not migrated.

  3. If the source flow had multiple parameter contexts in different process groups, you are asked to match parameter contexts to shared parameter groups or merge them into root level draft parameters in an intermediate step.

  4. The created flow draft opens in Flow Designer where you address issues that require manual intervention. In Flow Designer, unresolved issues are listed in the Migration Report pane. In the downloadable tar.gz package you find them in the migration_report.json artifact.
  5. Once you have resolved all outstanding issues, publish the flow draft to the Catalog as a NiFi 2.x flow.