Cloudera Data Flow NiFi 2 migrations
Cloudera Data Flow enables semi-automatic migrations of flows from NiFi 1.x to NiFi 2.x, using predefined transformation rules and logic. [Technical Preview].
Migrating a flow does not modify the source flow or the associated NiFi instances. The source flow is referenced as the starting point (NiFi 1.x) of a migration and a copy of that flow is migrated to NiFi 2.x. The source flow will remain in the catalog as is. The migration will always occur between two specific NiFi 1 and NiFi 2 versions.
Migrating a flow does not modify the source flow or the associated NiFi instances. The source flow is referenced as the starting point (NiFi 1.x) of a migration and a copy of that flow is migrated to NiFi 2.x. The source flow will remain in the catalog as is. The migration will always occur between two specific NiFi 1 and NiFi 2 versions.
Migrations of flows from NiFi 1.x to NiFi 2.x are initiated from the Catalog. After a flow goes through the migration process, Cloudera Data Flow attempts to open the migrated version as a Draft in Flow Design. Flows with multiple process groups require an intermediate step, where the user maps parameter contexts to shared parameter groups.
The following image represents the NiFi 1 to 2 migration workflow:
- You select the NiFi 1.x flow definition from the Cloudera Data Flow Catalog and start the migration from the Actions menu.
- The migration process determines whether the migrated flow can be opened in Flow Designer. If
the flow can be opened, the tool proceeds to create a Draft of the migrated flow in Flow
Designer. Using this draft, you are able to make any necessary manual changes to the flow that
the tool was not able to automatically complete on its own.
In the same step, the process checks whether the selected flow definition has multiple parameter contexts. Before opening the migrated flow in the Flow Designer, you will be prompted to map parameter contexts to shared parameter groups.
If the migrated flow cannot be opened in Flow Designer, the migration tool will make the migrated flow and the migration report available for download, such that you are able to review and make any necessary changes in your tool of choice.
- You provide the required inputs to the migration tool:
- If the migrated flow can be opened in Flow Designer, you need to select an environment, a project, and provide a flow name.
- If the migrated flow cannot be opened in Flow Designer, you only need to select the environment where the migration is executed.
- The migration process launches and checks the NiFi 1.x flow to identify the requirements that need to be addressed in order for it to run in NiFi 2. It then automatically addresses any variances or anomalies that it can handle on its own. In the event that it is unable to automate a change, it will flag it as something that requires manual review or manual intervention in the resulting migration report.
- If applicable, you are prompted to map parameter contexts to shared parameter groups.
- If possible, the migrated flow opens in Flow Designer as a flow draft. If that is not possible, you are prompted to download the migration artifacts in a tar.gz package to resolve any possible remaining issues manually.
- You address issues that require manual intervention either in the Flow Designer or in the external tool of your choice. In Flow Designer, unresolved issues are listed in the Migration Report pane. In the downloaded tar.gz package you find them in the migration_report.json artifact.
- Once you have resolved all outstanding issues, you either publish the flow draft from Flow Designer as a NiFi 2.x flow, or you import it to the Catalog.
