Migrating variables using a directory with flow definition JSON files as input
Learn how to use the Cloudera Flow Management Migration Tool to convert variables to parameters and parameter contexts from a directory of flow definition JSON files.
Example flows for migrating variables
The input directory contains the following flow definition files:
- TCP_Listener_flow_definition.json
This file contains the TCP Listener process group (ID: b41940d7-0194-1000-42fc-458834630567)
- Elastic_flow_definition.json
This file contains process group Elastic (ID: b42881c7-0194-1000-3cdf-1bd453a0ed0f)

TCP Listener Process Group
This process group contains the following simple flow:

The process group defines a variable called TCP Listener Port, which is referenced by the ListenTCPRecord processor.

Elastic Process Group
This process group also contains a simple flow:

The process group defines a variable called Elasticsearch Index, which is referenced in the PutElasticsearchJson processor’s Index property.

Your goal is to migrate the variables used in Cloudera Flow Management 2.1.7.2000 powered by NiFi 1 to parameters used in Cloudera Flow Management 4.10.0 powered by NiFi 2.
The example guides you through the variable migration process and shows how to maintain a clear activity log. Although the example uses simple flow definitions, the step-by-step approach can be applied to more complex scenarios. In real-world scenarios, Cloudera recommends defining a migration strategy based on your flow’s structure.
Copy the TCP_Listener_flow_definition.json and Elastic_flow_definition.json files to the Migration Tool’s input folder (/etc/migration-tool-input).
-
Run Stage 1 variable migration on the MigrationTool input directory
(/etc/migration-tool-input) using the following
command.
bin/migration.sh nifi migrate-variables \ -i /etc/migration-tool-input \ -od /etc/migration-tool-output/variables \ --sourceCompatibleOutput
This generates a sourceVersion folder that contains the output files of the Stage 1 migration.
variables └── sourceVersion ├── activity_log.json └── migrated_output ├── migrated_Elastic.json └── migrated_TCP_Listener.json
- activity_log.json
-
-
Log of all actions performed during this stage of the migration.
-
The following changes were made during the variable migration:
- A new parameter context was created with a new parameter called TCP Listener Port, which replaces the corresponding variable.
- The TCP Listener Port variable was removed.
- A new parameter context was created with a new parameter called Elasticsearch Index, which replaces the corresponding variable of the same name.
- The new parameter is referenced from the PutElasticsearchJson processor.
- The Elasticsearch Index variable was removed.
-
- migrated_TCP_Listener.json
-
-
A modified flow definition, which is not compatible with NiFi 2 yet.
-
It contains everything the original flow definition did, except the TCP Listener process group now references a parameter instead of the removed variable.
-
This is still a NiFi 1 flow definition, so do not load this file directly into NiFi 2. Use an empty NiFi 1 instance to review the modification made by the tool. The flow definition will appear unchanged, but variables will be replaced with parameters of the same name, which are now referenced by the processor instead.
-
Changes in the syntax: Variables were previously referenced using ${}, whereas parameters are now referenced using #{}.
- NiFi 1 variables: ${variable_name}
- NiFi 2 parameters: #{parameter_name}
-
- migrated_Elastic.json
-
-
A modified NiFi 1 flow definition, which is not compatible with NiFi 2 yet.
-
It contains everything the original flow definition did, but the Elastic process group now references a parameter instead of the removed variable.
-
-
Validate the Stage 1 variable migration output for
TCP_Listener_flow_definition.json.
-
Check the new flow definition in a NiFi 1 instance to verify that the
flow matches your expectations.
-
Start your NiFi instance.
-
Create a new process group.
-
Load sourceVersion/migrated_output/migrated_TCP_Listener.json to NiFi 1.
-
Remove the ‘migrated_‘ prefix from the process group name.
-
Confirm that variables were correctly converted to parameters.
-
-
Review the activity_log.json file and check for
any
manual-change-requests
ormanual-validation-requests
. -
If there are
manual-change-requests
ormanual-validation-requests
to handle, follow these steps:-
Make the modifications on the NiFi canvas, indicated by the
manual-change-requests
ormanual-validation-requests
in the sourceVersion/activity_log.json. -
Right-click on the TCPListener process group.
-
Select Download flow definition without external services.
-
Save the file as migrated_TCP_Listener.json and overwrite the one in the sourceVersion/migrated_output folder.
-
-
If no requests are present in the log,
-
Make a backup of the original TCP_Listener_flow_definition.json file.
-
Right-click on the TCPListener process group.
-
Select Download flow definition without external services.
-
Save the file as migrated_TCP_Listener.json and overwrite the one in the sourceVersion/migrated_output folder.
-
- Proceed to running Stage 1 variable migration on Elastic_flow_definition.json.
-
Check the new flow definition in a NiFi 1 instance to verify that the
flow matches your expectations.
-
Validate the Stage 1 variable migration output for
Elastic_flow_definition.json.
-
Check the new flow definition in a NiFi 1 instance to verify that the
flow matches your expectations.
-
Start your NiFi instance.
-
Create a new process group.
-
Load sourceVersion/migrated_output/migrated_Elastic.json to NiFi 1.
-
Remove the ‘migrated_‘ prefix from the process group name.
-
Confirm that variables were correctly converted to parameters.
-
-
Review the activity_log.json file and check for
any
manual-change-requests
ormanual-validation-requests
. -
If there are
manual-change-requests
ormanual-validation-requests
to handle, follow these steps:-
Make the modifications on the NiFi canvas, indicated by the
manual-change-requests
ormanual-validation-requests
in the sourceVersion/activity_log.json. -
Right-click on the Elastic process group.
-
Select Download flow definition without external services.
-
Save the file as migrated_Elastic.json and overwrite the one in the sourceVersion/migrated_output folder.
-
-
If no requests are present in the log,
-
Right-click on the Elastic process group.
-
Select Download flow definition without external services.
-
Save the file as migrated_Elastic.json and overwrite the one in the sourceVersion/migrated_output folder.
-
At this point, both flow definitions have been updated to replace variables with parameters. If the result aligns with your expectations, you can either perform a full variable migration to validate the flow in NiFi 2 or continue with migrating the components.
-
Check the new flow definition in a NiFi 1 instance to verify that the
flow matches your expectations.
-
Run full variable migration (Stage 1 and 2) using the following command.
bin/migration.sh nifi migrate-variables \ -i /etc/migration-tool-input \ -od /etc/migration-tool-output/variables
The contents of the previously generated sourceVersion folder will be overwritten, but this is not a concern as the tool generates the same sourceVersion output as before.
Additionally, a targetVersion directory is created, containing the output files of the Stage 2 part of the migration.
variables └── sourceVersion │ ├── activity_log.json │ └── migrated_output │ ├── migrated_Elastic.json │ └── migrated_TCP_Listener.json └──targetVersion ├── activity_log.json └── migrated_output ├── migrated_Elastic.json └── migrated_TCP_Listener.json
- migrated_Elastic.json
-
-
NiFi 2-compatible flow definition in terms of variables
-
You can load it into NiFi 2 and validate it
-
- migrated_TCP_Listener.json
-
-
NiFi 2-compatible flow definition in terms of variables
-
You can load it into NiFi 2 and validate it
-
- activity_log.json
-
-
List of all actions performed during this stage of the migration
-
- Proceed with component migration using the input folder from step 3.