Migrating templates
Learn how to use the Cloudera Flow Management Migration Tool to extract and transform templates for compatibility with NiFi 2. NiFi 2 does not support templates, so this step is required to ensure compatibility with the target version.
Example flow for migrating a template🔗
You have a flow that has a template named elastic_template, which was created from a process group called elastic. This process group contains the following simple flow:

The process group has a variable, Elasticsearch Index, referenced in the index property of the PutElasticsearchJson processor.

Your goal is to migrate this template to NiFi 2 used in Cloudera Flow Management 4.0.0.
-
Stop all processors and disable all controller services in NiFi.
-
Stop NiFi.
-
Copy the flow.json.gz file from NiFi’s conf directory to the Migration Tool’s input folder (/etc/migration-tool-input).
-
Unzip the file to obtain flow.json.
-
Run Stage 1 template migration using the following command.
bin/migration.sh nifi migrate-templates \ -i /etc/migration-tool-input/flow.json \ -od /etc/migration-tool-output/templates \ -s STAGE_1
This generates a v1 folder that contains the output files of the migration.
templates └── v1 ├── NiFi_Flow_ad3a86a6-0194-1000-78d9-6374298d9a0c │ ├── elastic_template.json │ └── elastic_template.xml └── activity_log.json
The folder name NiFi_Flow_ad3a86a6-0194-1000-78d9-6374298d9a0c indicates that the exported template belongs to the root process group, which is named NiFi Flow in NiFi. The unique ID for this group is ad3a86a6-0194-1000-78d9-6374298d9a0c.
- elastic_template.json
-
-
Flow definition containing the contents of the original template
-
Equivalent to manually converting a template to a flow definition in NiFi 1 by:
- Instantiating the template on the canvas.
- Right-clicking the process group.
- Selecting Download flow definition without external services.
-
Modified by the Migration Tool to ensure compatibility:
- Variables (not supported in NiFi 2) converted into parameters
- Parameter context created to hold the new parameters
- Processors updated to reference parameters instead of variables
-
This is still a NiFi 1 flow definition, so do not load this file directly into NiFi 2. Use an empty NiFi 1 instance to review the modification made by the tool. The flow will appear unchanged, but variables will be replaced with parameters of the same name, which are now referenced by the processor instead.
-
Changes in syntax:
Variables were previously referenced using
${}
, whereas parameters are now referenced using#{}
.- NiFi 1
variables:
${variable_name}
- NiFi 2
parameters:
#{parameter_name}
- NiFi 1
variables:
-
- elastic_template.xml
-
-
Original template exported from NiFi 1
-
Identical to downloading it from the Templates menu in NiFi 1
-
Compatible only with NiFi 1 as NiFi 2 does not support templates and cannot parse XML template files
-
- activity_log.json
-
-
Log of all actions performed during this stage of the migration.
-
The following were changes made during the template migration:
- A new parameter context was created with a new parameter in it.
- The parameter context was assigned to the process group.
- The PutElasticsearchJson processor was updated to reference the new parameter.
-
Components are referenced by a unique ID, not by name.
-
-
Validate the Stage 1 template migration output.
-
Check the new flow definition in a NiFi 1 instance to verify that the
flow matches your expectations.
-
Start your NiFi 1 instance.
-
Create a new process group.
-
Load elastic_template.json to NiFi 1.
-
Confirm that variables were correctly converted to parameters.
-
- Review activity_log.json and address any manual-change-requests or manual-inspection-requests.
-
If there are
manual-change-requests
ormanual-inspection-requests
to handle, follow these steps:-
Start a NiFi 1 instance.
-
Load the v1/<template_name>.json flow definition by creating a new process group and uploading the JSON file.
-
Make the modifications on the NiFi canvas, indicated by the
manual-change-requests
ormanual-inspection-requests
in the v1/activity_log.json. -
Stop the NiFi instance.
-
Fetch the flow.json.gz file from the NiFi conf directory.
-
Unzip it to produce a flow.json file.
-
Perform variable migration and then component migration on this flow.json file. For instructions, see the example scenario for variable and component migration.
-
- If no additional changes are required, proceed to Stage 2.
-
Check the new flow definition in a NiFi 1 instance to verify that the
flow matches your expectations.
-
Run full template migration (Stage 1 and 2), using the following command.
bin/migration.sh nifi migrate-templates \ -i /etc/migration-tool-input/flow.json \ -od /etc/migration-tool-output/templates
The contents of the previously generated v1 folder will be overwritten, but this is not a concern as the tool generates the same v1 output as before. Additionally, a v2 directory is created, containing the output files of the Stage 2 part of the migration.
v2 ├── NiFi_Flow_ad3a86a6-0194-1000-78d9-6374298d9a0c │ └── elastic_template.json └── activity_log.json
- elastic_template.json
-
-
NiFi 2-compatible flow definition
-
Contains the contents of the original template converted into an exported process group.
-
This version is compatible with NiFi 2, but no longer supports NiFi 1.
-
- activity_log.json
-
-
List of all actions performed during this stage of the migration.
-
-
Validate the Stage 2 template migration output.
-
Check the new flow definition in a NiFi 2 instance to verify that the
flow matches your expectations.
-
Start your NiFi 2 instance.
-
Create a new process group.
-
Load elastic_template.json into a NiFi 2 instance.
The new process group will be called elastic_template and will contain another process group named elastic, matching the name of the process group that was converted into a template in NiFi 1 before you started the migration.
-
Verify parameter replacements and processor updates.
-
-
Review activity_log.json and address any
manual-change-requests
ormanual-inspection-requests
.In this case, you can see the following
manual-change-request
:{ "sequence" : 4, "type" : "manual-change-request", "subject" : "a1468d69-3f30-3f83-a7c1-91dad09890f7", "message" : "New relationship [original] has been added, it has to be connected to a downstream processor or terminated.", "context" : { "rule" : "6501a693-b524-4d1a-b11e-69747c2651e8" }
-
Open the v2/elastic_template.json file and search for the "subject" ID, a1468d69-3f30-3f83-a7c1-91dad09890f7. This ID refers to the PutElasticsearchJson processor.
-
Go to the NiFi 2 canvas and check the processor. You will find that it has an unbound “original” relationship that needs to be connected to a downstream component.
-
Make the required change manually on the canvas.
-
Once done, export the process group. This exported process group is now a fully NiFi 2-compatible version of the original template.
-
Save the file.
-
-
Check the new flow definition in a NiFi 2 instance to verify that the
flow matches your expectations.
You have finished the template migration process.