Migrating templates using template.xml as input

Learn how to use the Cloudera Flow Management Migration Tool to migrate a downloaded template.xml file and transform templates for compatibility with NiFi 2. NiFi 2 does not support templates, so this step is required to ensure compatibility with the target version. This command combines the functionality of the migrate-variables and migrate-components commands so for templates it is equivalent with migrate-all. While the migrate-templates command simplifies the migration process, for larger flows, it is usually better to run the two migration commands separately. You may also want to migrate process groups individually to ensure activity logs and manual validation remain manageable.

Example flow for migrating a template

You have a template.xml file called Elastic_template.xml, which was created from a process group called elastic. This process group contains the following simple flow:

The process group has a variable, Elasticsearch Index, referenced in the index property of the PutElasticsearchJson processor.

Your goal is to migrate this template to NiFi 2 used in Cloudera Flow Management 4.10.0.

Copy the elastic_template.xml file to the Migration Tool’s input folder (/etc/migration-tool-input).

Run Stage 1 template migration using the following command.
```
bin/migration.sh nifi migrate-templates \
-i /etc/migration-tool-input/elastic_template.xml \
-od /etc/migration-tool-output/templates \
-sco
```
This generates a sourceVersion folder that contains the output files of the migration.
```
templates
└── sourceVersion
    ├── Elastic_template_b4175f57-0194-1000-8470-9251a24519b4
    │   └── Elastic_template.json
    └── activity_log.json
```
The folder name Elastic_template_b4175f57-0194-1000-8470-9251a24519b is a temporary process group which was created during the migration process and used to separate the components in the template from other components that may be present during the migration.
Elastic_template.json
- Flow definition containing the contents of the original template
- Equivalent to manually converting a template to a flow definition in NiFi 1 by:
  
  Instantiating the template on the canvas.
  
  Right-clicking the process group.
  
  Selecting Download flow definition without external services.
- Modified by the Migration Tool to ensure compatibility:
  
  Variables (not supported in NiFi 2) converted into parameters
  
  Parameter context created to hold the new parameters
  
  Processors updated to reference parameters instead of variables
- This is still a NiFi 1 flow definition, so do not load this file directly into NiFi 2. Use an empty NiFi 1 instance to review the modification made by the tool. The flow will appear unchanged, but variables will be replaced with parameters of the same name, which are now referenced by the processor instead.
- Changes in the syntax: Variables were previously referenced using ${}, whereas parameters are now referenced using #{}.
  
  NiFi 1 variables: ${variable_name}
  
  NiFi 2 parameters: #{parameter_name}
activity_log.json
- Log of all actions performed during this stage of the migration.
- The following were changes made during the template migration:
  
  A new parameter context was created with a new parameter in it.
  
  The parameter context was assigned to the process group.
  
  The PutElasticsearchJson processor was updated to reference the new parameter.
- Components are referenced by a unique ID, not by name.
Validate the Stage 1 template migration output.
1. Check the new flow definition in a NiFi 1 instance to verify that the flow matches your expectations.
  1. Start your NiFi 1 instance.
  2. Create a new process group.
  3. Load elastic_template.json to NiFi 1.
  4. Confirm that variables were correctly converted to parameters.
2. Review activity_log.json and address any manual-validation-requests or manual-validation-requests.
3. If there are manual-change-requests or manual-validation-requests to handle, follow these steps:
  1. Make the modifications on the NiFi canvas, indicated by the manual-change-requests or manual-validation-requests in the sourceVersion/activity_log.json.
  2. Fetch the flow.json.gz file from the NiFi conf directory.
  3. Right-click on the Elastic process group.
  4. Select Download flow definition without external services.
  5. Save the file as Elastic_Listener.json and overwrite the one in the sourceVersion/<process_group>folder.
4. If no requests are present in the log, proceed to the next step.
Run full template migration (Stage 1 and 2), using the following command.
```
bin/migration.sh nifi migrate-templates \
-i /etc/migration-tool-input/Elastic_template.xml \
-od /etc/migration-tool-output/templates
```
The contents of the previously generated sourceVersion folder will be overwritten, but this is not a concern as the tool generates the same sourceVersion output as before. Additionally, a targetVersion directory is created, containing the output files of the Stage 2 part of the migration.
```
targetVersion
├── Elastic_template_b4175f57-0194-1000-8470-9251a24519b4
│   └── Elastic_template.json
└── activity_log.json
```
Elastic_template.json
- NiFi 2-compatible flow definition
- Contains the contents of the original template converted into an exported process group.
- This version is compatible with NiFi 2, but no longer supports NiFi 1.
note

An elastic_template.xml is not generated as NiFi 2 does not support XML templates.
activity_log.json
- List of all actions performed during this stage of the migration.
Validate the Stage 2 template migration output.
1. Check the new flow definition in a NiFi 2 instance to verify that the flow matches your expectations.
  1. Start your NiFi 2 instance.
  2. Create a new process group.
  3. Load elastic_template.json into a NiFi 2 instance.
    
    The new process group will be called elastic_template and will contain another process group named elastic, matching the name of the process group that was converted into a template in NiFi 1 before you started the migration.
  4. Verify parameter replacements and processor updates.
2. Review activity_log.json and address any manual-change-requests or manual-validation-requests.
  In this case, you can see the following manual-change-request:
```
{
	"sequence" : 4,
	"type" : "manual-change-request",
	"subject" : "a1468d69-3f30-3f83-a7c1-91dad09890f7",
	"message" : "New relationship [original] has been added, it has to be connected to a downstream processor or terminated.",
	"context" : {
  	"rule" : "6501a693-b524-4d1a-b11e-69747c2651e8"
}
```
  1. Open the targetVersion/Elastic_template.json file and search for the "subject" ID, a1468d69-3f30-3f83-a7c1-91dad09890f7. This ID refers to the PutElasticsearchJson processor.
  2. Go to the NiFi 2 canvas and check the processor. You will find that it has an unbound “original” relationship that needs to be connected to a downstream component.
  3. Make the required change manually on the canvas.
  4. Once done, export the process group. This exported process group is now a fully NiFi 2-compatible version of the original template.
  5. Save the file.

You have finished the template migration process.