Cloudera Flow Management Migration ToolPDF version

Commands

This section contains information about the commands you can use with the Migration Tool when migration your data flows from NiFi 1 to NiFi 2.

The command list includes:

  • Command descriptions: A detailed explanation of each command and its role within the migration workflow.

  • Input arguments: A list of common arguments shared by all commands, along with those specific to individual commands or use cases.

  • Syntax and usage examples: Practical examples showcasing the correct syntax and usage for each command, helping you implement them in your environment.

  • Output artifacts: Information on the output generated by each command, ensuring traceability and transparency throughout the migration process.

Migrates NiFi templates using the Cloudera Flow Management Migration Tool.

migrate-templates --input {filePath} --outputDirectory {directoryPath} [--stage {stageName}] [--processGroupId {pgId}]
--input
  • Short format: -i {filePath}
  • Specifies the path to a local file containing the flow in JSON format for processing.
  • The file must be accessible to the Migration Tool, and reading privileges are required.
  • The original file remains unmodified.
--outputDirectory
  • Short format: -od {directoryPath}
  • Specifies the directory where migration results will be saved.
  • The directory must be accessible to the Migration Tool, and writing privileges are required.
  • Existing content in the specified directory may be overwritten by subsequent runs.
--stage
  • Short format: -s {stageName}
  • Specifies the stage(s) to process.
  • To process only Stage 1 migration, use STAGE_1
  • If omitted, both Stage 1 and Stage 2 are processed sequentially, using Stage 1's output as input for Stage 2.
  • Providing unsupported or custom stage names may result in invalid stage error.
--processGroupId
  • Short format: -pgid {pgId}
  • Defines the starting point for migration, applying transformations to the specified group and its child groups.
  • If omitted, the root group is processed.
Extracted templates
The resulting templates from the migration process.
activity_log.json
It is a log file recording details of the migration process. For more information, see Activity Log.
flow_definitions.json
It is a flow definition representation of their respective template.
1Migrate templates processing Stage 1 and export templates
./migration.sh nifi migrate-templates --input /etc/exported-flows/flow.json \
--outputDirectory /etc/output/ --stage STAGE_1 --processGroupId bb81df68-cd6a-461a-b724-384265875b53

Processes the process group specified by the ID bb81df68-cd6a-461a-b724-384265875b53 only in Stage 1. Reads the input from /etc/exported-flows/flow.json, and saves the results in /etc/output/.

2Migrate templates processing all stages
./migration.sh nifi migrate-templates --input /etc/exported-flows/flow.json \
--outputDirectory /etc/output/

Processes both Stage 1 and Stage 2 sequentially, using the output of Stage 1 as input for Stage 2.

Migrates variable configurations from NiFi 1 to NiFi 2 using the Cloudera Flow Management Migration Tool.

migrate-templates --input {filePath} --outputDirectory {directoryPath} [--stage {stageName}] [--processGroupId {pgId}]
--input
  • Short format: -i {filePath}
  • Specifies the path to a local file containing the flow in JSON format for processing.
  • The file must be accessible to the Migration Tool.
  • The original file remains unmodified.
--outputDirectory
  • Short format: -od {directoryPath}
  • Specifies the directory where migration results will be saved.
  • The directory must be accessible to the Migration Tool.
  • Existing content in the specified directory may be overwritten by subsequent runs.
--stage
  • Short format: -s {stageName}
  • Specifies the stage(s) to process.
  • To process Stage 1 migration, use STAGE_1
  • If omitted, both Stage 1 and Stage 2 are processed sequentially, using Stage 1's output as input for Stage 2.
  • Providing unsupported or custom stage names may result in invalid stage error.
--processGroupId
  • Short format: -pgid {pgId}
  • Defines the starting point for migration, applying transformations to the specified group and its child groups.
  • If omitted, the root group is processed.
NiFi flow
For example: migrated_flow.json.
activity_log.json
A log file recording the details of the migration process. For more information, see Activity Log.
1Migrate variables processing Stage 1
./bin/migration.sh nifi migrate-variables --input /etc/exported-flows/flow.json --outputDirectory /etc/output/ --stage STAGE_1

Processes the flow.json file only in Stage 1, located in /etc/exported-flows/. Saves the migrated variables and activity log in /etc/output/.

2Migrate variables processing all stages
./bin/migration.sh nifi migrate-variables --input /etc/exported-flows/flow.json --outputDirectory /etc/output/

Processes the flow.json file in all stages and saves the results in /etc/output/.

3Migrate variables using full customization with all arguments
./bin/migration.sh nifi migrate-variables --input /etc/exported-flows/flow.json --outputDirectory /etc/output/ --stage STAGE_1 --processGroupId bb81df68-cd6a-461a-b724-384265875b53

Runs Stage 1 of the migration, starting the processing from the process group ID bb81df68-cd6a-461a-b724-384265875b53. Saves the migrated flow and the Activity Log in /etc/output/.

Migrates components from NiFi 1 to NiFi 2 using the Cloudera Flow Management Migration Tool.

migrate-components --input {filePath} --outputDirectory {directoryPath} [--stage {stageName}] [--processGroupId {pgId}]
--input
  • Short format: -i {filePath}
  • Specifies the path to a local file containing the flow in JSON format for processing.
  • The file must be accessible to the Migration Tool.
  • The original file remains unmodified.
--outputDirectory
  • Short format: -od {directoryPath}
  • Specifies the directory where migration results will be saved.
  • The directory must be accessible to the Migration Tool.
  • Existing content in the specified directory may be overwritten by subsequent runs.
--stage
  • Short format: -s {stageName}
  • Specifies the stage(s) to process.
  • To process Stage 1 migration, use STAGE_1
  • If omitted, both Stage 1 and Stage 2 are processed sequentially, using Stage 1's output as input for Stage 2.
  • Providing unsupported or custom stage names may result in invalid stage error.
--processGroupId
  • Short format: -pgid {pgId}
  • Defines the starting point for migration, applying transformations to the specified group and its child groups.
  • If omitted, the root group is processed.
  • Management-level components, such as Controller Services, Parameter Providers, and Reporting Tasks, are only included when this argument is not used.
Flow file
activity_log.json
A log file recording the details of the migration process. For more information, see Activity Log.
1Migrate components processing Stage 1
./bin/migration.sh nifi migrate-components --input /etc/flow_original.json --outputDirectory /etc/output/ --stage STAGE_1

Processes the flow_original.json file located in /etc/. Applies transformation rules from Stage 1 on all components. Saves the migrated flow file and the Activity Log in /etc/output/.

2Migrate components in a specific Process Group
./bin/migration.sh nifi migrate-components --input /etc/flow_original.json --outputDirectory /etc/output/ --processGroupId 3f8d2cba-4d3b-4901-bd0b-4781ff5b5c9f

Starts the migration at the process group identified by 3f8d2cba-4d3b-4901-bd0b-4781ff5b5c9f. Transforms components within this group and its child groups. Outputs are saved in /etc/output/.

3Migrate components using full customization with all arguments
./bin/migration.sh nifi migrate-components --input /etc/flow_original.json --outputDirectory /etc/output/ --stage STAGE_1 --processGroupId 3f8d2cba-4d3b-4901-bd0b-4781ff5b5c9f

Runs Stage 1 only and the migration begins at the specified process group. Management level components are not migrated. Saves the migrated flow and the Activity Log in /etc/output/.

Aggregates the commands of all migration steps when using the Cloudera Flow Management Migration Tool.

  • Combines all migration steps into a single command.
  • Outputs of each step are used as inputs for the next step.
  • The order of the included steps is the following:
    • Stage 1: Migrate Templates → Migrate Variables → Migrate Components
    • Stage 2: Migrate Templates → Migrate Variables → Migrate Components

Displays a list of available commands when using the Cloudera Flow Management Migration Tool.

  • If you use it with a specific command ($[***COMMAND_NAME***] help), it displays the available arguments for it.
  • Specify the full command, including its group: ./migration.sh nifi migrate-all help

We want your opinion

How can we improve this page?

What kind of feedback do you have?