Commands

Syntax

migrate-templates --input {filePath} --outputDirectory {directoryPath} [--sourceCompatibleOutput] [--processGroupId {pgId}]

Required arguments

--input

• Short format: -i {filePath}
• Specifies the path to a local file containing the flow in JSON format for processing.
• The file must be accessible to the Migration Tool, and reading privileges are required.
• The original file remains unmodified.

--outputDirectory

• Short format: -od {directoryPath}
• Specifies the directory where migration results will be saved.
• The directory must be accessible to the Migration Tool, and writing privileges are required.
• Existing content in the specified directory may be overwritten by subsequent runs.

Supported arguments

--sourceCompatibleOutput

• Short format: -sco
• If specified, onlyStage 1 migration will be run.
• If omitted, both Stage 1 and Stage 2 are processed sequentially, using Stage 1's output as input for Stage 2.

--processGroupId

• Short format: -pgid {pgId}
• Defines the starting point for migration, applying transformations to the specified group and its child groups.
• If omitted, the root group is processed.

--inputType

• Short format: -it {inputType}
• The type of input provided, case sensitive. Supported types: FLOW, FLOW_GZ, FLOW_DEFINITION, TEMPLATE_XML, REGISTRY_SNAPSHOT
• If omitted, the tool tries to automatically identify the input type. Mixed inputs are not allowed.

--registryAware

• Short format: -ra
• Skips versioned process groups during migration so their content is handled by migrate-registry and other Registry-related commands instead.

Output files of Stage 1 and Stage 2

Extracted templates

The resulting templates from the migration process.

activity_log.json

It is a log file recording details of the migration process. For more information, see Activity Log.

flow_definitions.json

It is a flow definition representation of their respective template.

Examples

Syntax

migrate-templates --input {filePath} --outputDirectory {directoryPath} [--sourceCompatibleOutput] [--processGroupId {pgId}]

Required arguments

--input

• Short format: -i {filePath}
• Specifies the path to a local file containing the flow in JSON format for processing.
• The file must be accessible to the Migration Tool.
• The original file remains unmodified.

--outputDirectory

• Short format: -od {directoryPath}
• Specifies the directory where migration results will be saved.
• The directory must be accessible to the Migration Tool.
• Existing content in the specified directory may be overwritten by subsequent runs.

Supported arguments

--sourceCompatibleOutput

• Short format: -sco {stageName}
• If specified, only Stage 1 migration will be executed. Specifies the stage(s) to process.
• To process only Stage 1 migration, use STAGE_1.
• If omitted, both Stage 1 and Stage 2 are processed sequentially, using Stage 1's output as input for Stage 2.
• Providing unsupported or custom stage names may result in invalid stage error.

--processGroupId

• Short format: -pgid {pgId}
• Defines the starting point for migration, applying transformations to the specified group and its child groups.
• If omitted, the root group is processed.

--inputType

• Short format: -it {inputType}
• The type of input provided, case sensitive. Supported types: FLOW, FLOW_GZ, FLOW_DEFINITION, TEMPLATE_XML, REGISTRY_SNAPSHOT
• If omitted, the tool tries to automatically identify the input type. Mixed inputs are not allowed.

--registryAware

• Short format: -ra
• Skips versioned process groups during migration so their content is handled by migrate-registry and other Registry-related commands instead.

Output files of Stage 1 and Stage 2

NiFi flow

For example: migrated_flow.json.

activity_log.json

A log file recording the details of the migration process. For more information, see Activity Log.

Examples

Syntax

migrate-components --input {filePath} --outputDirectory {directoryPath} [--sourceCompatibleOutput] [--processGroupId {pgId}]

Required arguments

--input

• Short format: -i {filePath}
• Specifies the path to a local file containing the flow in JSON format for processing.
• The file must be accessible to the Migration Tool.
• The original file remains unmodified.

--outputDirectory

• Short format: -od {directoryPath}
• If specified, only Stage 1 migration will run. Specifies the stage(s) to process
• To process only Stage 1 migration, use STAGE_1.
• If omitted, both Stage 1 and Stage 2 are processed sequentially, using Stage 1's output as input for Stage 2.
• Providing unsupported or custom stage names may result in invalid stage error.

Supported arguments

--sourceCompatibleOutput

• Short format: -sco {stageName}
• To process Stage 1 migration, use STAGE_1
• If omitted, both Stage 1 and Stage 2 are processed sequentially, using Stage 1's output as input for Stage 2.
• Providing unsupported or custom stage names may result in invalid stage error.

--processGroupId

• Short format: -pgid {pgId}
• Defines the starting point for migration, applying transformations to the specified group and its child groups.
• If omitted, the root group is processed.
• Management-level components, such as Controller Services, Parameter Providers, and Reporting Tasks, are only included when this argument is not used.

--inputType

• Short format: -it {inputType}
• The type of input provided, case sensitive. Supported types: FLOW, FLOW_GZ, FLOW_DEFINITION, TEMPLATE_XML, REGISTRY_SNAPSHOT
• If omitted, the tool tries to automatically identify the input type. Mixed inputs are not allowed.

--registryAware

• Short format: -ra
• Skips versioned process groups during migration so their content is handled by migrate-registry and other Registry-related commands instead.

Output files of Stage 1 and Stage 2

Flow file

activity_log.json

A log file recording the details of the migration process. For more information, see Activity Log.

Examples

Usage

Supported arguments

--registryAware

• Short format: -ra
• Skips versioned process groups during template, variable, and component steps so their content is handled by migrate-registry and other Registry-related commands instead.

Description

This is the primary tool for migrating registries. While there are some auxiliary commands, this command is responsible for most of the work done during registry snapshot migration. Its main purpose is to apply changes to the snapshots provided to the command. Its behavior is very similar to the migration commands for flows or individual snapshot files (supported by other commands like migrate-components), but this command is optimized to support the migration of a high number of snapshots with the least amount of administrative overhead.

To enable this, execution of the command requires a Working Registry to be set up and properly configured in the properties. Please see the relevant section of the Prerequisites. While the command’s primary purpose is always to migrate and manage migrated snapshots, the Migrate Registry Snapshot command supports different modes of execution for different steps during the migration process.

Execution modes:

start

The initial step for registry migration. This step requires a Working Registry where no snapshots from an ongoing migration are stored. If the Migration Tool detects artifacts of an ongoing registry migration within the Working Registry, the command will fail. Otherwise, the command migrates the content (snapshots) of the provided registry exports and writes the results into the Working Registry as “initial migration” results. These migrated snapshots may then be subject to manual changes and further migrations.

Additionally, this step generates a modified flow.json, which is not migrated at this point, but all registered flow references are updated to point to the Working Registry. This modified flow serves as input for later steps. The start step can be executed only once within a sequence of Migrate Registry Snapshot commands, and only as the first step. Furthermore, it can be executed only during Stage 1. Running it with both stages will result in an error.

See Properties for details on the naming conventions that the Migration Tool applies to flows and buckets within the Working Registry.

restart

The restart step behaves the same as start in most respects. The difference lies in how it handles the state of the Working Registry: at the beginning of execution, instead of checking for an ongoing migration, restart cleans up the Working Registry by removing all flows and buckets considered part of a migration (based on the current properties). Be aware that if the Working Registry contains other flows, this step may unintentionally remove them. All subsequent steps match the behavior of the start step. The restart step is intended to simplify restarting the migration process.

See Properties for details on the naming conventions that the Migration Tool applies to flows and buckets within the Working Registry.

continue

This step is responsible for further migrations of the snapshots stored in the Working Registry. During Stage 1, this step can be repeated as many times as necessary, along with applying manual changes. In Stage 2, as in flow migration, it can be executed only once. This is the only step that can be executed during Stage 2. Unlike the other options, continue reads snapshots from and writes migrated snapshots back to the Working Registry.

Syntax

migrate-registry {stepArgument} [--input {filePath}] [--exportDirectory {directoryPath}] --outputDirectory {directoryPath} [--sourceCompatibleOutput] [--registry {registryName}] [--registryBucket {bucketName}] [--registryFlow {flowName}]

Required arguments

Step argument

• The following step arguments might be provided:
  • --start (short form: -s)
  • --restart (short form: -res)
  • --continue (short form: -c)
• A given time only exactly one step argument is expected.
• The step argument defines which step is being executed (see above)

--outputDirectory

• Short format: -od {directoryPath}
• Specifies the directory where migration results will be saved.
• The directory must be accessible to the Migration Tool.
• Existing content in the specified directory may be overwritten by subsequent runs.
• Required argument for start and restart steps.

Supported arguments

--input

• Short format: -i {filePath}
• Specifies the path to a local file containing the flow in JSON format for processing.
• The file must be accessible to the Migration Tool.
• The original file remains unmodified.
• Required argument for start and restart steps.

--exportDirectory

• Short form: -ed {directoryPath}
• Specifies the directory where the registry exports (need to be migrated) are stored.
• The exports themselves are represented by sub-directories.
• The name of the subdirectories must be the same as the name of the matching Registry Client.
• The directory must be accessible to the Migration Tool.
• The original directory and its content remains unmodified.
• The original file remains unmodified.

--sourceCompatibleOutput

• Short format: -sco
• If specified, only Stage 1 migration will be executed. Specifies the stage(s) to process.
• To process only Stage 1 migration, use this argument.
• If omitted, both Stage 1 and Stage 2 are processed sequentially, using Stage 1's output as input for Stage 2.
• Steps start and restart are not to be run within Stage 2 and requires this argument.

--registry

• Short format: -r {registryName}
• Helps narrowing down the scope of the migration to the snapshots within the registry with the given registryName.
• The registry name equals to the directory name of the original registry export (and the name of the Registry Client matches with that).
• Can be used only with the continue step and within Stage 1.

--registryBucket

• Short format: -rb {bucketName}
• This argument requires the argument registry.
• Helps narrowing down the scope of the migration to the snapshots within the bucket with the given bucketName.
• The bucketName is the origin bucket name, not the one stored within the Working Registry (prefixed with the registry name).
• Can be used only with the continue step and within Stage 1.

--registryFlow

• Short format: -rf {flowName}
• This argument requires the argument registryBucket.
• Helps narrowing down the scope of the migration to the snapshots within the flow with the given flowName.
• The flowName must match with the name as the flow stored within the Working Registry (including line part).
• Can be used only with the continue step and within Stage 1.

Output files of Stage 1 and Stage 2

NiFi flow

When executing the start or restart step, the NiFi flow is part of the output. Not migrated but with replaced versioned flows.

Activity logs

Above the activity_log.json which contains the generic events of the execution, every migrated snapshot has its own specific log, for example: R1_Bucket_1_Flow_2-activity_log.json.

Examples

Description

This command updates the flow.json by refreshing the versioned flows. For every managed versioned flow, the Migration Tool upgrades it to the newest version available in the Working Registry. The flow must have already been processed by the start step.

For Stage 1, its use is optional and mainly useful for testing purposes. In Stage 2, this command is mandatory and can be executed either explicitly or implicitly as part of the migrate-all command.

Syntax

upgrade-flow --input {filePath} --outputDirectory {directoryPath} [--sourceCompatibleOutput]

Required arguments

--input

• Short format: -i {filePath}
• Specifies the path to a local file containing the flow in JSON format for processing.
• The file must be accessible to the Migration Tool.
• The original file remains unmodified.

--outputDirectory

• Short format: -od {directoryPath}
• Specifies the directory where migration results will be saved.
• The directory must be accessible to the Migration Tool.
• Existing content in the specified directory may be overwritten by subsequent runs.

Supported arguments

--sourceCompatibleOutput

• Short format: -sco
• If specified, only Stage 1 migration will be executed. Specifies the stage(s) to process.
• To process only Stage 1 migration, use this argument.
• When running without this argument, this command has no affect during Stage 1.

Output files of Stage 1 and Stage 2

NiFi flow

With replaced versioned flow.

Examples

Description

During the migration process, the flow and the snapshots are in an “under migration” format. This means that snapshots are stored in the Working Registry, and all versioned flow references point to the location of the Working Registry. This is not the expected final state, but it cannot be changed during migration, as the absence of these references would make the manual work involved in the migration difficult to manage.

To address this, as the final step of the migration process, run the Export Registry Migration Result command. This command exports the snapshots in a format identical to a NiFi Toolkit registry export while updating the reference locations during the process. This command cannot be executed for Stage 1 only and applies only to Stage 2.

Syntax

export-registry-result --input {filePath} --outputDirectory {directoryPath} [--exportDirectory {directoryPath}] [--keepHistory]

Required arguments

--input

• Short format: -i {filePath}
• Specifies the path to a local file containing the flow in JSON format for processing.
• The file must be accessible to the Migration Tool.
• The original file is not modified.

--outputDirectory

• Short format: -od {directoryPath}
• Specifies the directory where migration results will be saved.
• The directory must be accessible to the Migration Tool.
• Existing content in the specified directory may be overwritten by subsequent runs.

Supported arguments

--exportDirectory

• Short form: -ed {directoryPath}
• Specifies the directory where the registry exports (need to be migrated) are stored.
• The exports themselves are represented by sub-directories.
• The directory must be accessible to the Migration Tool.
• The original directory and its content remains unmodified.
• The original file remains unmodified.
• Necessary if keepHistory is added to the command.

--keepHistory

• Short form: -kh
• Might be added if the original snapshots should be kept in the exported result as "historical versions". As a consequence, the original history can be stored in the target registry, but as NiFi 1 flows, not being usable in NiFi 1. The original files export will not be modified regardless of the usage of this argument.
• If provided, exportDirectory must be specified as well, pointing to the original registry exports.

Output for Stage 2

NiFi flow

With final versioned flow references

Registry exports

One folder per registries involved, in the same format as with NiFi Toolkit registry export

Activity log

Execution details

Examples

Description

Syntax

cleanup-working-registry --outputDirectory {directoryPath}

Required arguments

--outputDirectory

• Short format: -od {directoryPath}
• Specifies the directory where migration results will be saved.
• The directory must be accessible to the Migration Tool.
• Existing content in the specified directory may be overwritten by subsequent runs.

Output for Stage 1 and Stage 2

Activity log

Only the activity log serves as output, containing details about the removed resources.

Examples

Usage