Changing flow version
The change flow version feature eliminates the need to tear down and rebuild your flow deployments when changing to another flow definition version. Consider restrictions and available version change strategies.
Restrictions
The following restrictions apply to flow version changes:
-
Changing inbound connections is not supported.
-
Changing custom resource (custom NARs and custom Python resources) configurations is not supported.
- While you can add, change or remove assets when moving to a new version, you cannot introduce assets (text files, binaries, JARs, or similar) if the currently deployed version does not have any.
- Components where state or provenance and other repositories must be kept between flow versions must keep their flow JSON ids. The id changes if you move the component to a different process group or if you delete and then re-add the component to the same process group. NiFi Identifies components by these ids. If you move a component to a different process group between versions, its id changes and NiFi perceives it as a new component. This results in the original component being deleted during flow version change together with its state and a new, identical processor being created in a different process group. In an extreme case, you could change to an identical flow version with just the component ids changed and it would result in the deletion of the entire NiFi flow and the recreation of an identical one, with all history and data lost.
-
Remapping Parameter Group and Parameter Context assignments is not supported as the original assignment is not removed. For example, you have Process Group 1 (PG1) with Parameter Context 1 (PC1) and Process Group 2 (PG2) with Parameter Context 2 (PC2) assigned. If you initiate a flow version change where parameter contexts are flipped, resulting in a PG1-PC2 and PG2-PC1 assignment, NiFi will not re-map the PG to PC assignments.
Version change strategies
Depending on the type of your flow, you may select the flow version change strategy most appropriate to you.
- Stop & Process Data
-
This strategy prioritizes data consistency by stopping source processors and waiting until data is processed before stopping all other components. Once all components have stopped, the flow version is changed and components are started.
Use this strategy when your sources are durable and can handle your source processor being stopped. This generally works well when your source processors are pulling data from sources like Kafka or other messaging queues, databases or file systems.
Should the queued data not be processed within the set time, version change will fail and you can retry the operation with a bigger timeout or you can cancel
- Only Restart Affected Components
-
This strategy prioritizes uptime by identifying and stopping only components that have changed while keeping all others running, replacing and then starting affected components.
Use this strategy when you want to prioritize uptime of unchanged components or you have made only minor processor configuration changes.
This works well for deployments with inbound connections and will keep your source processors running if they have not changed compared to the previous version.
- Stop & Empty Queues
-
This strategy forces a version change by stopping all components, emptying all queues, changing flow version, and then starting all components.
Use this strategy only when you want to force a flow version change without keeping any processors running or attempting to process queued data.
All processors will be stopped and all queues will be emptied as part of this strategy.
Change flow version steps
Learn how to change the flow definition version of a running flow deployment. Using the ‘change flow version’ capability eliminates the need to terminate and re-create deployments when you want to deploy a new version of your flow definition.
- You must have DFFlowAdmin permission.
- There is at least one more version for the same flow definition present in the catalog
- The state of the flow deployment is either Good Health or Stopped.
- You have read the applicable restrictions and version change strategies.
After you click Deploy, you are redirected to the Alerts tab in the Flow Details view where you can track how the version change progresses.
Change flow version troubleshooting
If changing the flow version fails, you can investigate the root cause in Cloudera DataFlow and in NiFi. In ceerain cases you can retry or cancel the operation.