Changing flow version

The change flow version feature eliminates the need to tear down and rebuild your flow deployments when changing to another flow definition version. Consider restrictions and available version change strategies.

Restrictions

The following restrictions apply to flow version changes:

Changing inbound connections is not supported.
Changing custom resource (custom NARs and custom Python resources) configurations is not supported.
While you can add, change or remove assets when moving to a new version, you cannot introduce assets (text files, binaries, JARs, or similar) if the currently deployed version does not have any.
Components where state or provenance and other repositories must be kept between flow versions must keep their flow JSON ids. The id changes if you move the component to a different process group or if you delete and then re-add the component to the same process group. NiFi Identifies components by these ids. If you move a component to a different process group between versions, its id changes and NiFi perceives it as a new component. This results in the original component being deleted during flow version change together with its state and a new, identical processor being created in a different process group. In an extreme case, you could change to an identical flow version with just the component ids changed and it would result in the deletion of the entire NiFi flow and the recreation of an identical one, with all history and data lost.
Remapping Parameter Group and Parameter Context assignments is not supported as the original assignment is not removed. For example, you have Process Group 1 (PG1) with Parameter Context 1 (PC1) and Process Group 2 (PG2) with Parameter Context 2 (PC2) assigned. If you initiate a flow version change where parameter contexts are flipped, resulting in a PG1-PC2 and PG2-PC1 assignment, NiFi will not re-map the PG to PC assignments.

Version change strategies

Depending on the type of your flow, you may select the flow version change strategy most appropriate to you.

Stop & Process Data

This strategy prioritizes data consistency by stopping source processors and waiting until data is processed before stopping all other components. Once all components have stopped, the flow version is changed and components are started.

Use this strategy when your sources are durable and can handle your source processor being stopped. This generally works well when your source processors are pulling data from sources like Kafka or other messaging queues, databases or file systems.

Should the queued data not be processed within the set time, version change will fail and you can retry the operation with a bigger timeout or you can cancel

Only Restart Affected Components

This strategy prioritizes uptime by identifying and stopping only components that have changed while keeping all others running, replacing and then starting affected components.

Use this strategy when you want to prioritize uptime of unchanged components or you have made only minor processor configuration changes.

This works well for deployments with inbound connections and will keep your source processors running if they have not changed compared to the previous version.

Stop & Empty Queues

This strategy forces a version change by stopping all components, emptying all queues, changing flow version, and then starting all components.

Use this strategy only when you want to force a flow version change without keeping any processors running or attempting to process queued data.

All processors will be stopped and all queues will be emptied as part of this strategy.

Change flow version steps

Learn how to change the flow definition version of a running flow deployment. Using the ‘change flow version’ capability eliminates the need to terminate and re-create deployments when you want to deploy a new version of your flow definition.

You must have DFFlowAdmin permission.
There is at least one more version for the same flow definition present in the catalog
The state of the flow deployment is either Good Health or Stopped.
You have read the applicable restrictions and version change strategies.

Open Cloudera DataFlow by clicking the DataFlow tile in the CDP sidebar.
Click Deployments from the left navigation pane.
Select a flow deployment to expand the Deployment Details pane and get deployment, KPI, system metrics, alerts, and event history information.
In Deployment Details, click Actions > Manage Deployment.
You are redirected to the Deployment Manager page.
Click Actions in the Deployment Manager page.
Select Change Flow Version.
The Change Flow Version modal window opens. It shows the list of available flow versions. The current version is grayed out.
tip
You can filter versions by name or by tags.
Select the flow version you want to change to and click Continue .
Review a summary of the configuration changes caused by the version change and make any necessary edits from the left pane.
Select a flow version change strategy.
The available options are:
- Stop & Process Data - If you select this strategy, you can set the maximum wait time for data to be processed and queues to be emptied before the request timed out. The default value is 15 minutes.
  
  note
  If you initiate flow version change with this strategy on a stopped flow and there is no data in the queues, flow version change instantly happens regardless of the wait time you configured. However if there is data left in the queues of the stopped flow, the change operation instantly fails.
- Only Restart Affected Components
  note
  If you initiate flow version change with this strategy and there are queued flow files on any connection that is going to be removed in the new flow version, the version change will fail.
- Stop & Empty Queues - If you select this strategy, you must accept potential data loss by selecting I understand and choose to proceed with the configuration as is.
Click Change Flow Version.

After you click Deploy, you are redirected to the Alerts tab in the Flow Details view where you can track how the version change progresses.

Change flow version troubleshooting

If changing the flow version fails, you can investigate the root cause in CDF and in NiFi. In ceerain cases you can retry or cancel the operation.

Check the cause of the failure on the Alerts tab of the flow details view.
If the information provided on the Alerts tab is insufficient, try checking the NiFi UI.

Select Actions > View In NiFi.
If version change fails during the preparatory phase, before any actual changes were made, you have the option to cancel. This will roll back your flow to the state before initiating the version change. CDF only offers this option when a rollback is possible.
Select Actions > Manage Deployment > Cancel Version Change.
Another option when version change fails during the preparatory phase is to retry the operation, possibly with a longer timeout or different parameter values in case incorrect parameter values caused the version change to fail.
Select Actions > Manage Deployment > Retry Version Change.