Chapter 3. Upgrading NiFi

	Important
If you are upgrading a cluster, All Nodes must be upgraded and stopped before starting new Apache NiFi nodes. Your cluster cannot contain nodes with different versions.

Preserve Customizations Prior to Upgrade

Preserve your custom processors

If you have written custom processors, you can preserve them during the upgrade if you store them in a central location.

Create a second library directory, called custom_lib.
Move your custom processors to this new lib directory.

Add a new line to the nifi.properties file to specify this new lib directory:

nifi.nar.library.directory=./lib
nifi.nar.library.directory.custom=/opt/configuration_resources/custom_lib

Preserve your custom NAR files

If you have customized any of the default NiFi Archive (NAR) files, upgrading NiFi overwrites these changes. To preserve the customization:

Identify and save the changes you made to the default NAR files.
Perform your NiFi upgrade.
Implement the same NAR file changes in your new NiFi instance.
Start a dataflow.

Install the new NiFi version

About This Task

Install the new NiFi into a directory parallel to the existing NiFi installation.

Steps

Download the latest version of Apache NiFi. You can find the latest NiFi download location in the HDF Release Notes.
Uncompress the NiFi .tar file into a directory parallel to your existing NiFi directory:
```
tar -xvzf file-name
```
For example, if your existing NiFi installation is installed in /opt/nifi/nifi-1.0.0.2.0.2.0-17/, install your new NiFi version in a path similar to /opt/nifi/nifi-1.2.0.3.0.2.0-76/.
Create or update a symlink: /opt/nifi/current → /opt/nifi/<nifi build dir>
If you are upgrading a NiFi cluster, repeat these steps on each node in the cluster.

More Information

HDF Release Notes

Update the Configuration Files for Your New NiFi Installation

About This Task

Use the configuration files from your existing NiFi installation to manually update the corresponding properties your new NiFi deployment.

	Warning
	Do not copy configuration files from your existing NiFi version to the new NiFi version. The newer configuration files may introduce new properties that would be lost if you copy and paste configuration files.

Steps

Use the following table to update the configuration files located in <installation-directory>/conf.
Double check all configured properties for typos.

Table 3.1. Configuration File Changes

Configuration file Necessary changes

state-management.xml

For the “local-provider” value, verify the location of “state/local” directory.

If you have retained the default location (./state/local), copy complete directory tree to new NiFi. The existing NiFi should be stopped if you are copying this directory because it may be constantly writing to this directory while running.

Configuration best practices recommend that you move the state to an external directory like /opt/nifi/configuration-resources/ to facilitate future upgrades.

If this is a NiFi cluster, the “cluster-provider” ZooKeeper “Connect String" should be using the same external ZooKeeper as the existing NiFi installation.

Hortonworks does not support using the NiFi embedded ZooKeeper.

If you are using a NiFi cluster, ensure that the new “cluster-provider” ZooKeeper "Root Node" values exactly match your previous values.

login-identity-providers.xml

If you used a provider value in your existing the NiFi installation, copy the <provider> … </provider> configuration from the existing NiFi installation to this file in your new NiFi installation.

logback.xml

If you added any custom logging modification to the existing NiFi, make those same changes to the new NiFi installation.

bootstrap.conf

Use the existing NiFi bootstrap.conf file to update properties in new NiFi.

If you are upgrading to HDF 3.0.x, you should add the following line:

java.arg.100=-Djavax.security.auth.useSubjectCredsOnly=true

bootstrap-notification-services.xml

Update the values based on values from the existing NiFi file.

authorizers.xml

Copy the <authorizer> …. </authorizer> configured in the current NiFi to new the NiFi file.

If you are using “file-provider”, ensure that you copy the users.xml and authorizations.xml files from the current to the new NiFi.

Configuration best practices recommend creating a separate location outside of he NiFi base directory for storing such configuration files: for example, /opt/nifi/configuration-resources/. If you are storing these files in a separate directory, you do not need to move them. Instead, ensure that the new NiFi is pointing to the same files.

nifi.properties

Use the current file to populate the same properties in the new NiFi nifi.properties file.

This file contains the majority of NiFi configuration settings, so ensure that you have copied the values correctly.

If you followed NiFi best practices, the following properties should be pointing to external directories outside of the base NiFi installation path.

If the properties point to directories inside the NiFi base installation path, you must copy the target directories to the new NiFi. Stop your current NiFi installation before you do this.

nifi.flow.configuration.file=

If you have retained the default value, (./conf/flow.xml.gz), copy flow.xml.gz from the current to the new NiFi base install conf directory.

Alternately, you can copy to an external location and update the property value to point there.

nifi.flow.configuration.archive.dir=

Same applies as above if you want to retain archived copies of the flow.xml.gz.

nifi.database.directory=

Best practices recommends that you use an external location for each repository. Point new NiFi at same external database repository location.

nifi.flowfile.repository.directory=

Best practices recommends that you use an external location for each repository. Point new NiFi at same external database repository location.

	Warning
	You may experience data loss if flowfile repositories are not accessible to new NiFi.

nifi.content.repository.directory.default=

Best practices recommends that you use an external location for each repository. Point new NiFi at same external database repository location.

Your current NiFi may have multiple content repos defined. Make sure exact same property names are used and point to appropriate matching content repo locations. For example:

nifi.content.repository.directory.content1=
nifi.content.repository.directory.conten2=

	Warning
	You may experience data loss if content repositories are not accessible to new NiFi.

	Warning
	You may experience data loss may if property names are wrong or the property points to wrong content repository.

nifi.provenance.repository.directory.default=

Best practices recommends that you use an external location for each repository. Point new NiFi at same external database repository location.

Your current NiFi might define multiple content repositories. . Ensure that your new installation uses the same property names and point to appropriate matching content repository locations.

For example:

nifi.provenance.repository.directory.provenance1=
nifi.provenance.repository.directory.provenance2=

	Note
	You may not be able to query old events if you have not moved the provenance repositories or did not update the properties correctly.

Restarting the dataflow after upgrade

About This Task

After you have installed and configured the new NiFi cluster, complete the upgrade process by configuring your new NiFi cluster to take over flow management tasks.

Steps

Set the directory and file ownership of your new NiFi instance to match the current owner and permissions of the existing NiFi instance.

If you have not already done so, stop your existing NiFi instance.

	Warning
	Do not start your new NiFi installation while the existing NiFi installation is still running. You may experience data loss if both NiFi configurations are pointing at any of the same external configuration files, repositories, or similar.

Start your new NiFi instance as the same user as for your existing NiFi instance.
If your existing NiFi instance was configured to run as a service, update any symlinks or service scripts to point to new NiFi version executables.