Use case overview

Persistence providers in NiFi Registry store and manage metadata and versioned flow information. Each type of versioned item, such as a versioned flow or extension bundle, has its own persistence provider. Each persistence provider has its own configuration parameters, which can be configured in an XML file specified in You can use NiFi CLI to export and import flows in NiFi Registry when switching from one persistence provider to another, or for other use cases like syncing two different NiFi Registry instances.

Flow persistence providers store the content of the flows saved to the registry. In CDF for Data Hub, the Flow Management Data Hub cluster is configured to use the database provider as the flow persistence provider by default. DatabaseFlowPersistenceProvider uses a relational database for storing and managing the versioned flow information and metadata. GitFlowPersistenceProvider stores flow contents in a Git repository, leveraging the capabilities of Git, such as version control, collaboration, and auditing.

The GitFlowPersistenceProvider has the following limitations:

  • You cannot make changes to your flows in Git and commit these flows for NiFi Registry to pick up. All commits need to be done by NiFi Registry following actions through NiFi or NiFi Registry.

  • You cannot have multiple NiFi Registry instances sharing the same Git repository because Registry does not read from Git but only pushes for persistence.

  • Git repositories can grow in size over time, especially if they contain large files or a long history of commits. This can lead to increased storage requirements and slower operations, so it is important to manage repository size and consider strategies like using Git LFS (Large File Storage) for large binary files.

In this use case, you can see how to switch from DatabaseFlowPersistenceProvider to GitFlowPersistenceProvider with the help of NiFi CLI. You can find details on how to:
  • Export your flows from NiFi Registry using the CLI
  • Clean up the databases
  • Change the flow persistence provider
  • Import all flows to your local Git repository
  • Configure NiFi Registry so that versioned flows in NiFi are automatically saved to the Git repository.