Data Provenance
NiFi keeps a very granular level of detail about each piece of data that it ingests. As the data is processed through the system and is transformed, routed, split, aggregated, and distributed to other endpoints, this information is all stored within NiFi's Provenance Repository. In order to search and view this information, we can select Data Provenance from the Global Menu. This will provide us a table that lists the Provenance events that we have searched for:
Initially, this table is populated with the most recent 1,000 Provenance Events that have
occurred (though it may take a few seconds for the information to be processed after the
events occur). From this dialog, there is a Search
button that allows the
user to search for events that happened by a particular Processor, for a particular FlowFile
by filename or UUID, or several other fields. The nifi.properties
file
provides the ability to configure which of these properties are indexed, or made searchable.
Additionally, the properties file also allows you to choose specific FlowFile Attributes that
will be indexed. As a result, you can choose which Attributes will be important to your
specific dataflows and make those Attributes searchable.