Data Provenance
NiFi keeps a very granular level of detail about each piece of data that it ingests. As the data is processed through the system and is transformed, routed, split, aggregated, and distributed to other endpoints, this information is all stored within NiFi's Provenance Repository. In order to search and view this information, we can select Data Provenance from the Global Menu. This will provide us a table that lists the Provenance events that we have searched for:
Initially, this table is populated with the most recent 1,000 Provenance Events that
have occurred (though it may take a few seconds for the information to be processed after
the events occur). From this dialog, there is a Search
button that allows
the user to search for events that happened by a particular Processor, for a particular
FlowFile by filename or UUID, or several other fields. The
nifi.properties
file provides the ability to configure which of these
properties are indexed, or made searchable. Additionally, the properties file also allows
you to choose specific FlowFile Attributes that will be indexed. As a result, you can choose
which Attributes will be important to your specific dataflows and make those Attributes
searchable.