Monitoring your data flow

Learn about the different monitoring options for your Kafka ingest data flow.

You can monitor your data flow for information about health annd status, and for details about the operation of processors and connections. NiFi records and indexes data provenance information, so you can troubleshoot any issues in real time.

• Data statistics are rolled up on a summary screen (the little table icon on the top right toolbar which lists all the processors). You can use the MonitorActivity processor to alert you, if for example you have not received any data in your flow for a specified period of time.

• If you are worried about data being queued up, you can check how much data is currently queued. Process groups show the totals for all queues within them. This can often indicate if there is a bottleneck in your flow, and how far the data has got through in that pipeline.

• Another option to confirm that data has fully passed through your flow is to check out data provenance to see the full history of your data.