Performance Monitoring Tools
Before we get to tuning our cluster, it helps to describe what we might actually want to monitor as well as any potential pain points.
Prior to switching over to the new Storm Kafka client, which leverages the new Kafka consumer API under the hood, offsets were stored in ZooKeeper. While the broker hosts are still stored in ZooKeeper, this is no longer true for the offsets which are now stored in Kafka itself. This is a configurable option, and you may switch back to ZooKeeper if you choose, but Metron is currently using the new defaults. With this in mind, there are some useful tools that come with Storm and Kafka that we can use to monitor our topologies.
Consumer group offset lag viewer
There is a GUI tool to make creating, modifying, and generally managing your Kafka topics a bit easier - see kafka-manager
Console consumer - useful for quickly verifying topic contents
For more information on the Storm user interface, see Reading and Understanding the Storm UI.