Monitoring Kafka consumers

By monitoring Kafka consumer groups, you can track active and passive consumer groups, or all consumer groups, which use the default internal __consumer_offsets topic to store the consumed offset information. You can track additional details about consumer groups. You can also track details including number of consumers and consumer instances included in a group and consumer group lag in the consumer group profile.

Streams Messaging Manager (SMM) displays consumer groups that have offsets stored in Kafka's internal topic __consumer_offsets, which is also the default store if the auto.commit.enable property is set to true for consumers. SMM does not display consumer groups that have offsets stored anywhere else other than this default store.

Viewing summary information about consumer groups

The Overview page gives you summary information about consumer groups on the right side of the page. You can use the Active, Passive, and All tabs to view consumer groups only in the Active or Passives, or all of the consumer groups, which use the default internal __consumer_offsets topic to store the consumed offset information. Use the Lag tab to sort consumer groups based on ascending or descending amounts of lag.

Viewing details about a consumer group

To access detailed consumer group information:

From the left navigation pane, click Consumer Group.
Identify the consumer group about which you want information. You can either scroll through the list of consumer groups, or use the Search bar at the top left of the page.
Click the green hexagon at the left of the consumer group to view details.

Viewing the consumer group profile

The Consumer Group profile displays detailed information about each consumer group, including:

The number of consumers included in the group.
The number of consumer instances in the group.
Details about consumer group lag.

To access the Consumer Group profile:

From the Consumer Group page, select the consumer group for which you want to view the profile.
Click the profile icon in the upper right of the Consumer Group tile.

Resetting consumer offset

To reset the offset of a consumer group, perform the following steps:

From the left navigation pane, click Consumer Group.
Choose the consumer group for which you want to reset the offset, and click the Profile icon.

note
Resetting offsets is only possible for those groups whose state is Empty or Dead. Attempting to reset offset of any other group results in an error.
In the Metrics page, click Actions > Reset offset.
The Reset offsets for consumer group dialog appears.
Select the topic and partition(s) you want to reset.
You can use the arrow beside each topic to display the partitions of that topic, and select partition(s) as required. You can also select all topics and all related partitions by selecting the All topics option. You can use the Filter option to find a specific topic.
Select the reset option from the Reset mode.
The available options are as shown in the following image:
Click Apply.