What's New in this Release

Key new features in this release include:

Streams Messaging Manager 2.0.0 is a major release that contains new features and bug fixes.

Kafka cluster replication monitoring capability

You can use Streams Replication Manager (SRM) to implement cross-cluster Kafka topic replication in SMM.

Cloudera extensions for SRM collect and aggregate Kafka replication metrics to make them available through REST APIs. SMM uses the REST APIs to display metrics. You can use the metrics to understand the state of your Kafka cluster replications and take corrective actions if needed.

You can view information about Kafka cluster replication health and activity by selecting the cluster about which you want information. SMM shows all replications where the selected cluster is the target cluster.

You can monitor the status of the replication, source cluster name, target cluster name, number of topics to be replicated, number of consumer groups, throughput of the replication, replication latency, and checkpoint latency.

For more information about Kafka cluster replication monitoring, see the Monitoring Kafka Cluster Replications guide.

End-to-end latency monitoring capability

Latency is the time taken by a consumer to consume a message that is produced in a topic.

You can use SMM to monitor end-to-end latency in topics. You can find details about the number of messages produced in a topic, the number of messages consumed from a topic, and latency details during the consumption of the messages in the following two graphs in the SMM UI:

Messages Consumed. The graph provides you the overall produced message count and consumed message count across all the consumer groups for a topic within the selected time range. Any discrepancy in the produced and consumed message count is highlighted in red.
End-to-end Latency. The End-to-end Latency graph provides you the details of the latency range and average latency in consuming the messages, which are produced in a particular topic, in milliseconds within the selected time range.

For more information about end-to-end latency monitoring, see the Monitoring End to End Latency guide.

New metrics for alert policies

The following metrics are added for alert policies:

TOPIC_PARTITION_BYTES_IN_PER_SEC. The metric is added for topic component type.
TOPIC_PARTITION_BYTES_OUT_PER_SEC. The metric is added for topic component type.
TOPIC PARTITION CONSUMPTION PERCENTAGE. The metric is added for topic component type.
MILLISECONDS LAPSED SINCE CONSUMER WAS ACTIVE. The metric is added for consumer component type.
REQUEST HANDLER POOL AVG IDLE PERCENT. The metric is added for broker component type.
NETWORK PROCESSOR AVG IDLE PERCENT. The metric is added for broker component type.

For more information about metrics for alert policies, see the Managing Alert Policies guide.