Known Issues in Streams Replication Manager
This topic describes known issues for using Streams Replication Manager in this release of Cloudera Runtime.
Known Issues
- SRM does not sync re-created source topics until the offsets have caught up with target topic
- Messages written to topics that were deleted and re-created are not replicated until the source topic reaches the same offset as the target topic. For example, if at the time of deletion and re-creation there are a 100 messages on the source and target clusters, new messages will only get replicated once the re-created source topic has 100 messages. This leads to messages being lost.
- SRM may automatically re-create deleted topics
- If
auto.create.topics.enable
is enabled, deleted topics are automatically recreated on source clusters.
- CSP-462: Replication failing when SRM driver is present on multiple nodes
- Kafka replication fails when the SRM driver is installed on more than one node.
- CDPD-11074: The
srm-control
tool can be called without--target
- The
srm-control
tool can be initialized without specifying the--target
option. If the tool is called this way it will fail to run correctly.
- CDPD-13864 and CDPD-15327: Replication stops after the network configuration of a source or target cluster is changed
- If the network configuration of a cluster which is taking part in a replication flow is changed, for example, port numbers are changed as a result of enabling or disabling TLS, SRM will not update its internal configuration even if SRM is reconfigured and restarted. From SRM’s perspective, it is the cluster identity that has changed. SRM cannot determine whether the new identity corresponds to the same cluster or not, only the owner or administrator of that cluster can know. In this case, SRM tries to use the last known configuration of that cluster which might not be valid, resulting in the halt of replication.
- CSP-956: Topics or groups added to white or blacklists are not
returned when using
srm-control --list
- When polling the
srm-control.<alias>.internal
internal configuration topic, it may happen that not all records are returned at once. It can happen that the first poll only returns a single message. Remaining messages are only returned on a subsequent poll. As a result, only parts of the configuration are picked up. This causes thesrm-control
tool and the SRM driver to behave erratically as they are unable to read the full white and blacklists from the configuration topic.
- CDPD-11709: Blacklisted topics appear in the list of replicated topics
- If a topic was originally replicated but was later blacklisted,
it will still appear as a replicated topic under the
/remote-topics
REST API endpoint. As a result, if a call is made to this endpoint, the blacklisted topic will be included in the response. Additionally, the blacklisted topic will also be visible in the SMM UI. However, it's Partitions and Consumer Groups will be 0, its Throughput, Replication Latency and Checkpoint Latency will show N/A.
- CDPD-18300: SRM resolves configuration provider references in its internal configuration topic
- SRM saves its internal configuration topic with fully resolved properties. This means that even configuration provider references are resolved. Sensitive information can be emitted into the configuration topic this way.
- CDPD-22094: The SRM service role displays as healthy, but no metrics are processed
-
The SRM service role might encounter errors that make metrics processing impossible. An example of this is when the target Kafka cluster is not reachable. The SRM service role does not automatically stop or recover if such an error is encountered. It continues to run and displays as healthy in Cloudera Manager. Metrics, however, are not processed. In addition, no new data is displayed in SMM for the replications.
- CDPD-22389: The SRM driver role displays as healthy, but replication fails
-
During startup, the SRM driver role might encounter errors that make data replication impossible. An example of this is when one of the clusters added for replication is not reachable. The SRM driver role does not automatically stop or recover if such an error is encountered. It will start up, continue to run, and display as healthy in Cloudera Manager. Replication, however, will not happen.
- CDPD-23683: The replication status reported by the SRM service role for healthy replications is flaky
- The replication status reported by the SRM service role is flaky. The replication status might change between active and inactive frequently even if the replication is healthy. This status is also reflected in SMM on the replications tab.
Limitations
- SRM cannot replicate Ranger authorization policies to or from Kafka clusters
- Due to a limitation in the Kafka-Ranger plugin, SRM cannot
replicate Ranger policies to or from clusters that are configured to use Ranger for
authorization. If you are using SRM to replicate data to or from a cluster that uses
Ranger, disable authorization policy synchronization in SRM. This can be achieved by
clearing the Sync Topic Acls Enabled
(
sync.topic.acls.enabled
) checkbox.