Cloudera Docs

Topics and Groups Subcommand

Learn how to use the topics and groups subcommand of the srm-control command line tool.

The topics and groups subcommands are used to manipulate the topic or group allowlist (whitelist) and denylist (blacklist). Both subcommands support the same set of command options.

If you are modifying allow and denylists which target newly created topics or consumer groups, changes made with the srm-control tool may not be instantaneous A topic or group needs to be discovered by SRM before it can be added to or removed from white or blacklists. New topic and group discovery happens every 10 minutes by default. As a result, you may need to wait up to 10 minutes until you can see the changes made.

Add topics or groups to a whitelist:
srm-control topics --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --add [TOPIC1],[TOPIC2]
srm-control groups --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --add [GROUP1],[GROUP2]
Remove topics or groups from a whitelist:
srm-control topics --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --remove [TOPIC1],[TOPIC2]
srm-control groups --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --remove [GROUP1],[GROUP2]
Add topics or groups to a blacklist:
srm-control topics --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --add-blacklist [TOPIC1],[TOPIC2]
srm-control groups --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --add-blacklist [GROUP1],[GROUP2]
Remove topics or groups from a blacklist:
srm-control topics --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --remove-blacklist [TOPIC1],[TOPIC2]
srm-control groups --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --remove-blacklist [GROUP1],[GROUP2]
Specifying topics or groups is also possible with regular expressions. The following example adds all topics to the whitelist, meaning that every topic on the source cluster will be replicated to the target cluster.
srm-control topics --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --add ".*"

In addition to adding or removing items, you can also use the tool to look at the contents of a black or whitelist.

srm-control topics --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --list

Client Override Options

The topics and groups subcommands support a number of client override options. Client override options allow users to temporarily specify or override configuration properties used for replication. These options also enable users to issue srm-control commands even if the SRM’s configuration file is not available on the host that the command is being issued from. While it is possible to specify a range of properties with the client override options, and they can prove to be a powerful tool in certain scenarios, Cloudera recommends that you use Cloudera Manager to manage client configuration options.

The following client override options are available:
  • --bootstrap-servers: Specifies the bootstraps servers.
  • --producer-props: Specifies producer configuration properties.
  • --consumer-props: Specifies consumer configuration properties.
  • --props: Specifies client configuration properties.

A simple example of using client override options is when you want to change the bootstrap server. This can be done in two ways.

You can specify the bootstrap server with the --bootstrap-servers option. 
srm-control --bootstrap-servers localhost:9092 topics --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --list
Alternatively, you can also use the --props option together with the bootstrap.servers Kafka property to define the bootstrap server.
srm-control --props bootstrap.servers=localhost:9092 topics --source [SOURCE_CLUSTER] --list