Topics and Groups Subcommand
Learn how to use the topics
and groups
subcommand of
the srm-control
command line tool.
If you are modifying allow and denylists which target newly created topics or consumer
groups, changes made with the srm-control
tool may not be instantaneous A
topic or group needs to be discovered by SRM before it can be added to or removed from allow
or denylists. New topic and group discovery happens every 10 minutes by default. As a result,
you may need to wait up to 10 minutes until you can see the changes made.
srm-control topics --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --add [TOPIC1],[TOPIC2]
srm-control groups --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --add [GROUP1],[GROUP2]
srm-control topics --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --remove [TOPIC1],[TOPIC2]
srm-control groups --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --remove [GROUP1],[GROUP2]
srm-control topics --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --add-blacklist [TOPIC1],[TOPIC2]
srm-control groups --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --add-blacklist [GROUP1],[GROUP2]
srm-control topics --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --remove-blacklist [TOPIC1],[TOPIC2]
srm-control groups --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --remove-blacklist [GROUP1],[GROUP2]
In addition to adding or removing items, you can also use the tool to look at the contents of a deny or allowlist.
srm-control topics --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --list
Using Regular Expressions
Specifying topics or groups is also possible with regular expressions. This is done by
specifying a regular expression instead of topic names in the --add
,
add-blackilist
, or remove-blacklist
options.
srm-control
command using the .*
regular expression.
srm-control topics --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --add ".*"
You
can also use regular expressions to specify a set of topics. For example, assume that you
have many topics in your Kafka cluster, but only want to add the following topics to the
allowlist.testTopic1
testTopic2
testTopic3
In a case like this, you
can use the testTopic.*
or ^test.*
regular expressions to
add the
topics.srm-control topics --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --add "testTopic.*"
srm-control topics --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --add "^test.*"
Client Override Options
The topics and groups subcommands support a number of client override options. Client
override options allow users to temporarily specify or override configuration properties
used for replication. These options also enable users to issue srm-control
commands even if the SRM’s configuration file is not available on the host that the
command is being issued from. While it is possible to specify a range of properties with the
client override options, and they can prove to be a powerful tool in certain scenarios,
Cloudera recommends that you use Cloudera Manager to manage client configuration options.
--bootstrap-servers
: Specifies the bootstraps servers.--producer-props
: Specifies producer configuration properties.--consumer-props
: Specifies consumer configuration properties.--props
: Specifies client configuration properties.
A simple example of using client override options is when you want to change the bootstrap server. This can be done in two ways.
--bootstrap-servers
option.
srm-control --bootstrap-servers localhost:9092 topics --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --list
--props
option together with the
bootstrap.servers
Kafka property to define the bootstrap
server.srm-control --props bootstrap.servers=localhost:9092 topics --source [SOURCE_CLUSTER] --list