Offsets Subcommand

Learn how to use the offsets subcommand of the srm-client command line tool.

SRM automatically translates consumer group offsets between clusters. The offset mappings are created by SRM, but are not applied to the consumer groups of the target cluster directly. Consumers can be migrated from one cluster to another without losing any progress by using the offsets subcommand on the target cluster to export the translated offsets of the source cluster. For example:

srm-control offsets --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --group [GROUP1] --export > out.csv

Exported offsets can then be applied to the consumer groups in the target cluster with the kafka-consumer-groups tool. For detailed steps on cluster migration, see Migrating Consumer Groups Between Clusters.

The offsets subcommand reads configuration properties from a configuration file. When not specified otherwise, it uses the default configuration file. If required, you can use the --config option to specify a different configuration file.

srm-control --config [path/to/srm.proerpties] offsets --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --group [GROUP1] --export > out.csv

Client Override Options

The offset subcommand supports client override options. Client override options allow users to temporarily specify or override configuration properties. These options also enable users to issue srm-control commands even if the SRM’s configuration file is not available on the host that the command is being issued from. While it is possible to specify a range of properties with the client override options, and they can prove to be a powerful tool in certain scenarios, Cloudera recommends that you use the srm.properties configuration file to manage client configuration options.

The following client override options are available:

--bootstrap-servers: Specifies the bootstraps servers.
--props: Specifies client configuration properties.

A simple example of using client override options is when you want to change the bootstrap server. This can be done in two ways.

You can specify the bootstrap server with the --bootstrap-servers option.

srm-control --bootstrap-servers localhost:9092 offsets --source [SOURCE_CLUSTER] --group [GROUP] --export > out.csv

Alternatively, you can use the --props option together with the bootstrap.servers Kafka property to define the bootstrap server.

srm-control --props bootstrap.servers=localhost:9092 offsets --source [SOURCE_CLUSTER] --group [GROUP] --export > out.csv