Migrate brokers by modifying broker IDs in meta.properties

Learn how to migrate brokers between hosts in a Kafka cluster by modifying broker IDs in meta.properties.

In case of catastrophic hardware failure brokers can be moved to a new host in a Kafka cluster. The following steps walk you through broker migration by modifying broker IDs in meta.properties. Compared to migration with the help of the kafka-reassign-partitions tool, this method involves less manual work, but requires modifying an internal configuration file.
  • Verify that the cluster is healthy.
  • Verify that all replicas are in sync.
  1. Start up the new broker as a member of the old cluster.

    This creates files in the data directory.

  2. Stop both the new broker and the old broker that it is replacing.
  3. Change broker.id of the new broker to the broker.id of the old one both in Cloudera Manager and in data directory/meta.properties.
  4. Optional: Run rsync to copy files from one broker to another.
  5. Start up the new broker.
    When started the new broker re-replicates data from the other nodes.
The Kafka broker is migrated to a new host.