Upgrade Apache Kafka
Before you can upgrade Apache Kafka, you must have first upgraded your HDP components to the latest version (in this case, 2.4.0). This section assumes that you have already upgraded your components for HDP 2.4.0. If you have not already completed these steps, return to Getting Ready to Upgrade and Upgrade 2.3 Components for instructions on how to upgrade your HDP components to 2.4.0.
Upgrade each Kafka node one at a time. You can stop each Kafka broker and upgrade the component without downtime if you have enough replication for your topic.
Shut down the current Kafka daemon, switch to the new version, and start the daemon:
su - kafka -c "/usr/hdp/current/kafka-broker/bin/kafka stop" hdp-select set kafka-broker 2.4.0.0-$BUILD su - kafka -c "usr/hdp/current/kafka-broker/bin/kafka start"
To verify that the Kafka daemon joined the cluster, create a topic and submit it to Kafka. Send a test message for that topic, and then validate that it was received by a consumer.
If the upgrade process fails, follow the steps in "Downgrading Kafka" to return to your previous version of Kafka.