Cloudera Docs

Creating Kafka topic

After you create a Kafka cluster, you can create Kafka topics. You create Kafka topics in the environment where your Kafka cluster is running by using the Streams Messaging Manager web UI.

You must have created a Kafka cluster and a Machine User account, and granted environment access to the Machine User.

You create two Kafka topics in the tutorial:

  • machine-data

    Topic containing machine usage data in plain text format.

  • machine-data-avro

    Topic containing machine usage data in Avro format.

  1. Navigate to Management Console > Environments, and select the environment where your Kafka cluster is running.
  2. On the Data Hubs tab of your environment, select the Kafka cluster you created.
  3. Click Streams Messaging Manager on the Services pane to open the Streams Messaging Manager web UI.

    Streams Messaging Manager is the Cloudera tool to monitor and manage Kafka. You use it to create the topics you need.

  4. In the Streams Messaging Manager web UI, click the topics icon () on the left-hand bar.
  5. Click Add New.
  6. Enter the following details and click Save:

    • Topic name: machine-data

    • Partitions: 10

    • Availability: Maximum

    • Cleanup Policy: delete

  7. Click Add New to add a second topic.
  8. Enter the following details for the new topic and click Save:

    • Topic name: machine-data-avro

    • Partitions: 10

    • Availability: Maximum

    • Cleanup Policy: delete

In the Streams Messaging Manager Topics page, type machine in the search field to filter the topics. You must see both topics you created.