Create a Kafka Topic to Store your Events

You must create a Kafka topic to store the events that you plan to stream. You can create a topic from the command line or the from the SMM UI.

Kafka is a distributed event streaming platform that lets you read, write, store, and process events (also called records or messages) across many machines.

Example events are payment transactions, geolocation updates from mobile phones, shipping orders, sensor measurements from IoT devices or medical equipment, and much more. These events are organized and stored in topics. Very simplified, a topic is similar to a folder in a filesystem, and the events are the files in that folder.

So before you can write your first events, you must create a topic.

  • You have installed the trial version of CDP Private Cloud Base and setup the streaming cluster.
  1. To create a Kafka topic from the SMM UI:
    1. Go to the Cloudera Manager UI and navigate to the SMM > Topics page.
    2. Click Add New.
    3. Provide the following information:
      • Topic name
      • Number of partitions
      • Level of availability
      • Cleanup policy
    4. SMM has automatically set Kafka topic configuration parameters. To manually adjust them, click Advanced.
    5. Click Save when done.
  2. To create a Kafka topic from the command line:
    1. Open a terminal session and run:
      $ bin/ --create --topic quickstart-events --bootstrap-server localhost:9092
    2. Run the command without any arguments to display usage information.
    3. You can view details such as the partition count of the new topic:
      $ bin/ --describe --topic quickstart-events --bootstrap-server localhost:9092
      Topic:quickstart-events  PartitionCount:1    ReplicationFactor:1 Configs:
          Topic: quickstart-events Partition: 0    Leader: 0   Replicas: 0 Isr: 0
    For more Kafka command-line tools, see Kafka Command Line Tools.
Write a few events into the topic.