Importing Kafka entities into Atlas

Existing Kafka topics and clients can be imported into Atlas as entities using the Import Kafka Topics Into Atlas action available in Cloudera Manager. Importing Kafka entities allows you to manage, govern, and monitor your topics and clients in Atlas.

The following list of steps walk you through how you can import Kafka topics and clients as entities (metadata) into Atlas using the Import Kafka Topics Into Atlas action in Cloudera Manager. This action only imports existing topics and clients. Topics and clients created after the import is completed are not automatically imported into Atlas.

Continuous import can be enabled by configuring the Atlas hook in Kafka. For more information, see Configuring the Atlas hook in Kafka.

  • An Atlas service is deployed on the Kafka cluster or the data context cluster. The Atlas service must be started and must be in a healthy state.
  • You are logged in to Cloudera Manager as a user that can perform service and role level configuration changes (Cluster Administrator, Full Administrator, Configurator, Limited Cluster Administrator).
  • If authorization is enabled, ensure that the kafka user is authorized to create, update, delete, and read Kafka entities in Atlas.
  1. In Cloudera Manager select the Kafka service.
  2. Click Actions > Import Kafka Topics Into Atlas.
  3. Click Import Kafka Topics Into Atlas and wait until the process is finished.
Existing Kafka topics and clients imported into the Atlas service.
Access the Atlas dashboard to view, monitor, and manage imported Kafka entities.