Setting up Atlas Kafka import tool
You can use the Atlas-Kafka import tool to load Atlas with Kafka topics.
The Kafka type definition in Atlas contains multiple fields. The import tool fills out the following field values:
clusterName - Contains the value provided by the
atlas.metadata.namespaceproperty in the application configuration file. The default value is
topic, name, description, URI - Contains the topic name.
qualifiedName - Contains the topic name and the cluserName which is joined by ‘@’. For instance,
my-topic@cm. This serves as the unique identifier for topics.
partitionCount - Displays the number of partitions of the topic.
To set up the Atlas - Kafka import tool, follow these steps:
- Select the Atlas service in Cloudera Manager.
- Deploy client configs: Actions > Deploy Client Configuration.
- If Ranger is enabled, create a Ranger policy with the user running the tool in
cm_atlas to allow the <user> to create, update,
delete, and read Kafka entities:
- Log into Ranger UI.
- Navigate to cm_atlas policies.
- Select the policy which has create_entity and delete_entity permissions.
- Add the kerberos user that is used with the
- SSH into a host where the client configuration is deployed.
- Run kinit as kafka if Kerberos is used to set up the Ticket Granting Ticket (TGT).
- Run the command
/opt/cloudera/parcels/CDH/lib/atlas/hook-bin/import-kafka.shto import Kafka topics into Atlas.