Command Line Installation
Also available as:
PDF

Configuring Atlas For Apache Kafka Notifications

To generate Apache Kafka notifications, you must modify certain Atlas properties. This change is required for Ranger integration or anytime there is a consumer of entity change notifications.

  1. Modify the following Atlas properties by adding the prefix 'atlas.kafka'. These properties are located in the conf/application.properties file in the ranger-tagsync installation directory:

    Received from Tom's doc:

    atlas.notification.embedded=true
    atlas.kafka.data=${sys:atlas.home}/data/kafka
    atlas.kafka.zookeeper.connect=localhost:9026
    atlas.kafka.bootstrap.servers=localhost:9027
    atlas.kafka.zookeeper.session.timeout.ms=400
    atlas.kafka.zookeeper.sync.time.ms=20
    atlas.kafka.auto.commit.interval.ms=1000
    atlas.kafka.hook.group.id=atlas
    [Note]Note

    For “atlas.kafka.bootstrap.servers” and “atlas.kafka.zookeeper.connect”, modify host names based on your cluster topology. The directory specified in “atlas.kafka.data” should exist.

  2. Add an export of HADOOP_CLASSPATH which includes the required Atlas directories to the hive-env configuration export HADOOP_CLASSPATH=/etc/atlas/conf:/usr/hdp/current/atlas-server/hook/hive:${HADOOP_CLASSPATH}.

    [Note]Note

    It is important that the Atlas directories are prepended to the existing classpath.

  3. Restart the Atlas and Hive services after the cluster is fully provisioned.