Command Line Installation
Also available as:
PDF
loading table of contents...

Configuring Atlas For Apache Kafka Notifications

To generate Apache Kafka notifications, you must modify certain Atlas properties. This change is required for Ranger integration or anytime there is a consumer of entity change notifications.

  1. Modify the following Atlas properties by adding the prefix 'atlas.kafka'. These properties are located in the conf/application.properties file in the ranger-tagsync installation directory:

    Received from Tom's doc:

    atlas.notification.embedded=true
    atlas.kafka.data=${sys:atlas.home}/data/kafka
    atlas.kafka.zookeeper.connect=localhost:9026
    atlas.kafka.bootstrap.servers=localhost:9027
    atlas.kafka.zookeeper.session.timeout.ms=400
    atlas.kafka.zookeeper.sync.time.ms=20
    atlas.kafka.auto.commit.interval.ms=1000
    atlas.kafka.hook.group.id=atlas
    [Note]Note

    For “atlas.kafka.bootstrap.servers” and “atlas.kafka.zookeeper.connect”, modify host names based on your cluster topology. The directory specified in “atlas.kafka.data” should exist.

  2. Add an export of HADOOP_CLASSPATH which includes the required Atlas directories to the hive-env configuration export HADOOP_CLASSPATH=/etc/atlas/conf:/usr/hdp/current/atlas-server/hook/hive:${HADOOP_CLASSPATH}.

    [Note]Note

    It is important that the Atlas directories are prepended to the existing classpath.

  3. Restart the Atlas and Hive services after the cluster is fully provisioned.