Configuring Atlas For Apache Kafka Notifications
To generate Apache Kafka notifications, you must modify certain Atlas properties. This change is required for Ranger integration or anytime there is a consumer of entity change notifications.
Modify the following Atlas properties by adding the prefix 'atlas.kafka'. These properties are located in the
conf/application.properties
file in the ranger-tagsync installation directory:Received from Tom's doc:
atlas.notification.embedded=true atlas.kafka.data=${sys:atlas.home}/data/kafka atlas.kafka.zookeeper.connect=localhost:9026 atlas.kafka.bootstrap.servers=localhost:9027 atlas.kafka.zookeeper.session.timeout.ms=400 atlas.kafka.zookeeper.sync.time.ms=20 atlas.kafka.auto.commit.interval.ms=1000 atlas.kafka.hook.group.id=atlas
Note For “atlas.kafka.bootstrap.servers” and “atlas.kafka.zookeeper.connect”, modify host names based on your cluster topology. The directory specified in “atlas.kafka.data” should exist.
Add an export of HADOOP_CLASSPATH which includes the required Atlas directories to the hive-env configuration export
HADOOP_CLASSPATH=/etc/atlas/conf:/usr/hdp/current/atlas-server/hook/hive:${HADOOP_CLASSPATH}
.Note It is important that the Atlas directories are prepended to the existing classpath.
Restart the Atlas and Hive services after the cluster is fully provisioned.