Fixed Issues in Apache Atlas

Review the list of Atlas issues that are resolved in Cloudera Runtime 7.1.8.

CDPD-32226: Atlas is upgraded to use Log4j2 version 2.17.1 to avoid CVE-2021-44228.

CDPD-32432: Atlas is upgraded to use Logredactor version 2.0.13 to avoid CVE-2021-44228.

CDPD-34903: Log4j-core dependency is removed from Atlas to avoid CVE.

CDPD-35212: If Kerberos is enabled:

1. kinit -kt /cdep/keytabs/atlas.keytab atlas@ROOT.HWX.SITE

2. Add below to DEFAULT_JVM_OPTS in repair_index.py

3. Djavax.security.auth.useSubjectCredsOnly=false -Djava.security.auth.login.config=<path to atlas_jaas.conf>

If SSL is enabled, we need to make sure Solr cert or RootCA certificate is added which make use of below atlas-application properties

1. keystore.file <path to keystore jks file>

2. truststore.file <path to truststore jks file>

3. cert.stores.credential.provider.path <path to jceks file>

export HADOOP_CREDSTORE_PASSWORD=<password>

OPSAPS-64201: Default value set false for Atlas Server / Gateway Roles and Hook configurations for both 7.2.16 and 7.1.8 CSD.'s

OPSAPS-62184: HDFS Lineage configurations are available and user can configure the Blacklist and Whitelist paths from Atlas configurations.

OPSAPS-61342: When Atlas Solr collections are created through CM, the number of shards for the collection is now configured to the number of live infra-solr nodes if default shards is 1 and no of live solr nodes are more than one.

OPSAPS-61195: Added error handling for this scenario, this will avoid throwing NPE exception.

OPSAPS-57415: Required Kafka topics for Atlas will be pre-created for fresh install on clusters for kerberos enabled environments.

CDPD-43761: This is expected behaviour since upgrade scenario doesn't create the newly added default policies.

CDPD-42633: ElasticSearch dependency is removed from Atlas.

CDPD-35792: Updated the jquery-ui library from 1.12.1 to 1.13.1.

CDPD-35609: Jersey version upgrade to 1.19.4

CDPD-30950: Instead of loading entire lineage, fetch lineage on demand to improve performance.

CDPD-30163: Atlas is upgraded to use commons-io version 2.11.0.

CDPD-29335: Handle null pointer exception when retrieving tasks using the admin/tasks endpoint.

CDPD-29307: The Kafka-Atlas plugin now fully creates Producer and Consumer entities and won't generate incomplete ones.

CDPD-28919: Atlas is upgraded to use junit version 4.13.1 to avoid CVE-2020-15250.

CDPD-28865: Allow AWS and ADLS path entities through Hive DDL filter.

CDPD-28151: Atlas is upgraded to use nimbus-jose-jwt version 9.8.1 to avoid CVE-2021-27568.

CDPD-26509: Create FS entity successfully for load data inpath.

CDPD-14241: Dynamically handle indexing(secondary persistence) failures.

CDPD-29663: Error while connecting topic with schema in Atlas

The error occurred when the Schema Registry tried to make a relationship in Apache Atlas between a schema and a non-existent corresponding topic. When the error occurred, an Atlas integration related exception showed up in the Schema Registry logs and a DB entry related to the Atlas task was left in a failed state. The error is safe to ignore.

CDPD-43058: Entity created via hook sometimes doesn't get ingested by Atlas, esp seen in HDFS Lineage script

By disabling and removing the async message processing - (atlas.notification.hook.asynchronous), the creation of entities is ensured after running the hdfs-lineage.sh script

ENGESC-17926: HBase goes down due to huge payload from Atlas

After an update in Apache Hue, enabling DSL search for Hue will not cause any issues related to heap space.

CDPD-27390: [Entity Audits] 'Propagated Classification Added' timestamp is < 'Entity Created' timestamp

The issue caused by the Propagated Classification Added timestamp being earlier than the Entity Created timestamp is invalid since the classification is propagated once the entity is created. Such inconsistencies are fixed by ensuring that timestamps are based on the current system time, reducing the likelihood of invalid timestamp relationships.