Fixed Issues in Atlas

Review the list of Apache Atlas issues that are resolved in Cloudera Runtime 7.2.17.

CDPD-54852: Backward compatibility for check provided for AttributeName in Parent and Child TypeDef

This patch provides backward compatibility for 2 changes mentioned: https://issues.apache.org/jira/browse/ATLAS-3872 Restrict typedef creation when a child type attribute conflicts with parent type attribute of same name https://issues.apache.org/jira/browse/ATLAS-4522 Updating typedef with new supertype should be allowed only if attributes are unique compared to other existing supertypes

CDPD-53717: Failed to extract ADLS Meta Data in Azure cdp_atlas_nat run

Fixed failed to extract ADLS Meta Data in Azure cdp_atlas_nat run

CDPD-50914: Atlas - Upgrade reactor-netty to 1.0.24+ due to CVE-2022-31684

Upgrade reactor-netty to 1.0.24+ due to CVE-2022-31684

CDPD-49792: Dynamic Index Recovery issues and improvements

Merged in CDH-7.2.17.x

CDPD-48720: The attributes of __AtlasAuditEntry type where not getting indexed in the Solr, because the entities of __AtlasAuditEntry gets created before the attributes gets indexed (TypeDef gets created).

This issue is resolved.

CDPD-49053: Atlas - Upgrade Tinkerpop to 3.5.4

Upgrade Tinkerpop to 3.5.4

CDPD-48166: Atlas - Upgrade snakeyaml due to CVE-2022-1471

Upgrade snakeyaml due to CVE-2022-1471

CDPD-48090: Atlas - Upgrade icu4j to 66.1+ due to CVE-2020-21913

Upgrade icu4j to 66.1+ due to CVE-2020-21913

CDPD-48034: Atlas - Upgrade jettison to 1.5.2 due to CVE-2022-45685 and CVE-2022-45693

Upgrade jettison to 1.5.2 due to CVE-2022-45685 and CVE-2022-45693

CDPD-48026: Atlas - Upgrade Woodstox to 5.4.0/6.4.0 due to multiple CVEs

Upgrade Woodstox to 5.4.0/6.4.0 due to multiple CVEs

CDPD-47988: Atlas - Upgrade Netty to 4.1.86.Final due to CVE-2022-41881, CVE-2022-41915

Upgrade Netty to 4.1.86.Final due to CVE-2022-41881, CVE-2022-41915

CDPD-47912: Atlas - Upgrade moment.js to 2.29.4 due to CVE-2022-24785, CVE-2022-31129

Updated moment.js version to 2.29.4

CDPD-46519: aws-s3-extractor.sh fails with ClassNotFoundException: com.fasterxml.jackson.core.util.JacksonFeature

Fixed aws-s3-extractor.sh Atlas script fails in 7.2.16 AWS test runs

CDPD-46409: Atlas - Upgrade Spring-security to 5.6.9/ 5.7.5 due to CVE-2022-31692, CVE-2022-31690

Upgrade Spring-security to 5.7.5 due to CVE-2022-31692, CVE-2022-31690

CDPD-45097: Atlas - Prevent re-indexing of deleted relationship edges to improve entity-update execution process

Implementing this fix along with fix in CDPD-36992 together would ensure that deleted relationship edges are not reattempted for deletion and re-indexing is not done for such edges. As a result, execution time for messages piling on the Atlas hook (Kafka lag) would be reduced drastically leading to significant performance improvement.

CDPD-44811: [PbC] Atlas - Replace log4j 1.x with log4j2

Merge to CDH 7.2.17 (7.2.17.x), pre-cdpd-master (7.2.17) branch,

CDPD-44104: Every hive insert generates an Atlas audit event

As per the current architecture, Atlas maintains audit data for all DML events also. By introducing "DML audit filter" feature, Atlas can skip all the DML event processing which improves performance by reducing unnecessary processing efforts from Atlas. Eventually, there will not be a need to store any such audit events which will reduce use of storage. The DML audit filter will be enabled by default with the below explained configuration. If the user wants to have audit data for DML events, it can be configured using the same configuration. Filtering DML events will be handled by Atlas hook, DML audit filter configuration should be enabled at Hive hook. Note that DML Audit filter is enabled by default, set to false to disable it Hive configuration Property : Hive Service Advanced Configuration Snippet (Safety Valve) for atlas-application.properties Configuration value : atlas.hook.hive.skip.dml.messages=true

CDPD-40822: Atlas - Upgrade Spring Framework to 5.3.20 due to CVE-2022-22971, CVE-2022-22970

Upgrade Spring Framework to 5.3.20 due to CVE-2022-22971, CVE-2022-22970

CDPD-36991: Backward compatibility for check provided for AttributeName in Parent and Child TypeDef

This patch provides backward compatibility for 2 changes mentioned: https://issues.apache.org/jira/browse/ATLAS-3872 Restrict typedef creation when a child type attribute conflicts with parent type attribute of same name https://issues.apache.org/jira/browse/ATLAS-4522 Updating typedef with new supertype should be allowed only if attributes are unique compared to other existing supertypes

CDPD-35438: When classification is created with multiple super types having same attributes , Atlas doesn't throw an exception

This patch restricts classification typedef creation when a child type attribute conflicts with parent type attribute of same name.

CDPD-31728: Audits : For db create , there are 2 update audits instead of 1 create 1 update

This patch will check for the Shell entities created and then assign the Entity status to the Audit entry.

CDPD-29307: Kafka producer entity stays in incomplete state in Atlas

The Kafka-Atlas plugin now fully creates Producer and Consumer entities and won't generate incomplete ones.

CDPD-48122: Operations like admin/audits, admin/purge fail with a 500 internal server error message

Before the update, the attributes of __AtlasAuditEntry type where not getting indexed in the Solr, because the entities of __AtlasAuditEntry gets created before the attributes gets indexed (TypeDef gets created).