Fixed Issues in Apache Atlas

Review the list of Atlas issues that are resolved in Cloudera Runtime 7.1.9.

OPSAPS-67782: ZDU | During rolling upgrade one ATLAS server failed to start while other server went fine
Exclusion of jackson databind from parent dependency
OPSAPS-67878: [CKP-2,3,4] Apply config injectors for all Atlas csd
Changes made in 7.1.8 CSD, for 7.1.9 CSD CM team has already taken care.
CDPD-56309: [Atlas] Download Search with Basic Search gives java.io.FileNotFoundException
Default value for the download directory will be read from system property "user.dir" instead of property "atlas.home"
CDPD-49053: Atlas - Upgrade Tinkerpop to 3.5.4
Upgrade Tinkerpop to 3.5.4
CDPD-48090: Atlas - Upgrade icu4j to 66.1+ due to CVE-2020-21913
Upgrade icu4j to 66.1+ due to CVE-2020-21913
CDPD-50022: Test failure - Atlas 'Service Unavailable' error
Circular dependency issue caused by repository.src.main.java.org.apache.atlas.tasks.TaskRegistry class is resolved by adding @Lazy annotation on @Component definition of this class.
CDPD-29307: Kafka producer entity stays in incomplete state in Atlas
The Kafka-Atlas plugin now fully creates Producer and Consumer entities and won't generate incomplete ones.
CDPD-57305: Atlas - Upgrade moment.js to 2.29.4 due to CVE-2022-24785, CVE-2022-31129
Updated moment.js version to 2.29.4
CDPD-45073: Implement aging for audits stored by Atlas.
Feature to reduce Atlas audit storage by aging out audit data, using different criteria like TTL and audit count
CDPD-58592: [CKP4 (same value)] Atlas not null filter on classification returns null values
Cause: It is because of Solr version upgrade, untill 8.4.1, Solr supported non empty string. Fix: For IndexQuery : ["" TO *] works to get nonEmpty field entities. For Inmemory Predicates: Used NonEmptyPredicate
CDPD-55098: 7.1.9 - Dynamic Index Recovery issues and improvements
Introduced AtlasClient API to perform index recovery
CDPD-50762: Regression : admin/audits , admin/purge fail with "[__AtlasAuditEntry.startTime] is not indexed in the targeted index [vertex_index]" 7.1.9
Cause: The attributes of AtlasAuditEntry type where not getting indexed in the Solr, because the entity of AtlasAuditEntry gets created before the attributes gets indexed. FIx: Ordering the typeDefChangeListeners helped, to create the AtlasAuditEntry typeDef first before auditing.
CDPD-59409: Tags are not getting synced from one node of rangertagsync
Log4j version incompatibility between Atlas and Ranger led to this issue. A temporary fix has been merged to avoid the exception.
CDPD-54645: Ranger tag sync for Iceberg table type
Added iceberg support for Ranger Tagsync
CDPD-59713: [backport] Indexed string field (solr.StrField) which is too large ERROR
Imapala process entities created by ImpalaHook saves query-string in name field. Since query-string can be large, you are getting the longer than the max error.
To store qualifiedName in name field instead of query-string
CDPD-49495: OOM issue with DSL search caching added by CDPD-27872
Fixed OOM seen after large number DSL search requests. This was caused by the fix for ATLAS-4347
CDPD-57277: Atlas - Upgrade Spring Framework to 5.3.27/6.0.8 due to CVE-2023-20861, CVE-2023-20860 and CVE-2023-20863
Upgrade Spring Framework to 5.3.27 from 5.3.21
CDPD-57433: Atlas - Upgrade gremlin shaded to 3.5.5+ due to jackson-databind CVEs
Upgrade gremlin shaded to 3.5.5 from 3.5.4
CDPD-49450: Atlas - Upgrade jettison to 1.5.4 due to CVE-2022-45685 and CVE-2022-45693
Upgrade jettison to 1.5.4 from 1.3.7.
CDPD-53745: commits in CDH-7.1.8.x but NOT IN CDH-7.1.9.x
This issue is fix in CDPD-52776.
CDPD-54846: Atlas: CVE-2023-24998-upgrade commons-fileupload library to version 1.5
Upgrade commons-fileupload library to version 1.5 from 1.3.3.
CDPD-54852: Backward compatibility for check provided for AttributeName in Parent and Child TypeDef
This patch provides backward compatibility for two changes mentioned: https://issues.apache.org/jira/browse/ATLAS-3872 Restrict typedef creation when a child type attribute conflicts with parent type attribute of same name https://issues.apache.org/jira/browse/ATLAS-4522. Updating typedef with new supertype should be allowed only if attributes are unique compared to other existing supertypes.
CDPD-49451: Atlas - Upgrade snakeyaml due to CVE-2022-1471
Upgrade snakeyaml to 2.0 from 1.33.
CDPD-50740: [7.1.9]Atlas - Upgrade azure-storage libraries due to CVE-2022-30187
Upgrade azure-storage-blob version to 12.9.0 from 12.20.2 Upgrade azure-storage-queue version to 12.7.0 from 12.15.2.
CDPD-55440: Atlas - Upgrade snakeyaml to 2.0 due to CVE-2022-1471
Upgrade snakeyaml to 2.0 from 1.33.
CDPD-58492: Atlas - Upgrade Netty Project to 4.1.94.Final due CVE-2023-34462
Upgrade Netty Project to 4.1.94.Final from 4.1.86.Final.
CDPD-49452: Atlas - Upgrade Netty to 4.1.86.Final due to CVE-2022-41881, CVE-2022-41915
Upgrade Netty to 4.1.86.Final from 4.1.77.Final.
CDPD-57685: Atlas docs failures in CDH builds
Added the library "fix-esm" to handle the dependency update issue which was causing the build to fail.
CDPD-55617: Atlas - Upgrade Nimbus-JOSE-JWT to 9.24 due to CVEs coming from json-smart
Upgrade Nimbus-JOSE-JWT to 9.24 from 9.8.1.
CDPD-49978: Atlas - Upgrade icu4j to 66.1+ due to CVE-2020-21913
Upgrade icu4j to 66.1.
CDPD-58596: Large number of warn messages logged in tagsync log indicating process entity notification dropped
Added code to send version in the messages sent from Atlas to Ranger, so that the warning messages are not seen.
CDPD-53808: Atlas - Upgrade Spring Framework to 5.3.27/6.0.8 due to CVE-2023-20861, CVE-2023-20860 and CVE-2023-20863
Upgrade Spring Framework to 5.3.27 from 5.3.20.
CDPD-51181: Atlas - Upgrade Woodstox to 5.4.0/6.4.0 due to multiple CVEs
Upgrade Woodstox to 5.4.0 from 5.0.3.
CDPD-49980: Atlas - Upgrade Tinkerpop to 3.5.4
Upgrade Tinkerpop to 3.5.4 from 3.5.2.
CDPD-55876: Atlas - Upgrade Spring Security to 5.7.8+/5.8.3+/6.0.3+ due to CVE-2023-20862
Upgrade Spring Security to 5.8.3 to 5.7.5.
CDPD-55252: Atlas - Upgrade jackson-databind to 2.12.7.1/2.13.4.1+ due to CVE-2022-42003, CVE-2022-42004
Upgrade jackson-databind to 2.12.7.1.
CDPD-50741: [7.1.9]Atlas - Upgrade reactor-netty to 1.0.24+ due to CVE-2022-31684
Upgrade reactor-netty to 1.0.24.
CDPD-54864: Every hive insert generates an Atlas audit event
Introduced "DML audit filters" feature to skip unnecessary DML audit events using configuration property atlas.hook.hive.skip.dml.messages=true (Enabled by default).
CDPD-48641: Atlas - Support for JDK17 in all sub-components
Added JDK17 support for Atlas.
CDPD-56497: Regression : DSL queries redirected to passive server fails
DSL search request sent to Passive server having HTML encoded characters are now properly redirected to Active server.
CDPD-59758: [ST][Atlas] test_export_import_api_sanity test fails
Upgraded tinkerpop version from 3.5.5 to 3.5.6 : This resolved unsupported class major version error faced while running export API in JDK17 Runtime.
CDPD-56594: Lineage (spark_process) is not created for views created on Iceberg tables.
Added lineage support for creating views to Spark Atlas Connector for Spark 3 which is required for Iceberg tables.
CDPD-54990: The in-place migration of Hive table to Iceberg table with ALTER TABLE storage_handler using Beeline creates new iceberg_table entity but retains the old hive_table entity as is
Running the query no longer results with Atlas having two entities with same name but different types. One with hive_table and another with iceberg_table.

Apache patch information

  • ATLAS-4733
  • ATLAS-4754
  • ATLAS-4442
  • ATLAS-4735
  • ATLAS-4768
  • ATLAS-4762
  • ATLAS-4727
  • ATLAS-4571
  • ATLAS-4576
  • ATLAS-4757