Fixed Issues in Atlas

Review the list of Apache Atlas issues that are resolved in Cloudera Runtime 7.2.6.

OPSAPS-58847: Atlas TLS protocol excludes changed to TLSv1 and TLSv1.1 instead of earlier TLSv1.2
This issue is now resolved.
CDPD-1138: Spark Atlas Connector tracks column-level lineage
This issue is now resolved.
CDPD-11790: Shell entity is not resolved to the Complete entity under certain conditions.
Shell entities with duplicate qualifiedName are no longer created when processing message from Spark Atlas Connector. This issue is now resolved.
CDPD-14031: In the Spark Atlas Connector, few S3 entities are created using the V1 S3 model instead of the updated V2 S3 model.
Use Atlas S3 v2 models in Spark Atlas Connector. This issue is now resolved.
OPSAPS-57947: Kafka Broker SSL configuration is not correct in High Availability mode.
When deploying the DataHub in High Availability mode, some of the Ranger and Atlas configurations are not computed correctly. In particular in Atlas, the SSL properties and the REST URL services depending on Ranger.
CDPD-13645: Contains sortBy=name, 'name' attribute is not in hive_storagedesc definition. Also if sortBy is not passed, default attribute is name
Validated if sortBy attribute passed in the request is present in relationship end definition, if not present, ignore sorting.
Validated if sortBy attribute is not passed, default attribute name is present in relationship end definition, if not present, ignore sorting.
1) Fixed quick search aggregation metrics when filtered with System Attributes
2) Fixed quick search aggregation metrics when filtering with more than one filter
3) Fixed quick search aggregation metrics when filtering with negation operator
CDPD-13805: Relationship api request will have provision to specify attributes to be present in search result.
Example Request: /v2/search/relationship?guid=ac9e04cc-f927-4334-af08-c83bc3733f5b&relation=columns&sortBy=name&sortOrder=ASCENDING&attributes=dcProfiledData
1. Filter Search Results with multiple entity type by 'comma' separated string of typeName in the request Eg. "typeName": "hive_table,hive_db".
2. Filter Search Results with multiple tag by 'comma' separated string of tags in the request Eg. "classification": "tag1,tag2".
CDPD-13199: Incorrect attribute values in bulk import
When importing Business Metadata attribute assignments, Atlas used only the last assigned attribute value instead of individual values for each entity in the import list.
CDPD-372: All Spark Queries from the Same Spark Session were included in a Single Atlas Process
A Spark session can include multiple queries. When Atlas reports the Spark metadata, it creates a single process entity to correspond to the Spark session. The result was that an Atlas lineage picture showed multiple input entities or multiple output entities for a process, but the inputs and outputs were only related by the fact that they were included in operations in the same Spark session. In this release, the Spark Atlas Connector produces a spark_application entity for each Spark job. Each data flow produced by the job creates a spark_process entity in Atlas, which tracks the actual input and output data sets for that process. For more information, see Spark metadata collection.
CDPD-12620: Migration progress bar not refreshed
During the import stage of Cloudera Navigator to Apache Atlas migration, the migration progress bar does not correctly refresh the migration status. The Statistics page in the Atlas UI displays the correct details of the migration.
This issue is resolved.
CDPD-10151: Short Spark job processes may be lost
In rare occasions, it is possible for events captured for Atlas by the Spark Atlas Connector to be dropped before the metadata reaches Atlas. It is more likely that an event is lost in very short running jobs.
This issue is resolved.
CDPD-6042: Hive Default Database Location Incorrect in Atlas Metadata
The location of the default Hive database as reported through the HMS-Atlas plugin does not match the actual location of the database. This problem does not affect non-default databases.
This issue is resolved.
CDPD-4662: Lineage graph links not working
Atlas lineage graphs do not include hyperlinks from assets to the assets' detail pages and clicking an asset does not provide an error in the log. Clicking an edge in a graph still provides access to edge behavior options such as controlling how classifications propagate.
This issue is resolved.
CDPD-3700: Missing Impala and Spark lineage between tables and their data files
Atlas does not create lineage between Hive tables and their backing HDFS files for CTAS processes run in Impala or Spark.
This issue is resolved.
Additional Cloudera JIRAs: CDP-5027, CDPD-3700, and IMPALA-9070

Apache patch information

Apache patches in this release. These patches do not have an associated Cloudera bug ID.

  • ATLAS-4055: Basic search doesn't work if the search text has special characters
  • ATLAS-3984: Add UI Date Timezone and UI Date Format in Session api
  • ATLAS-3935: Use Audit framework to capture audit entries for Import/Export operations
  • ATLAS-3957: Use Audit framework to capture audit entries for Server Start and Server In Active mode (HA)
  • ATLAS-3954: UI: Type system property table improvement ATLAS-3991: UI: Handlebar helper number format issue ATLAS-3990: UI: When user clicks to view hive-table details, shown some wrong tabs ATLAS-3943: UI: Show Import/Export operations in administration audit. ATLAS-3986: UI Allow user to update the date format from JAVA property file ATLAS-3986: UI Allow user to update the date format from JAVA property file.
  • ATLAS-3992: UI: Upgrade to JQuery 3.5.1 (Classic UI Jquery-UI issue)
  • ATLAS-4000: updated jetty version to 9.4.31.v20200723
  • ATLAS-3999: updated build to remove copying of dist artifacts into directories
  • ATLAS-3427: Atlas Hook Enhancements for improved resiliancy.
  • ATLAS-3938: Deletion of non existing hive entities (CDPD-17225)
  • ATLAS-3995: Atlas should support additional keystore/truststores beside JKS
  • ATLAS-3989: Updated Export-Import Audits Writer to use metadata.namespace.
  • ATLAS-3978: In Administration, Audits filter for 'OR' condition does not work
  • ATLAS-3977: Patch handler for addressing deleted entities after migration.
  • ATLAS-3981: UI: Create Entity button is shown for a user without create entity permission in rangeri (CDPD-13925) ATLAS-3979: Beta UI : Removing one of the filters with an attribute name removes all of the filters with that attribute name. (CDPD-7259)
  • ATLAS-3983: solr index query escape character handling
  • ATLAS-3971: Move authorization check for Business Metadata before type exist check.
  • ATLAS-3964: Atlas UI displays large numbers incorrectly
  • ATLAS-3953: Export: ZipSink: Specify character endcoding when writing to ZIP file.
  • ATLAS-3968: Refactor the typedef API authoriation error msg from type Name to type guid
  • ATLAS-3962: Include business metadata def header in typdefs headers API
  • ATLAS-3955: Read Type Auth, UI : detailsPage doesn't load for user who has read permission for entity but no read auth for entity's type
  • ATLAS-3949: Relationship search API, add parameter to get classification attributes in search results (CDPD-16124)
  • ATLAS-3952: Authorize SuperTypes and depend entityType for type-read access while creating Classificationdef.
  • ATLAS-3911: UI: Type system managment
  • ATLAS-3827: UI: Use moment date format instead of default date format.
  • ATLAS-3945: UI: Entity details page, Show N/A for date if date value is 0 or null(CDPD-17206)
  • ATLAS-3947: Skip authorization for read of _ALL_ENTITY_TYPES and _ALL_CLASSIFICATION_TYPES types
  • ATLAS-3946: Filter TypeDefs in Metrics API and show types data accordingly
  • ATLAS-3934: docker scripts to build and run Apache Atlas in containers
  • ATLAS-3939: added profile berkeley-solr
  • ATLAS-3875: updated sample client to improve logging
  • ATLAS-3948: Entity Creation: Index Consistency: Java Patch Handler: Provide Option to Disable
  • ATLAS-3944: Logs of import scripts should go to Atlas default log directory
  • ATLAS-3583: Use Audit framework to generate audit entries for TypeDefs CREATE, UPDATE and DELETE
  • ATLAS-3942: delete-type fails if the user doesn't have permission for type-read
  • ATLAS-3920: Update joda-time to version 2.10.latest.
  • ATLAS-3925: UI: (Regression) Properties with value Null not showing N/A
  • ATLAS-3898: authorization updates to require TYPE_READ permission for type-def read