Fixed Issues in Atlas
This section lists the issues that have been fixed since the previous version.
- CDPD-13199: Incorrect attribute values in bulk import
- When importing Business Metadata attribute assignments, Atlas used only the last assigned attribute value instead of individual values for each entity in the import list.
- CDPD-372: All Spark Queries from the Same Spark Session were included in a Single Atlas Process
- A Spark session can include multiple queries. When Atlas reports
the Spark metadata, it creates a single process entity to correspond
to the Spark session. The result was that an Atlas lineage picture
showed multiple input entities or multiple output entities for a
process, but the inputs and outputs were only related by the fact that
they were included in operations in the same Spark session. In this
release, the Spark Atlas Connector produces a
spark_applicationentity for each Spark job. Each data flow produced by the job creates a
spark_processentity in Atlas, which tracks the actual input and output data sets for that process. For more information, see Spark metadata collection.