Behavioral Changes in Spark Atlas Connector

Behavioral changes denote a marked change in behavior from the previously released version to this version of Spark Atlas Connector.

Behavioral changes in Cloudera Runtime 7.3.1.500 SP3

There are no behavioral changes in Spark Atlas Connector in this release.

Summary: Spark plan is optional in Spark process entity sent to Atlas: Previous behavior:; Atlas would experience an Out-of-Memory error during processing if the spark_process entity's details and sparkPlanDescription fields contained too large strings.; New behavior:; A new Cloudera Manager property atlas.spark.plan.enabled was added to enable users to enable or disable the details and sparkPlanDescription field. (Default value: true.); When set to true (default), the fields can be used as before. When set to false, it reduces memory usage, but removes the fields from the Atlas UI.

There are no behavioral changes in Spark Atlas Connector in this release.