Spark Connector configuration in Apache Atlas

Learn to configure the Spark Atlas Connector to prevent sending the attributes details and sparkPlanDescription. This helps you avoid out-of-memory errors in Atlas if the query plan details contained a large amount of text up-to megabytes.

1. Go to Cloudera Manager > Clusters > Atlas > Configuration > Atlas Client Advanced Configuration Snippet (Safety Valve) for atlas-conf/atlas-client.properties.
2. Set the atlas.spark.plan.enabled parameter to false.

Setting this parameter to false prevents sending the attributes details and sparkPlanDescription in the Spark process entity. This greatly reduces the memory consumption.