Using generic ignore patterns
Next to Hive metadata entities, generic ignore patterns can be set up to reduce resource consumption caused by the incoming metadata from any type of service.
Generic ignore patterns allow you to specify entities using regex patterns to be excluded from metadata capture and processing. This helps you to focus its metadata management efforts on relevant data entities, reducing clutter and improving the accuracy and efficiency of metadata operations.
- Apache Hbase Hook
- Apache Hive Metastore Hook
- Apache HiveServer2 Hook
- Apache Impala Hook
- Apache Spark Hook
Generic ignore patterns on the server side
Ignore pattern can be configured to ignore entities based on Type Name,
qualifiedName or both. You can configure the generic ignore pattern in Atlas
Server Advanced Configuration Snippet (Safety Valve) for
conf/atlas-application.properties:
- Go to .
- Search for conf/atlas-application.properties.
- Enter the following configurations combined with your regex expression:
- Configuration for
Type Name -
atlas.notification.consumer.preprocess.entity.type.ignore.pattern
- Configuration for
qualifiedName -
atlas.notification.consumer.preprocess.entity.ignore.pattern
- Configuration for
- Click Save Changes.
- Click to apply your changes.
Ignoring all Apache Hive entities
atlas.notification.consumer.preprocess.entity.type.ignore.pattern=hive_.*
Ignoring all entities with suffix _tmp in their name
atlas.notification.consumer.preprocess.type.ignore.pattern=.*\\..*_tmp.*
Ignoring all Apache Hive entities with column name "password" and "confidential" as table name
atlas.notification.consumer.preprocess.entity.ignore.pattern=.*\\..*\\..*password.*,.*\\..*confidential.* atlas.notification.consumer.preprocess.entity.type.ignore.pattern=hive_column,hive_table_ddl
Ignoring all Apache Hive entities with suffix _tmp in their name
atlas.notification.consumer.preprocess.entity.type.ignore.pattern=hive_.* atlas.notification.consumer.preprocess.entity.ignore.pattern=.*\\..*_tmp.*
Generic ignore patterns on the hook side
Ignore pattern can be configured on the hook side to ignore entities based on
qualifiedName. You can configure the hook side generic ignore patterns Cloudera Manager Advanced Configuration Snippet (Safety Valve) for
atlas-application properties section of each individual hook:
- Go to .
- Search for atlas-application.properties.
- Enter the following configurations combined with your regex expression:
- Configuration for
qualifiedName -
atlas.hook.entity.ignore.pattern
- Configuration for
- Click Save Changes.
- Click to apply your changes.
Ignoring all entities with “test” in their name
atlas.hook.entity.ignore.pattern=.*\\.test.*
