Differential entity audits
Apache Atlas can store differential information in entity audit events to optimize storage space when entities are updated.
By default, when an entity core attribute, relationship attribute, custom attribute, or associated classifications is updated for an entity, the Atlas captures the changed entity either as a full entity object or partial object (with only updated attributes or relations) and stores it in an HBase table.
While processing update requests, Atlas generates entity audit events in JSON format and stores them in the HBase table. Repeated updates to a single entity duplicate full instances in storage, consuming significant space over time.
For example, if entity A1 is updated or modified five times, the entire
entity is stored five times in the HBase tables along with the minimal changes
To optimize storage space, you can configure Atlas to store only differential information in audit events.
You must enable the application flag by adding the
atlas.entity.audit.differential=true parameter to for conf/atlas-application.properties.
To reduce the footprint of stored audit payloads, only the difference between the original and updated entity state is captured in Atlas. In default deployments that footprint is primarily in the HBase tables that hold entity audits.
When differential auditing is enabled, the Atlas UI displays only the modified attributes and values for user-defined properties and system attributes rather than the full entity state.
When you add data under user-defined-properties, for example, entity_type and Aircraft, Atlas displays only the changed attributes and values instead of all attributes.
