Cloudera Docs

Limitations of Atlas-NiFi integration

Some limitations of this integration.

  • Requires Atlas 0.8-incubating or later. The reporting task requires Atlas REST API version 2, which is introduced in Atlas 0.8-incubating. Older versions of Atlas are not supported.

  • Supports limited datasets and Processors. To report lineage to Atlas, the reporting task should be familiar with what a given processor does with a certain dataset. Later, create an 'Atlas Object Id' for a dataset which uniquely identifies an entity in Atlas. Atlas Object ID has a unique properties map, and mostly 'qualifiedName' is set in the unique properties map to identify an entity. The format of a qualifiedName depends on each dataset. To create this Atlas Object ID, you must implement Processor-specific code that analyzes configured properties.

  • Requires restart of NiFi to update some ReportingTask properties. As the underlying Atlas client library caches configurations when it runs the first time, some properties of this reporting task can not be updated by stopping, configuring, and restarting the reporting task. The NiFi process needs to be restarted in such cases.