Manually integrating with Atlas when Auto-TLS is not enabled

If CFM or the CDP Private Cloud Base cluster does not have Auto-TLS enabled and you want to Atlas, then you must manually integrate with Atlas by creating the ReportLineageToAtlas reporting task.

Perform this task if:

  • CFM does not have TLS enabled; AND
  • The CDP Private Cloud Base cluster does not have auto-TLS enabled; AND
  • You do not want to enable auto-TLS; AND
  • You want Atlas as part of CFM on your CDP Private Cloud Base deployment.
  1. From the Global Menu located in NiFi’s upper right corner, select Controller Services and click the Reporting Tasks tab.
  2. Click the Add (+) icon to launch the Add Reporting Task dialog.
  3. Select ReportLineageToAtlas and click Add.
  4. Click the Edit icon to launch the Configure Reporting Task dialog. The following properties are required:
    • Atlas URLs – a comma-separated list of Atlas Server URLs. Once you have started reporting, you cannot modify an existing Reporting Task to add a new Atlas Server. When you need to add a new Atlas Server, you must create a new reporting task.

    • Atlas Configuration Directory - This specifies where the is created.

      The directory must:
      • Be located and accessible/writable by the user running the NiFi process.
      • Be available on each NiFi node.
      • Pre-exist. It will not be created by the reporting task.
      • Not be in the /tmp directory.
    • Create Atlas Configuration File – Set to True. When set to True, the atlas-application-properties file and the Atlas Configuration Directory are automatically created when the Reporting Task starts.

    • Lineage Strategy – Specifies the level of granularity for your NiFi dataflow reporting to Atlas. Once you have started reporting, you should not switch between simple and complete lineage reporting strategies.

    • Provenance Record Start Position – Specifies where in the Provenance Events stream the Reporting Task should start.

    • Provenance Record Batch Size – Specifies how many records you want to send in a single batch

    • NiFi URL for Atlas – Specifies the NiFi cluster URL.

    • Atlas Authentication Method – Specifies how to authenticate the Reporting Task to the Atlas Server. Basic authentication is the default.

    • Kafka Security Protocol – Specifies the protocol used to communicate with Kafka brokers to send Atlas hook notification messages. This value should match Kafka's security.protocol property value.