Cloudera Docs

Manually integrate with Atlas

Provides steps to manually integrate with Atlas by creating the ReportLineageToAtlas reporting task.

If you are upgrading from a CFM 1.1.0 installation where:

  • CFM does not have TLS enabled; AND
  • The CDP Private Cloud Base cluster does not have Auto-TLS enabled; AND
  • You do not want to enable Auto-TLS; AND
  • You want Atlas as part of CFM 2.0.1 on your CDP Private Cloud Base 7.1.1 deployment.
  1. Start NiFi.
    1. From Cloudera Manager, click the Clusters tab in the left-hand navigation.
    2. Click NiFi in the list of services to display the NiFi service page.
    3. Click the Actions drop-down, and then click Start.
  2. From the Global Menu located in NiFi’s upper right corner, select Controller Services and click the Reporting Tasks tab.
  3. Click the Add (+) icon to launch the Add Reporting Task dialog.
  4. Select ReportLineageToAtlas and click Add.
  5. Click the Edit icon to launch the Configure Reporting Task dialog. The following properties are required:

    • Atlas URLs – a comma-separated list of Atlas Server URLs. Once you have started reporting, you cannot modify an existing Reporting Task to add a new Atlas Server. When you need to add a new Atlas Server, you must create a new reporting task.

    • Atlas Authentication Method – Specifies how to authenticate the Reporting Task to the Atlas Server. Basic authentication is the default.

    • NiFi URL for Atlas – Specifies the NiFi cluster URL

    • Lineage Strategy – Specifies the level of granularity for your NiFi dataflow reporting to Atlas. Once you have started reporting, you should not switch between simple and complete lineage reporting strategies.

    • Provenance Record Start Position – Specifies where in the Provenance Events stream the Reporting Task should start.

    • Provenance Record Batch Size – Specifies how many records you want to send in a single batch

    • Create Atlas Configuration File – If enabled, the atlas-application-properties file and the Atlas Configuration Directory are automatically created when the Reporting Task starts.

    • Kafka Security Protocol – Specifies the protocol used to communicate with Kafka brokers to send Atlas hook notification messages. This value should match Kafka's security.protocol property value.