Cloudera Docs

Setting schema access strategy in NiFi

Learn how to add Controller Services in NiFi globally or per Process Group and then how to configure them so that they can fetch schemas from Schema Registry.

You can configure Controller Services either globally, before you have created a Process Group, or at any time, on a per Process Group basis.

Adding Controller Services globally

  1. To access the Controller Services configuration dialog for gobal configuration, click the Global Menu at the top right of your canvas, and select Controller Settings.
  2. Click the + icon to display the NiFi Settings dialog.
  3. Use the Filter box to search for the Controller Service you want to add, select that service, and click Add.

Adding Controller Services per Process Group

  1. Click on your Process Group, and then right-click anywhere on your canvas.
  2. Click Configure to display the Process Group Configuration dialog.
  3. Click the Controller Services tab, and then click + to display the Add Controller Service dialog.
  4. Use the Filter box to search for the Controller Service you want to add, select that service, and click Add.

Configuring record reader and writer Controller Services for integration with Schema Registry

  1. From the Process Group Configuration view, click the Edit icon from the right-hand column. The Configure Controller Service dialog appears.
  2. Click the Properties tab.
  3. The Schema Access Strategy specifies how to obtain the schema to be used for interpreting FlowFile data. To ensure integration with Schema Registry, configure Schema Access Strategy with one of the following two values:

    • HWX Schema Reference Attributes

      The NiFi FlowFile is given a set of 3 attributes to describe the schema:

      • schema.identifier

      • schema.version

      • schema.protocol.version

    • HWX Content-Encoded Schema Reference

      Each NiFi FlowFile contains a reference to a schema stored in Schema Registry. The reference is encoded as a single byte indicating the protocol version, 8 bytes indicating the schema identifier and 4 bytes indicating the schema version.

  4. The Schema Write Strategy specifies how the schema for a record should be added to FlowFile data. To ensure integration with Schema Registry, configure Schema Write Strategy with either HWX Schema Reference Attributes or HWX Content-Encoded Schema Reference.