Overview
Also available as:
PDF

Use Case 3: Dataflow Management with Schema-Based Routing

Consider a scenario in which you are using Apache NiFi to move different types of syslog events to downstream systems. You have data movement requirements to parse the syslog event to extract the event type and then route the event to a certain downstream system (different Kafka topics, for example), based on the event type.

Without Schema Registry, NiFi uses regular expressions or other utilities to parse the event type value from the payload and store that value in a flowfile attribute. Then NiFi uses routing processors such as RouteOnAttribute to use the parsed value for routing decisions. If the structure of the data changes considerably, this type of extract and routing pattern is brittle and requires frequent changes.

With Schema Registry, NiFi queries the registry for schema and then retrieves the desired valueAs long as compatibility policies are adhered to, NiFi extraction and routing rules do not change. .