Use Case 3: Dataflow Management with Schema-Based Routing
Consider a scenario in which you are using Apache NiFi to move different types of syslog events to downstream systems. You have data movement requirements to parse the syslog event to extract the event type and then route the event to a certain downstream system (different Kafka topics, for example), based on the event type.
Without Schema Registry, NiFi uses regular expressions or other utilities to
parse the event type value from the
payload
and store that
value
in
a
flowfile
attribute. Then NiFi uses routing processors
such as
RouteOnAttribute
to use the parsed value for routing decisions.
If
the structure of the data changes considerably, this type of extract and routing pattern is
brittle and requires frequent changes.
With Schema Registry, NiFi queries the registry for schema and then retrieves the desired valueAs long as compatibility policies are adhered to, NiFi extraction and routing rules do not change. .