Flume can modify or drop events in-flight. This is done with the help of Interceptors, which can be attached to any Flume Source. Flume Solr UUIDInterceptor is an Interceptor that sets a universally unique identifier on all events that are intercepted. An example UUID is b5755073-77a9-43c1-8fad-b7a586fc1b97, which represents a 128-bit value.

Consider using UUIDInterceptor to automatically assign a UUID to a (document) event if no application level unique key for the event is available. It is important to assign UUIDs to events as soon as they enter the Flume network; that is, in the first Flume Source of the flow. This enables subsequent deduplication of documents in the face of replication and redelivery in a Flume network that is designed for high availability and high performance. If an application level key is available, this is preferable to an auto-generated UUID because it enables subsequent updates and deletes of the document in Solr using that well-known application-level key.

Flume Solr UUIDInterceptor provides the following configuration options in the flume.conf file:

org.apache.flume.sink.solr.
morphline.UUIDInterceptor$Builder

For examples, see the BlobHandler and BlobDeserializer.