Flume Solr UUIDInterceptor Configuration Options
Flume can modify or drop events in-flight with the help of Interceptors, which can be attached to any Flume Source. Flume Solr UUIDInterceptor sets a universally unique identifier on all intercepted events. For example, UUID b5755073-77a9-43c1-8fad-b7a586fc1b97 represents a 128-bit value.
You can use UUIDInterceptor to automatically assign a UUID to a document event if no application-level unique key for the event is available. Assign UUIDs to events as soon as they enter the Flume network—that is, in the first Flume source of the flow. This enables deduplicating documents that may be accidentally duplicated as a result of replication and redelivery in a Flume network that is designed for high availability and high performance. If available, an application-level key is preferable to an auto-generated UUID because it enables subsequent updates and deletion of the document in Solr using that key.
Flume Solr UUIDInterceptor provides the following configuration options in the flume.conf file:
|
Property Name |
Default |
Description |
|---|---|---|
|
type |
|
The FQCN of this class:
org.apache.flume.sink.solr. morphline.UUIDInterceptor$Builder |
|
headerName |
id |
The name of the Flume header to modify. |
|
preserveExisting |
true |
If the UUID header already exists, determine whether it is preserved. |
|
prefix |
"" |
The prefix string constant to prepend to each generated UUID. |
For examples, see the BlobHandler and BlobDeserializer.