This is the documentation for CDH 5.0.x. Documentation for other versions is available at Cloudera Documentation.

Flume Solr UUIDInterceptor Configuration Options

Flume can modify or drop events in-flight. This is done with the help of Interceptors, which can be attached to any Flume Source. Flume Solr UUIDInterceptor is an Interceptor that sets a universally unique identifier on all events that are intercepted. An example UUID is b5755073-77a9-43c1-8fad-b7a586fc1b97, which represents a 128-bit value.

Consider using UUIDInterceptor to automatically assign a UUID to a (document) event if no application level unique key for the event is available. It is important to assign UUIDs to events as soon as they enter the Flume network; that is, in the first Flume Source of the flow. This enables subsequent deduplication of documents in the face of replication and redelivery in a Flume network that is designed for high availability and high performance. If an application level key is available, this is preferable to an auto-generated UUID because it enables subsequent updates and deletes of the document in Solr using that well-known application-level key.

Flume Solr UUIDInterceptor provides the following configuration options in the flume.conf file:

Property Name

Default

Description

type

 

The FQCN of this class:
org.apache.flume.sink.solr.
morphline.UUIDInterceptor$Builder

headerName

id

The name of the Flume header to modify.

preserveExisting

true

If the UUID header already exists, determine whether it is preserved.

prefix

""

The prefix string constant to prepend to each generated UUID.

For examples, see the BlobHandler and BlobDeserializer.

Page generated September 3, 2015.