This is the documentation for CDH 5.0.x. Documentation for other versions is available at Cloudera Documentation.

Flume Morphline Solr Sink Configuration Options

You can use the flume.conf configuration file to configure one or more Flume agents, including their sources, sinks, and channels. This is the standard Flume configuration mechanism. For more information about the flume.conf configuration file, see the Flume User Guide.

Flume Morphline Solr Sink provides the following configuration options in the flume.conf file:

Property Name	Default	Description
type		The FQCN of this class: org.apache.flume.sink.solr. morphline.MorphlineSolrSink
batchSize	100	The maximum number of events to take per flume transaction.
batchDurationMillis	1000	The maximum duration per flume transaction (ms). The transaction commits after this duration or when batchSize is exceeded, whichever comes first.
indexerClass	org.apache.flume.sink.solr. morphline.MorphlineSolrIndexer	The FQCN of a class implementing org.apache.flume.sink.solr. morphline.SolrIndexer
morphlineFile	n/a	The location of the morphline configuration file. In a Cloudera Managed Deployment, use: agent.sinks.solrSink. morphlineFile=morphlines.conf In unmanaged deployments, provide the relative or absolute path on the local file system to the morphline configuration file. For example: /etc/flume-ng/conf/ tutorialReadAvroContainer.conf
morphlineId	null	Name used to identify a morphline if there are multiple morphlines in a morphline configuration file.

For example, here is a flume.conf section for a SolrSink for the agent named "agent":

agent.sinks.solrSink.type = org.apache.flume.sink.solr.morphline.MorphlineSolrSink
agent.sinks.solrSink.channel = memoryChannel
agent.sinks.solrSink.batchSize = 100
agent.sinks.solrSink.batchDurationMillis = 1000
agent.sinks.solrSink.morphlineFile = /etc/flume-ng/conf/morphline.conf
agent.sinks.solrSink.morphlineId = morphline1

Note: The examples in this document use a Flume MemoryChannel to easily get started. For production use it is often more appropriate to configure a Flume FileChannel instead, which is a high performance transactional persistent queue.

Page generated September 3, 2015.