Streaming Profiler Properties
Use the profiler properties to configure the streaming profiler.
Ambari Configs Field. | Settings | Description |
Profiler Setup | ||
Period Duration | profiler.period.duration |
The duration of each profile period. This value should be defined along
with profiler.period.duration.units . |
Period Units | profiler.period.units |
The units is used to specify the
profiler.period.duration . This value should be define along with
profiler.period.duration . |
Window Duration | profiler.window.duration |
The duration of each profile window. |
Window Units | profiler.window.duration.units |
The units used to specify the
profiler.window.duration . |
Time to Live | profiler.ttl |
If a message has not been applied to a Profile in this period of time,
the Profile will be terminated and its resources will be cleaned up. This
value should be defined along with profiler.ttl.units. This time-to-live
does not affect the persisted Profile data in HBase. It only affects the
state stored in memory during the execution of the latest profile period.
This state will be deleted if the time.to.live is
exceeded. |
Time to Live Units | profiler.ttl.units |
The units used to specify the profiler.ttl . |
Window Time Lag | profiler.window.log |
The maximum time lag for timestamps. |
Window Lag Units | profiler.window.log.units |
The units used to specify the
profiler.window.lag . |
Max Routes Per Bolt | profiler.max.routes.per.bolt |
The maximum number of routes that will be maintained by the bolt. After this value is exceeded, lesser used routes will be evicted from the internal cache. |
Kafka | ||
Input Topic Start | profiler.kafka.start |
One of EARLIEST, LATEST, UNCOMMITTED_EARLIEST, UNCOMMITTED_LATEST |
Kafka Writer Batch Size | profiler.writer.batchSize |
The number of records to batch when writing to Kakfa. |
Kafka Writer Batch Timeout | profiler.writer.batchTimeout |
The timeout in ms for batching when writing to Kakfa. |
Storm | ||
topology.worker.childopts | profiler.topology.worker.childopts |
Extra topology child opts for the storm opts. |
Number of Workers | profiler.topology.workers |
The profiler storm topology storm workers |
Number of Acker Executors | profiler.acker.executors |
The profiler storm topology acker executors |
Profiler Topology Message Timeout | topology.message.timeout.secs |
Maximum amount of time given to the topology to fully process a tuple tree from the core-storm API, or a batch from the Trident API, emitted by a spout. If the message is not acked within this time frame, Storm fails the operation on the spout. The default is 30 seconds. |
Spout Max Pending Tuples | topology.max.spout.pending |
Maximum number of messages that can be pending in a spout at any time. The default is null (no limit). |
HBase | ||
HBase Table | profiler.hbase.table |
The name of the HBase table the profiler is written to. The profiler expects that the table exists and is writable. |
HBase Table Column Family | profiler.hbase.cf |
The column family used to store profile data in HBase. |
HBase Batch Size | profiler.hbase.batch |
The number of puts that are written to HBase in a single batch. |
HBase Flush Interval | profiler.hbase.flush.interval |
The maximum number of seconds between batch writes to HBase. |
N/A | profiler.writer.batchSize |
The size of the batch that is written to Kafka at once. Defaults to 15 (size of 1 disables batching). |
N/A | profiler.writer.batchTimeout |
The timeout after which a batch will be flushed even if batchSize has
not been met. Optional. If unspecified or set to 0, it defaults to a
system-determined duration which is a fraction of the Storm parameter
topology.message.timeout.secs . Ignored if batchsize is 1
because this disables batching. |