Configuring the Profiler
The configuration for the Profiler topology is stored in ZooKeeper at
/metron/topology/profiler
. These properties also exist in the default
installation of HCP at $METRON_HOME/config/zookeeper/profiler.json
. You can
change these values two ways: with Ambari or on disk and then uploaded to ZooKeeper using
$METRON_HOME/bin/zk_load_configs.sh
. The following task uses Ambari to
configure the Profiler.
You might need to work with your Platform Engineer to modify Profiler values.
Note | |
---|---|
The Profiler can persist any serializable object, not just numeric values. |
To configure the Profiler, complete the following steps:
Display the Ambari user interface and click the Services tab.
Click Metron in the list of services, then click the Configs tab.
Click the Profiler tab.
Ambari displays a list of Profiler properties that you can use to configure Profiler.
Use these properties to configure the Profiler, then click the Save button near the top of the window.
The following table provides a list of the configurable properties and descriptions.
Table 2.1. Profiler Properties
Ambari Configs Field. | Settings. | Description |
Kafka Input Topic Start | profiler_kafka_start | One of EARLIEST, LATEST, UNCOMMITTED_EARLIEST, UNCOMMITTED_LATEST |
Profiler Setup Period Duration | profiler_period_duration | The duration of each profile period. This value should be defined along with
profiler_period_duration_units . |
Period Units | profiler_period_units | The units is used to specify the profiler_period_duration . This
value should be define along with profiler_period_duration . |
Time to Live | profiler_ttl | If a message has not been applied to a Profile in this period of time, the
Profile will be terminated and its resources will be cleaned up. This value should be
defined along with profiler_ttl_units. This time-to-live does not affect the persisted
Profile data in HBase. It only affects the state stored in memory during the execution
of the latest profile period. This state will be deleted if the
time_to_live is exceeded. |
Time to Live Units | profiler_ttl_units | The units used to specify the profiler_ttl . |
HBase Table | profiler_hbase_table | The name of the HBase table the profiler is written to. The profiler expects that the table exists and is writable. |
HBase Table Column Family | profiler_hbase_cf | The column family used to store profile data in HBase. |
HBase Batch Size | profiler_hbase_batch | The number of puts that are written to HBase in a single batch. |
HBase Flush Interval | profiler_hbase_flush_interval | The maximum number of seconds between batch writes to HBase. |
Storm topology.worker.childopts | profiler_topology_worker_childopts | Extra topology child opts for the storm opts. |
Number of Workers | profiler_topology_workers | The profiler storm topology storm workers |
Number of Acker Executors | profiler_acker_executors | The profiler storm topology acker executors |
Note | |
---|---|
The profiler input topic is bound to the enrichment output topic. If that enrichment output topic is changed, then the profiler will restart as well as the enrichment topology. |