Cloudera Docs

Run the Batch Profiler

CCP provides a script called start_batch_profiler.sh to simplify running the batch profiler.

The start_batch_profiler.sh script assumes the following:
  • The script does one of the following:
    • Builds the profiles defined in $METRON_HOME/config/zookeeper/profiler.json
    • Utilizes the profiles already loaded into ZooKeeper quorum at $ZOOKEEPER if the environment variable $SPARK_PROFILER_USE_ZOOKEEPER is set.
  • The properties defined in $METRON_HOME/config/batch-profiler.properties are passed to both the profiler and Spark. You can define both Spark and profiler properties in this same file.
  • The script will also configure the event time field to use if the field value is stored in the ${SPARK_PROFILER_EVENT_TIMESTAMP_FIELD} environment variable.
  • Spark is installed at /usr/hdp/current/spark-client. This can be overridden if you define an environment variable called SPARK_HOME prior to executing the script.
Start the batch profile by entering the following: 
source /etc/default/metron
cd $METRON_HOME
bin/start_batch_profiler.sh