2. Install the Configuration Files

There are several configuration files that need to be set up for Ganglia. Use the following instructions ot install the configuration files for Ganglia:

  1. Extract the Ganglia configuration files.

    From the downloaded scripts.zip file (downloaded in Download Companion Files), copy the files in the configuration_files/ganglia  directory to a temporary directory.

    The ganglia directory contains two sub-directories, objects and scripts.

  2. Copy the configuration files.

    On the Ganglia server host, complete the following instructions:

    1. Create a directory for the objects directory and copy the objects files:

      mkdir -p /usr/libexec/hdp/ganglia
      cp $tmp-directory/ganglia/objects/* /usr/libexec/hdp/ganglia
    2. Copy the contents of the scripts directory to init.d directory.

      cp $tmp-directory/ganglia/scripts/* /etc/init.d
  3. On each host in the cluster, copy the Ganglia monitoring init script to init.d directory:

    cp $tmp-directory/ganglia/scripts/hdp-gmond /etc/init.d
  4. Set up Ganglia hosts.

    1. On the Ganglia server, execute the following commands to configure the gmond collector:

      /usr/libexec/hdp/ganglia/setupGanglia.sh -c HDPJobTracker -m   
      /usr/libexec/hdp/ganglia/setupGanglia.sh -c HDPNameNode -m
      /usr/libexec/hdp/ganglia/setupGanglia.sh -c HDPSlaves -m
      /usr/libexec/hdp/ganglia/setupGanglia.sh -t
    2. If HBase is installed, execute the following command on the HBase Master host machine:

      /usr/libexec/hdp/ganglia/setupGanglia.sh -c HDPHBaseMaster -m
    3. On the NameNode and SecondaryNameNode servers, execute the following command to configure the gmond emitters:

      /usr/libexec/hdp/ganglia/setupGanglia.sh -c HDPNameNode
    4. On the JobTracker server, execute the following command to configure the gmond emitters:

      /usr/libexec/hdp/ganglia/setupGanglia.sh -c HDPJobTracker
    5. On all hosts, execute the following command to configure the gmond emitters:

      /usr/libexec/hdp/ganglia/setupGanglia.sh -c HDPSlaves
    6. If HBase is installed, execute the following command on the HBase Master host machine to configure the gmond emitter:

      /usr/libexec/hdp/ganglia/setupGanglia.sh -c HDPHBaseMaster

  5. Set up configurations.

    1. On the Ganglia server, confirm that the bind property in each of the following files is set to the Ganglia server hostname:

      /etc/ganglia/hdp/HDPNameNode/conf.d/gmond.master.conf
      /etc/ganglia/hdp/HDPJobTracker/conf.d/gmond.master.conf
      /etc/ganglia/hdp/HDPSlaves/conf.d/gmond.master.conf

      And if HBase is installed:

      /etc/ganglia/hdp/HDPHBaseMaster/conf.d/gmond.master.conf
    2. On the Ganglia server, open the /etc/ganglia/hdp/gmetad.conf file and confirm that the data_source properties are set to the Ganglia server hostname.

      For example:

      data_source "HDPSlaves" $my.ganglia.server.hostname:8660
      data_source "HDPNameNode" $my.ganglia.server.hostname:8661
      data_source "HDPJobTracker" $my.ganglia.server.hostname:8662

      And if HBase is installed:

      data_source "HDPHBaseMaster" $my.ganglia.server.hostname:8663
    3. On all hosts except the Ganglia server, open the slave configuration files and confirm that the host property is set to the Ganglia Server hostname:

      /etc/ganglia/hdp/HDPNameNode/conf.d/gmond.slave.conf
      /etc/ganglia/hdp/HDPJobTracker/conf.d/gmond.slave.conf
      /etc/ganglia/hdp/HDPSlaves/conf.d/gmond.slave.conf 

      And if HBase is installed:

      /etc/ganglia/hdp/HDPHBaseMaster/conf.d/gmond.slave.conf

  6. Set up Hadoop metrics. On each host in the cluster, complete the following instructions:

    1. Stop the Hadoop services using the instructions provided here.

    2. Change to the Hadoop configuration directory.

      cd $HADOOP_CONF_DIR

      where $HADOOP_CONF_DIR is the is the directory for storing the Hadoop configuration files. For example, /etc/hadoop/conf.

    3. Copy the Ganglia metrics properties file into place.

      mv hadoop-metrics2.properties-GANGLIA  hadoop-metrics2.properties
    4. Edit the metrics properties file and set the Ganglia server hostname.

      namenode.sink.ganglia.servers=$my.ganglia.server.hostname:8661
      datanode.sink.ganglia.servers=$my.ganglia.server.hostname:8660
      jobtracker.sink.ganglia.servers=$my.ganglia.server.hostname:8662
      tasktracker.sink.ganglia.servers=$my.ganglia.server.hostname:8660
      maptask.sink.ganglia.servers=$my.ganglia.server.hostname:8660
      reducetask.sink.ganglia.servers=$my.ganglia.server.hostname:8660
    5. Restart the Hadoop services using the instructions provided here.


loading table of contents...