DocsHortonworks Data Platform

 

 Chapter 23. Installing Ganglia (Deprecated)

This section describes installing and testing Ganglia, a system for monitoring and capturing metrics from services and components of the Hadoop cluster.

 1. Install the Ganglia RPMs

On the host that you have chosen for the Ganglia server, install the server RPMs.

  • For RHEL/CentOS/Oracle Linux:

    yum install ganglia-gmond-3.5.0-99 ganglia-gmetad-3.5.0-99 ganglia-web-3.5.7-99

  • For SLES:

    zypper install ganglia-gmond-3.5.0-99 ganglia-gmetad-3.5.0-99 ganglia-web-3.5.7-99

On each host in the cluster, install the client RPMs:

  • For RHEL/CentOS/Oracle Linux:

    yum install ganglia-gmond-3.5.0-99

  • For SLES:

    zypper install ganglia-gmond-3.5.0-99

 2. Install the Configuration Files

There are several configuration files that need to be set up for Ganglia.

 3. Extract the Ganglia Configuration Files

From the HDP companion files, open the configuration_files folder and copy the files in the ganglia folder to a temporary directory. The ganglia folder contains two sub-folders, objects and scripts.

 4. Copy the Configuration Files

On each host in the cluster:

  1. Grant execute permissions on the following scripts:

    • /usr/libexec/hdp/ganglia/setupGanglia.sh

    • /usr/libexec/hdp/ganglia/startRrdcached.sh

  2. Change permissions on the RRD base directory to grant access to nobody:

    chown -R nobody:nobody $RRDCACHED_BASE_DIR chmod -R 755 $RRDCACHED_BASE_DIR

  3. Create the directory for the objects folder:

    mkdir -p /usr/libexec/hdp/ganglia

  4. Copy the object files:

    cp <tmp-directory>/ganglia/objects/*.* /usr/libexec/hdp/ganglia/

  5. Copy the Ganglia monitoring init script to init.d:

    cp <tmp-directory>/ganglia/scripts/hdp-gmond /etc/init.d

  6. On the Ganglia Server Host, copy the entire contents of the scripts folder to init.d:

    cp -R <tmp-directory>/ganglia/scripts/* /etc/init.d/

 5. Set Up Ganglia Hosts

  1. On the Ganglia server, to configure the gmond collector:

    /usr/libexec/hdp/ganglia/setupGanglia.sh -c HDPHistoryServer -m

    /usr/libexec/hdp/ganglia/setupGanglia.sh -c HDPNameNode -m

    /usr/libexec/hdp/ganglia/setupGanglia.sh -c HDPSlaves -m

    /usr/libexec/hdp/ganglia/setupGanglia.sh -t

  2. If HBase is installed, on the HBase Master:

    /usr/libexec/hdp/ganglia/setupGanglia.sh -c HDPHBaseMaster -m

  3. On the NameNode and SecondaryNameNode servers, to configure the gmond emitters:

    /usr/libexec/hdp/ganglia/setupGanglia.sh -c HDPNameNode

  4. On the ResourceManager server, to configure the gmond emitters:

    /usr/libexec/hdp/ganglia/setupGanglia.sh -c HDPResourceManager

  5. On all hosts, to configure the gmond emitters:

    /usr/libexec/hdp/ganglia/setupGanglia.sh -c HDPSlaves

  6. If HBase is installed, on the HBase Master, to configure the gmond emitter:

    /usr/libexec/hdp/ganglia/setupGanglia.sh -c HDPHBaseMaster

 6. Set Up Configurations

  1. On the Ganglia server, use a text editor to open the following master configuration files: 

    /etc/ganglia/hdp/HDPNameNode/conf.d/gmond.master.conf
/etc/ganglia/hdp/HDPHistoryServer/conf.d/gmond.master.conf
/etc/ganglia/hdp/HDPResourceManager/conf.d/gmond.slave.conf
/etc/ganglia/hdp/HDPSlaves/conf.d/gmond.master.conf

    And if HBase is installed:

    /etc/ganglia/hdp/HDPHBaseMaster/conf.d/gmond.master.conf

  2. Confirm that the “bind” property in each of these files is set to the Ganglia server hostname.

  3. On the Ganglia server, use a text editor to open the gmetad configuration file:

    /etc/ganglia/hdp/gmetad.conf

  4. Confirm that the "data_source" properties are set to the Ganglia server hostname. For example:

    data_source "HDPSlaves" my.ganglia.server.hostname:8660 
data_source "HDPNameNode" my.ganglia.server.hostname:8661 
data_source "HDPResourceManager" my.ganglia.server.hostname:8664 
data_source "HDPHistoryServer" my.ganglia.server.hostname:8666

    And if HBase is installed:

    data_source "HDPHBaseMaster" my.ganglia.server.hostname:8663

  5. On all hosts except the Ganglia server, use a text editor to open the slave configuration files: 

    /etc/ganglia/hdp/HDPNameNode/conf.d/gmond.slave.conf/
etc/ganglia/hdp/HDPHistoryServer/conf.d/gmond.slave.conf/
etc/ganglia/hdp/HDPResourceManager/conf.d/gmond.slave.conf/
etc/ganglia/hdp/HDPSlaves/conf.d/gmond.slave.conf

    And if HBase is installed:

    /etc/ganglia/hdp/HDPHBaseMaster/conf.d/gmond.slave.conf

  6. Confirm that the host property is set to the Ganglia Server hostname.

 7. Set Up Hadoop Metrics

On each host in the cluster:

  1. Stop the Hadoop services.

  2. Change to the Hadoop configuration directory.

    cd $HADOOP_CONF_DIR

  3. Copy the Ganglia metrics properties file into place.

    mv hadoop-metrics2.properties-GANGLIA hadoop-metrics2.properties

  4. Edit the metrics properties file and set the Ganglia server hostname.

    namenode.sink.ganglia.servers=my.ganglia.server.hostname:8661 datanode.sink.ganglia.servers=my.ganglia.server.hostname:8660
resourcemanager.sink.ganglia.servers=my.ganglia.server.hostname:8664 
nodemanager.sink.ganglia.servers=my.ganglia.server.hostname:8660
historyserver.sink.ganglia.servers=my.ganglia.server.hostname:8666 
maptask.sink.ganglia.servers=my.ganglia.server.hostname:8660
reducetask.sink.ganglia.servers=my.ganglia.server.hostname:8660

  5. Restart the Hadoop services.

 8. Validate the Installation

Use these steps to validate your installation.

  1. Start the Ganglia server.

  2. On the Ganglia server:

    service httpd restart

    /etc/init.d/hdp-gmetad start

  3. Start Ganglia Monitoring on all hosts.

    On all hosts:

    /etc/init.d/hdp-gmond start

  4. Confirm that Ganglia is running.

    Browse to the Ganglia server:

    http://{ganglia.server}/ganglia

Legal notices
loading table of contents...