Using JMX for accessing HDFS metrics

Using the HDFS daemon web interface

You can access JMX metrics through the web interface of an HDFS daemon. This is the recommended method.

For example, use the following command format to access the NameNode JMX:

curl -i http://localhost:50070/jmx

You can use the qry parameter to fetch only a particular key:

curl -i http://localhost:50070/jmx?qry=Hadoop:service=NameNode,name=NameNodeInfo

Directly accessing the JMX remote agent

This method requires that the JMX remote agent is enabled with a JVM option when starting HDFS services.

For example, the following JVM options in hadoop-env.sh are used to enable the JMX remote agent for the NameNode. It listens on port 8004 with SSL disabled. The user name and password are saved in the mxremote.password file.

export HADOOP_NAMENODE_OPTS="-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.password.file=$HADOOP_CONF_DIR/jmxremote.password
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.port=8004 $HADOOP_NAMENODE_OPTS"

See the Oracle Java SE documentation for more information about the related settings.

You can also use the jmxquery tool to retrieve information through JMX.

Hadoop has a built-in JMX query tool, jmxget. For example:

hdfs jmxget -server localhost -port 8004 -service NameNode

Using JMX can be challenging for operations personnel who are not familiar with JMX setup, especially JMX with SSL and firewall tunnelling. Therefore, we recommend that you collect JMX information through the web interface of HDFS daemons rather than directly accessing the JMX remote agent.