Using JMX for accessing HDFS metrics
You can access HDFS metrics over Java Management Extensions (JMX) through either the web interface of an HDFS daemon or by directly accessing the JMX remote agent.
Using the HDFS Daemon Web Interface
You can access JMX metrics through the web interface of an HDFS daemon. This is the recommended method.
For example, use the following command format to access the NameNode JMX:
curl -i http://localhost:50070/jmx
You can use the qry
parameter to fetch only a particular key:
curl -i http://localhost:50070/jmx?qry=Hadoop:service=NameNode,name=NameNodeInfo
Directly Accessing the JMX Remote Agent
This method requires that the JMX remote agent is enabled with a JVM option when starting HDFS services.
For example, the following JVM options in hadoop-env.sh
are used to
enable the JMX remote agent for the NameNode. It listens on port 8004 with SSL disabled.
The user name and password are saved in the mxremote.password
file.
export HADOOP_NAMENODE_OPTS="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.password.file=$HADOOP_CONF_DIR/jmxremote.password -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port=8004 $HADOOP_NAMENODE_OPTS"
See the Oracle Java SE documentation for more information about the related settings.
You can also use the jmxquery tool to retrieve information through JMX.
Hadoop has a built-in JMX query tool, jmxget
. For example:
hdfs jmxget -server localhost -port 8004 -service NameNode
Note | |
---|---|
jmxget requires that authentication be disabled, as it does not
accept a user name and password. |
Using JMX can be challenging for operations personnel who are not familiar with JMX setup, especially JMX with SSL and firewall tunnelling. Therefore, we recommend that you collect JMX information through the web interface of HDFS daemons rather than directly accessing the JMX remote agent.