Run Hadoop Smoke Tests
To smoke test your Hadoop upgrade, you can run the following MapReduce job as a regular user.
The job uses MapReduce to write 100MB of data into HDFS with RandomWriter.
hadoop jar /usr/hdp/current/hadoop-mapreduce-client/hadoop-mapreduce-examples.jar randomwriter -Dtest.randomwrite.total_bytes=10000000 test-after-upgrade.
You should see messages similar to:
map 0% reduce 0% …map 100% reduce 100% Job …. completed successfully
MapReduce works successfully. You can now upgrade your other components.
Basic Troubleshooting
To find the number of active nodes and NodeManagers, access the ResourceManager web UI:
http://<resource manager host>:8088/cluster/nodes
The number of active nodes should be equal to the number of nodemanagers.
Accessing error messages:
Access the ApplicationMaster WebUI to view the container logs.
In your console log for the MapReduce job, look for a line in this format:
13/10/02 17:57:21 INFO mapreduce.Job: The url to track the job: http://<resource manager host>:8088/proxy/application_1380673658357_0007/Select the logs link under ApplicationMaster table. It redirects you to the container logs. Error messages display here.