Run Hadoop Smoke Tests
To smoke test your Hadoop upgrade, you can run the following MapReduce job as a regular user. The job uses MapReduce to write 100MB of data into HDFS with RandomWriter.
hadoop jar /usr/hdp/current/hadoop-mapreduce-client/hadoop-mapreduce-examples.jar
randomwriter - Dtest.randomwrite.total_bytes=100000000 test-after-upgrade
You should see messages similar to:
map 0% reduce 0% …map 100% reduce 100% Job …. completed successfully
You have successfully submitted your first MapReduce job in HDP 2.x. The next steps upgrade your other components to 2.x.
Basic troubleshooting:
To find the number of active nodes and NodeManagers, access the ResourceManager web UI:
http://<resource manager host>:8088/cluster/nodesThe number of active nodes should be equal to the number of nodemanagers.
Error messages. Access the ApplicationMaster WebUI to view the container logs.
Looking at your console logs for MapReduce job, there is a line in this format:
13/10/02 17:57:21 INFO mapreduce.Job: The url to track the job: http://<resource manager host>:8088/proxy/application_1380673658357_0007/Go to the URL and select the job link.
Select the logs link under ApplicationMaster table. It redirects you to the container logs. Error messages display here.