To smoke test your Hadoop upgrade, you can run the following MapReduce job.
Run this command as regular user. The job uses MapReduce to write 100MB of data
into HDFS with RandomWriter. hadoop jar
/usr/lib/hadoop-mapreduce/*examples*.jar randomwriter
-Dtest.randomwrite.total_bytes=100000000 test-after-upgrade You should
see messages similar to:
map 0% reduce 0% … map 100% reduce 100% Job …. completed successfully
You just submitted your first MapReduce job in HDP 2.x. Good job! The next steps are to upgrade your other components.
Basic troubleshooting:
To find the number of active nodes and NodeManagers, access the ResourceManager web UI:
http://<resource manager host>:8088/cluster/nodes
The number of active nodes should be equal to the number of nodemanagers.
Error messages. Access the ApplicationMaster WebUI to view the container logs.
Looking at your console logs for MapReduce job, there is a line in this format:
13/10/02 17:57:21 INFO mapreduce.Job: The url to track the job: http://<resource manager host>:8088/proxy/application_1380673658357_0007/
Go to the URL and select the job link.
Select the logs link under ApplicationMaster table. It will redirect you to the container logs. Error messages display here.
