12. Run Hadoop Smoke Tests

To smoke test your Hadoop upgrade, you can run the following MapReduce job as a regular user. The job uses MapReduce to write 100MB of data into HDFS with RandomWriter.

hadoop jar /usr/hdp/current/hadoop-mapreduce-client/hadoop-mapreduce-examples.jar randomwriter - Dtest.randomwrite.total_bytes=100000000 test-after-upgrade

You should see messages similar to:

map 0% reduce 0%
…map 100% reduce 100%
Job …. completed successfully

You have successfully submitted your first MapReduce job in HDP 2.x. The next steps will upgrade your other components to 2.x.

Basic troubleshooting:

To find the number of active nodes and NodeManagers, access the ResourceManager web UI:
http://<resource manager host>:8088/cluster/nodes
The number of active nodes should be equal to the number of nodemanagers.
Error messages. Access the ApplicationMaster WebUI to view the container logs.
1. Looking at your console logs for MapReduce job, there is a line in this format:
  13/10/02 17:57:21 INFO mapreduce.Job: The url to track the job: http://<resource manager host>:8088/proxy/application_1380673658357_0007/
2. Go to the URL and select the job link.
3. Select the logs link under ApplicationMaster table. It will redirect you to the container logs. Error messages display here.

Legal notices