6.1. Known Issues for Hadoop

  • JobTracker UI, JMX queries, and RPC calls somtimes hang in HA mode.

    Problem: JobTracker becomes slow and non-responsive in HA mode because dfs.client.retry.policy.enabled is not set to 'final' and 'false' for the JobTracker.

    Workaround:Set dfs.client.retry.policy.enabled to 'final' and 'false' only for the JobTracker. Clients (such as MapReduce, Pig, Hive, Oozie) should still be set to 'true' in HA mode.

    Problem: While uploading files to NFS-MountDir, the following error is reported in the DataNode log file:

    INFO org.apache.hadoop.hdfs.nfs.nfs3.OpenFileCtx: requesed offset=4980736 and current filesize=0

    Workaround: On some environments, especially for virtualized environments, copying large files of size close to 1GB fails intermittently. This issue is expected to be addressed in the upcoming release.

  • Use of init.d scripts for starting or stopping Hadoop services, is not recommended.


loading table of contents...