8.3. Hadoop Common/HDFS Known Issues

BUG-38088, BUG-36247, BUG-38009: DataNode restart fails during Rolling Upgrade from 2.2.0-2.2.6 to 2.3.x
Problem: DataNode restart fails during Rolling Upgrade from 2.2.0-2.2.6 to 2.3.x. Not all writers have responder running, but sendOOB() tries anyway. This causes NPE and the shutdown thread dies, halting the shutdown after only shutting down DataXceiverServer
Workaround: Skip the failed datanodes, downgrade them and try to upgrade them again.
Workaround: Manually restart the failed datanodes.
BUG-30030: Incomplete installation of hadoop-httpfs.
Problem: hadoop-httpfs RPM scripts are incomplete for HDP 2.2. This affects proxy setup that allows Hue to interact with HA clusters.
Workaround: Have Falcon write to target clusters with wire encryption disabled.
Workaround: The following steps should be executed on the node that will be hosting the HttpFS service. This assumes that HDP 2.2 has already been deployed and configured.
1. Install the hadoop_2_2_0_0_2041-httpfs package:
```
yum install hadoop-httpfs
```
2. Create a symlink in /usr/lib to the location where httpfs is installed:
```
ln -s /usr/hdp/current/hadoop-client/../hadoop-httpfs /usr/lib/hadoop-httpfs
```
3. Create a symlink to libexec:
```
ln -s /usr/hdp/current/hadoop-client/../hadoop/libexec /usr/lib/hadoop-httpfs/libexec
```
4. Create configuration directories and copy the default set of configurations:
```
mkdir -p /etc/hadoop-httpfs/conf
mkdir -p /etc/hadoop-httpfs/tomcat-deployment/conf
cp -r /usr/hdp/2.2.0.0-2041/etc/hadoop-httpfs/conf.empty/* /etc/hadoop-httpfs/conf/
cp -r /usr/hdp/2.2.0.0-2041/etc/hadoop-httpfs/tomcat-deployment.dist/conf/* /etc/hadoop-httpfs/tomcat-deployment/conf
```
5. Enable the init.d script for HttpFS by creating the following symlink:
```
ln -s /usr/hdp/2.2.0.0-2041/etc/rc.d/init.d/hadoop-httpfs /etc/init.d/hadoop-httpfs
```
6. Create the following symlinks to point to the Tomcat libraries and binary scripts that HttpFS will use:
```
ln -s /usr/lib/bigtop-tomcat/lib /etc/hadoop-httpfs/tomcat-deployment/libln -s /usr/lib/bigtop-tomcat/bin /etc/hadoop-httpfs/tomcat-deployment/bin
```
7. Start the HttpFs server:
```
service hadoop-httpfs start
```

Legal notices