17. Setting up HDP

Open a command prompt using the hadoop user in the Hadoop home directory:

runas /user:hadoop "cmd /K cd %HDFS_HOME%\bin"

Make, and then set, ownership and permissions for the following directories:

hdfs dfs -mkdir -p /mapred/history/done /mapred/history/
done_intermediate
hdfs dfs -chmod -R 1777 /mapred/history/done_intermediate 
hdfs dfs -chmod 770 /mapred/history/done 
hdfs dfs -chown -R hadoop:hadoopUsers /mapred 
hdfs dfs -chmod 755 /mapred/mapred/history 
hdfs dfs -mkdir /tmp 
hdfs dfs -chmod 777 /tmp 
hdfs dfs -mkdir /app-logs 
hdfs dfs -chown hadoop:hadoopUsers /app-logs 
hdfs dfs -chmod 1777 /app-logs hdfs dfs -mkdir -p /yarn /yarn/generic-history/ 
hdfs dfs -chmod -R 700 /yarn
hdfs dfs -chown -R hadoop:hadoop /yarn 
hdfs dfs -mkdir -p /apps/hbase/data 
hdfs dfs -chown hadoop:hadoop /apps/hbase/data 
hdfs dfs -chown hadoop:hadoop /apps/hbase 
hdfs dfs -mkdir -p /user/hbase
hdfs dfs -chown hadoop:hadoop /user/hbase 
hdfs dfs -mkdir -p /user/hive /hive/warehouse 
hdfs dfs -chown hadoop:hadoop /user/hive 
hdfs dfs -chmod -R 755 /user/hive 
hdfs dfs -chown -R hadoop:users /hive/warehouse 
hdfs dfs -chmod -R 777 /hive/warehouse 
hdfs dfs -mkdir -p /user/hcat 
hdfs dfs -chown -R hadoop:hadoop /user/hcat 
hdfs dfs -mkdir -p /apps/webhcat 
hdfs dfs -chown -R hadoop:users/apps/webhcat 
hdfs dfs -chmod -R 755 /apps/webhcat

When using Tez for Hive, you must also Upgrade Tez.