Use the following instructions to install a single Hadoop cluster node from the command line using a cluster properties file:
On the host nodes, complete all the prerequisites described in Before You Begin. Be especially careful to identify:
Supported operating system
Dependent software and environment variable settings
Enable PowerShell remote scripting and set cluster nodes as trusted hosts
Resolvable host names and static IPv4 addresses
Open ports required for HDP operation
Download the HDP Windows Installation package. This package includes a sample cluster properties file, called
clusterproperties.txt
.Create a cluster properties file using your host information; see Defining Cluster Properties.
Note Nodes in the cluster communicate with each other using the host name or IP address defined in the cluster properties file. For multi-homed systems (systems that can be accessed internally and externally) and systems with more than one NIC, ensure that the preferred name or IP address is specified in the Cluster Properties file.
Place the MSI and custom cluster properties file in a local subdirectory on the host. Only the Hadoop Services that match the system's host name in the cluster properties file will be installed.
Use the same cluster properties file on every node in the cluster.
(Optional) When installing HDP with HDFS compression enabled, put the following three files (from the HDP for Windows Installation zip) into the directory that contains the HDP for Windows Installer and the cluster properties file:
hadoop-lzo-0.4.19.2.2.0.0-2060
gplcompression.dll
lzo2.dll
The following two examples show
msiexec
commands with HDP parameters.Warning These examples assume that you would like to destroy any existing HDP data. If you have existing data that you wish to keep, set
DESTROY_DATA
tono
.Open a command prompt with the
run as Administrator
option, and enter the following:msiexec /qn /i c:\MSI_Download\hdp-2.3.4.0\hdp-2.3.4.0\hdp-2.3.4.0.winpkg.msi /lv c:\MSI_Download\hdp.log HDP_LAYOUT=c:\MSI_Download\Cluster.properties HDP_DIR=c:\hdp HDP=yes DESTROY_DATA=yes USEROOT=yes HDP_USER_PASSWORD=TestUser123 KNOX=no FALCON=no STORM=no RANGER="YesorNo" KNOX_MASTER_SECRET=password
To install a basic cluster with HBase, use the following command on every node:
msiexec /qn /i D:\MSI_Install\hdp-2.3.4.0.winpkg.msi /lv D:\MSI_Install\hdp.log MSIUSEREALADMINDETECTION=1 HDP_LAYOUT=D:\MSI_Install\cluster.properties HDP_DIR=D:\hdp DESTROY_DATA=yes HDP_USER_PASSWORD=password HDP=yes KNOX=no FALCON=no HBase=yes STORM=no FLUME=no RANGER=No KNOX_MASTER_SECRET=password
For a description of command line options, see Standard Installer Options.
The HDP Installer unpacks the MSI contents to
SystemDrive\HadoopInstallFiles
. A detailed installation log is located atSystemDrive\HadoopInstallFiles\HadoopSetupTools\hdp-2.3.0.0.winpkg.install
. This folder is required to uninstall HDP; do not remove it.Warning If you are reinstalling HDP and wish to delete existing data but you did not specify DESTROY_DATA=YES, you need to format the HDFS file system. Do not format the file system if you are upgrading an existing cluster and wish to preserve your data. Formatting will delete your existing HDP data.
To format the HDFS file system, open the Hadoop Command Line shortcut on the Windows desktop, and then enter:
runas /user:hadoop "cmd /K HADOOP_HOME\bin\hadoop namenode -format"