Use the following instructions to deploy Hadoop using HDP:
Complete all the prerequisites provided here.
Download the HDP for Windows MSI from here.
Note Downloaded MSI includes full source, binary-only MSI, and documentation for all components. Hortonworks recommends using the binary-only MSI for faster downloads.
Use the instructions provided here to complete the configuration for HDP installer.
Launch the MSI installer with the
clusterproperties.txt
file created previously.Important This MSI must be executed on each and every cluster node and must use the same
clusterproperties.txt
file.On each node, run a command prompt in Administrator mode (to use Administrator privileges), and execute the following command:
msiexec /i "<$MSI_PATH>" /lv "<$PATH_to_Installer_Log_File>" HDP_LAYOUT="<$PATH_to_clusterproperties.txt_File>" HDP_DIR="<$PATH_to_HDP_Install_Dir>" DESTROY_DATA="<Yes_OR_No>"
Ensure that you provide appropriate values for the following mandatory command line option:
HDP_LAYOUT: Mandatory parameter. Provide location of the
clusterproperties.txt
file (For example,d:\config\clusterproperties.txt
).Important The path to the
clusterproperties.txt
file must be absolute. Relative paths will not work.
Optionally, you can also use the following command line options:
HDP_DIR: Install directory for HDP (For example,
d:\hdp
). Default value is<$Default_Drive>/hdp
.DESTROY_DATA: Specifies whether to preserve or delete existing data in target data directories (allowed values are
undefined
(default),yes
, andno
).The
DESTROY_DATA
parameter takes care of the following conditions:During installation, when
HDP_DATA_DIR
has data from previous installation ifDESTROY_DATA
is set toundefined
, installation will fail.During installation, if
DESTROY_DATA
is set tono
, the installer will reuse the existing data and would not format the NameNode.Note Installer does not check for the data correctness.
During installation, if
DESTROY_DATA
is set toyes
, installation will remove previous data and format the NameNode.During installation, if no data exists in
$HDP_DATA_DIR
then theHDP_DATA_DIR
is created irrespective of the value ofDESTROY_DATA
and NameNode is formatted.
The following example illustrates the command to launch the MSI installer:
msiexec /i "hdp-win-1.1.msi" /lv "hdp.log" HDP_LAYOUT="D:\config\clusterproperties.txt" HDP_DIR="D:\hdp\hadoop" DESTROY_DATA="no"
(Optional): Configure compression for HDFS. Download the
zlib1.dll
from here. Copy the downloaded file to either
or to$HADOOP_HOME
\lib\nativeC:\Windows\System32
. To use GzipCodec, ensure that you copy the downloaded file toC:\Windows\System32
.
Use the instructions provided here to start the HDP Services.
On a cluster node, open a command shell and execute the smoke test command script as shown below:
cd %HADOOP_NODE_INSTALL_ROOT% Run-SmokeTests
The smoke tests validate the installed functionality by executing a set of tests for each HDP component.
Note It is recommended to re-install HDP, if you see installation failures for any HDP component.