4. Option III - Installing HDP from the Command-line

[Note]Note

Before installation you must set an environment variable for JAVA_HOME. Do not install Java in a location that has spaces in the path name.

Use the following instructions to install a single Hadoop Cluster node from the command line using a Cluster Properties file:

  1. On the host nodes, complete all the prerequisites. In Preparing the Environment see the following sections :

  2. Download the HDP Windows Installation package from here, which includes a sample clusterproperties.txt file.

  3. Optionally, download the LZO compression DLL from here.

  4. Create a Cluster Properties file using your host information, see Define Cluster Properties.

    [Important]Important

    Nodes in the cluster communicate with each other using the host name or IP address defined in the cluster properties file. For multi-homed systems (systems that can be access internally and externally) and systems with more than one NIC, ensure that the preferred name or IP address is specified in the Cluster Properties file.

  5. Place the MSI and custom clusterproperties.txt file in a local subdirectory on the host. Only the Hadoop Services that match the system's hostname in the cluster properties file will get installed.

  6. (Optional): When installing HDP with HDFS compression enabled, put the following three files in the same directory as the HDP for Windows Installer and the cluster.properties file:

    • hadoop-lzo-0.4.19.2.1.1.0-1621.jar from the HDP for Windows Installation zip.

    • gplcompression.dll from the HDP for Windows Installation zip.

    • lzo2.dll download from here.

    Open a command prompt with the runas Administrator option, and execute the following command:

    msiexec /qn /i  "msi_file_name" /lv "log_file_name" MSIUSEREALADMINDETECTION=1 HDP_USER_PASSWORD="Password" HDP_LAYOUT="cluster_properties_file" HDP_DIR="install_dir" DESTROY_DATA=yes HDP=yes KNOX=no FALCON=no STORM=no

    See Understanding the HDP MSI Installer Properties for a detailed description of the command line options.

The HDP Installer unpacks the MSI contents to %SystemDrive%\HadoopInstallFiles. A detailed installation log is located at %SystemDrive%\HadoopInstallFiles\HadoopSetupTools\hdp-2.1.5.0.winpkg.install. This folder is required to uninstall HDP, do not remove it.

[Note]Note

If you did not select the "Delete existing HDP data"check box, and you are reinstalling Hadoop the HDFS file system must be formatted.

To format the HDFS file system, open the Hadoop Command Line shortcut on the Windows desktop, then run the following command:

runas /user:hadoop "cmd /K %HADOOP_HOME%\bin\hadoop namenode -format"

loading table of contents...