4. Option 3: Installing HDP from the Command Line

Use the following instructions to install a single Hadoop cluster node from the command line using a cluster properties file:

  1. On the host nodes, complete all the prerequisites described in Before You Begin. Be especially careful to identify:

    • Supported operating system

    • Dependent software and environment variable settings

    • Enable PowerShell remote scripting and set cluster nodes as trusted hosts

    • Resolvable host names and static IPv4 addresses

    • Open ports required for HDP operation

  2. Download the HDP Windows Installation package. This package includes a sample cluster properties file, called clusterproperties.txt.

  3. Create a cluster properties file using your host information; see Defining Cluster Properties.

    [Note]Note

    Nodes in the cluster communicate with each other using the host name or IP address defined in the cluster properties file. For multi-homed systems (systems that can be accessed internally and externally) and systems with more than one NIC, ensure that the preferred name or IP address is specified in the Cluster Properties file.

  4. Place the MSI and custom cluster properties file in a local subdirectory on the host. Only the Hadoop Services that match the system's host name in the cluster properties file will be installed.

  5. Use the same cluster properties file on every node in the cluster.

  6. (Optional) When installing HDP with HDFS compression enabled, put the following three files (from the HDP for Windows Installation zip) into the directory that contains the HDP for Windows Installer and the cluster properties file:

    • hadoop-lzo-0.4.19.2.2.0.0-2060

    • gplcompression.dll

    • lzo2.dll

  7. The following two examples show msiexec commands with HDP parameters.

    [Warning]Warning

    These examples assume that you would like to destroy any existing HDP data. If you have existing data that you wish to keep, set DESTROY_DATA to no.

    Open a command prompt with the run as Administrator option, and enter the following:

    msiexec /qn /i c:\MSI_Download\hdp-2.4.0.0\hdp-2.4.0.0\hdp-2.4.0.0.winpkg.msi
    /lv c:\MSI_Download\hdp.log 
    HDP_LAYOUT=c:\MSI_Download\Cluster.properties 
    HDP_DIR=c:\hdp 
    HDP=yes 
    DESTROY_DATA=yes 
    USEROOT=yes 
    HDP_USER_PASSWORD=TestUser123 
    KNOX=no 
    FALCON=no 
    STORM=no 
    RANGER="YesorNo" 
    KNOX_MASTER_SECRET=password

    To install a basic cluster with HBase, use the following command on every node:

    msiexec /qn /i D:\MSI_Install\hdp-2.4.0.0.winpkg.msi 
    /lv D:\MSI_Install\hdp.log 
    MSIUSEREALADMINDETECTION=1 
    HDP_LAYOUT=D:\MSI_Install\cluster.properties 
    HDP_DIR=D:\hdp 
    DESTROY_DATA=yes 
    HDP_USER_PASSWORD=password 
    HDP=yes 
    KNOX=no 
    FALCON=no 
    HBase=yes 
    STORM=no 
    FLUME=no 
    RANGER=No 
    KNOX_MASTER_SECRET=password

    For a description of command line options, see Standard Installer Options.

  8. The HDP Installer unpacks the MSI contents to SystemDrive\HadoopInstallFiles. A detailed installation log is located at SystemDrive\HadoopInstallFiles\HadoopSetupTools\hdp-2.3.0.0.winpkg.install. This folder is required to uninstall HDP; do not remove it.

    [Warning]Warning

    If you are reinstalling HDP and wish to delete existing data but you did not specify DESTROY_DATA=YES, you need to format the HDFS file system. Do not format the file system if you are upgrading an existing cluster and wish to preserve your data. Formatting will delete your existing HDP data.

    To format the HDFS file system, open the Hadoop Command Line shortcut on the Windows desktop, and then enter:

     runas /user:hadoop "cmd /K HADOOP_HOME\bin\hadoop namenode -format"


loading table of contents...