2. Meet Minimum System Requirements

To run the Hortonworks Data Platform, your system must meet minimum requirements.

 2.1. Hardware Recommendations

Although there is no single hardware requirement for installing HDP, there are some basic guide­lines. You can see sample setups here: Suggested Hardware for a Typical Hadoop Cluster.

 2.2. Operating Systems Requirements

The following operating systems are supported:

  • 64-bit Red Hat Enterprise Linux (RHEL) 5 or 6

  • 64-bit CentOS 5 or 6

  • 64-bit Oracle Linux 5 or 6

  • 64-bit SUSE Linux Enterprise Server (SLES) 11, SP1

  • 64-bit Ubuntu Precise (12.04)

 2.3. Software Requirements

On each of your hosts:

  • yum [for RHEL or CentOS]

  • zypper [for SLES]

  • php_curl [for SLES]

  • apt-get [for Ubuntu]

  • rpm

  • scp

  • curl

  • wget

  • unzip

  • tar

 2.4. Database Requirements

To use external database for Hive or Oozie metastore, have a MySQL, Oracle, or PostgreSQL database deployed and available.

By default, Hive and Oozie use Derby database for its metastore. To use an external database for Hive and Oozie metastore, ensure that a MySQL database is deployed and available.

  • You can choose to use a current instance of MySQL or install a new instance for its use. For more information, see Install MySQL (Optional).

  • For instructions on configuring an existing Oracle database instance, see here.

    [Note]Note

    To deploy a new Oracle instance, consult your database administrator.

  • For instructions on deploying and/or configuring an existing PostgreSQL database instance, see here.

  • Ensure that your database administrator creates the following databases and users.

    • For Hive, ensure that your database administrator creates hive_dbname, hive_dbuser, and hive_dbpasswd.

    • For Oozie, ensure that your database administrator creates oozie_dbname, oozie_dbuser, and oozie_dbpasswd.

    [Note]Note

    For instructions on creating users for MySQL, see here.

 Instructions to configure an Oracle database

  • Run following SQL script against your Hive schema:

    /usr/lib/hive/scripts/metastore/upgrade/oracle/hive-schema-0.12.0.oracle.sql

 Instructions to deploy and configure a PostgreSQL database

  1. Connect to the host machine where you plan to deploy PostgreSQL instance and from a terminal window, type:

    • For RHEL and CentOS:

      yum install postgresql-server
    • For SLES:

      zypper install postgresql-server
    • For Ubuntu:

      apt-get install postgresql-server
  2. Start the instance. For RHEL and CentOS:

    /etc/init.d/postgresql start
    [Note]Note

    For some newer versions of PostgreSQL, you might need to execute the following command:

    /etc/init.d/postgresql initdb
  3. Reconfigure PostgreSQL server:

    1. Edit the /var/lib/pgsql/data/postgresql.conf file and change the value of #listen_addresses = 'localhost' to the following:

      listen_addresses = '*'
    2. Edit the /var/lib/pgsql/data/postgresql.conf file and change the port setting #port = 5432 to the following:

      port = 5432
    3. Edit the /var/lib/pgsql/data/pg_hba.conf and add the following:

      host all all 0.0.0.0/0 trust
    4. Optional - If you are using PostgreSQL v9.1 or later, add the following to the /var/lib/pgsql/data/postgresql.conf file:

      standard_conforming_strings = off

  4. Create users for PostgreSQL server:

    echo "CREATE DATABASE $dbname;" | psql -U postgres
    echo "CREATE USER $user WITH PASSWORD '$passwd';" | psql -U postgres
    echo "GRANT ALL PRIVILEGES ON DATABASE $dbname TO $user;" | psql -U postgres 
    [Note]Note

    For access to Hive metastore, create hive_dbuser and for access to Oozie metastore, create oozie_dbuser.

  5. Run the following SQL script against your Hive schema:

    /usr/lib/hive/scripts/metastore/upgrade/postgres/hive-schema-0.12.0.postgres.sql

 2.5. JDK Requirements

Your system must have the correct JDK installed on all the nodes of the cluster. HDP supports the following JDKs.

  • Oracle JDK 1.6 update 31 64-bit

  • Oracle JDK 7 64-bit

  • OpenJDK 7 64-bit

 2.5.1. Oracle JDK 1.6 update 31

Use the following instructions to manually install JDK 1.6 update 31:

  1. Check the version. From a terminal window, type:

    java -version
  2. Optional - Uninstall the Java package if the JDK version is less than v1.6 update 31.

    rpm -qa | grep java
    yum remove {java-1.*}
  3. Optional - Verify that the default Java package is uninstalled.

    which java
  4. Download the Oracle 64-bit JDK (jdk-6u31-linux-x64.bin) from the Oracle download site. From your browser window, go to http://www.oracle.com/technetwork/java/javasebusiness/downloads/java-archive-downloads-javase6-419409.html#jdk-6u31-oth-JPR.

    Accept the license agreement and download jdk-6u31-linux-x64.bin to a temporary directory ($JDK_download_directory).

  5. Change directory to the location where you downloaded the JDK and run the install.

    mkdir /usr/jdk1.6.0_31
    cd /usr/jdk1.6.0_31
    chmod u+x $JDK_download_directory/jdk-6u31-linux-x64.bin
    ./$JDK_download_directory/jdk-6u31-linux-x64.bin 
    
  6. Create symbolic links (symlinks) to the JDK.

    mkdir /usr/java
    ln -s /usr/jdk1.6.0_31/jdk1.6.0_31 /usr/java/default
    ln -s /usr/java/default/bin/java /usr/bin/java
    
  7. Set up your environment to define JAVA_HOME to put the Java Virtual Machine and the Java compiler on your path.

    export JAVA_HOME=/usr/java/default
    export PATH=$JAVA_HOME/bin:$PATH
    
  8. Verify if Java is installed in your environment. Execute the following from the command line console:

    java -version

    You should see the following output:

    java version "1.6.0_31"
    Java(TM) SE Runtime Environment (build 1.6.0_31-b04)
    Java HotSpot(TM) 64-Bit Server VM (build 20.6-b01, mixed mode)

 2.5.2. Oracle JDK 7 update 40

Use the following instructions to manually install JDK 7:

  1. Check the version. From a terminal window, type:

    java -version
  2. (Optional) Uninstall the Java package if the JDK version is less than 7.

    rpm -qa | grep java
    yum remove {java-1.*}
  3. (Optional) Verify that the default Java package is uninstalled.

    which java
  4. Download the Oracle 64-bit JDK (jdk-7u40-linux-x64.tar.gz) from the Oracle download site. From your browser window, go to http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-1880260.html.

    Accept the license agreement and download jdk-7u40-linux-x64.tar.gz to a temporary directory ($JDK_download_directory).

  5. Change directory to the location where you downloaded the JDK and run the install.

    mkdir /usr/jdk1.7.0_40
    cd /usr/jdk1.7.0_40
    chmod u+x $JDK_download_directory/jdk-7u40-linux-x64.bin
    ./$JDK_download_directory/jdk-7u40-linux-x64.bin
    
  6. Create symbolic links (symlinks) to the JDK.

    mkdir /usr/java
    ln -s /usr/jdk1.7.0_40/jdk1.7.0_40 /usr/java/default
    ln -s /usr/java/default/bin/java /usr/bin/java
    
  7. Set up your environment to define JAVA_HOME to put the Java Virtual Machine and the Java compiler on your path.

    export JAVA_HOME=/usr/java/default
    export PATH=$JAVA_HOME/bin:$PATH
    
  8. Verify if Java is installed in your environment. Execute the following from the command line console:

    java -version

    You should see the following output:

    java version "1.7.0_40"
    Java(TM) SE Runtime Environment (build 1.7.0_40-b04)
    Java HotSpot(TM) 64-Bit Server VM (build 20.6-b01, mixed mode)

 2.5.3. OpenJDK 7

Use the following instructions to manually install OpenJDK 7:

  1. Check the version. From a terminal window, type:

    java -version
  2. (Optional) Uninstall the Java package if the JDK version is less than 7.

    rpm -qa | grep java
    yum remove {java-1.*}
  3. (Optional) Verify that the default Java package is uninstalled.

    which java
  4. Download OpenJDK 7 RPMs. From the command-line, run:

    yum install java-1.7.0-openjdk java-1.7.0-openjdk-devel
  5. Create symbolic links (symlinks) to the JDK.

    mkdir /usr/java
    ln -s /usr/openjdk1.7.0/openjdk1.7.0 /usr/java/default
    ln -s /usr/java/default/bin/java /usr/bin/java
    
  6. Set up your environment to define JAVA_HOME to put the Java Virtual Machine and the Java compiler on your path.

    export JAVA_HOME=/usr/java/default
    export PATH=$JAVA_HOME/bin:$PATH
    
  7. Verify if Java is installed in your environment. Execute the following from the command-line console:

    java -version

    You should see output similar to the following:

    openjdk version "1.7.0"
    OpenJDK Runtime Environment (build 1.7.0)
    OpenJDK Client VM (build 20.6-b01, mixed mode)

 2.6. Virtualization and Cloud Platforms

HDP is certified and supported when running on virtual or cloud platforms (for example, VMware vSphere or Amazon Web Services EC2) as long as the respective guest operating system (OS) is supported by HDP and any issues detected on these platforms are reproducible on the same supported OS installed on bare metal.

See Operating Systems Requirements for the list of supported operating systems for HDP.


loading table of contents...