To run the Hortonworks Data Platform, your system must meet minimum requirements.
Although there is no single hardware requirement for installing HDP, there are some basic guidelines. You can see sample setups here: Hardware Recommendations for Apache Hadoop.
The following operating systems are supported:
64-bit Red Hat Enterprise Linux (RHEL) 5 or 6
64-bit CentOS 5 or 6
64-bit SUSE Linux Enterprise Server (SLES) 11, SP1
On each of your hosts:
yum [for RHEL or CentOS]
zypper [for SLES]
rpm
scp [for multiple node installs]
curl
wget
unzip
tar
pdsh [for multiple node installs over many hosts]
To use external database for Hive or Oozie metastore, ensure that a MySQL or Oracle database is deployed and available.
(By default, Hive and Oozie use Derby database for the metastore.)
Ensure that your database administrator creates the following databases and users. (For instructions on creating users for MySQL, see here.):
If deploying Hive:
hive_dbname: Required if using MySQL database for Hive Metastore.
hive_dbuser
hive_dbpasswd
If deploying Oozie:
oozie_dbname: Required if using MySQL database for Oozie Metastore.
oozie_dbuser
oozie_dbpasswd
Instructions to setup MySQL database
Connect to the host machine where you plan to deploy MySQL instance and from a terminal window, type:
For RHEL and CentOS:
yum install mysql-server
For SLES:
zypper install mysql
Start the instance.
For RHEL and CentOS:
/etc/init.d/mysqld start
For SLES:
/etc/init.d/mysql start
Set the
root
user password and remove unnecessary information fromlog
andSTDOUT
.mysqladmin -u root password ‘$password’
mysqladmin -u root 2>&1 >/dev/null
Install the MySQL connector jar
yum install mysql-connector-java-5.0.8-1
Instructions to configure Oracle database
Download the Oracle JDBC (OJDBC) driver from here and copy the downloaded JAR file to the following locations:
Note The following instructions are for OJDBC driver for Oracle 11g.
If installing Hive: Copy the JAR file to
/usr/lib/hive/lib/
.If using external Oracle database for Oozie metastore: Copy the JAR file to
/usr/lib/oozie/libtools/
.If installing Sqoop: Copy the JAR file to
/usr/lib/sqoop/lib
.
Ensure that the following SQL script is run against your Hive schema:
$master-install-location/gsInstaller/confSupport/sql/oracle/hive-schema-0.10.0.oracle.sql
Instructions on manually creating users for MySQL:
As
root,
use mysql (or other client tool) to create the “dbuser” and grant it adequate privileges.(For access to Hive metastore, create
hive_dbuser
and for access to Oozie metastore, createoozie_dbuser
.CREATE USER 'dbusername'@'%' IDENTIFIED BY 'dbuserpassword'; GRANT ALL PRIVILEGES ON *.* TO 'dbusername'@'%'; flush privileges;
See if you can connect to the database as that user. You are prompted to enter the
dbuserpassword
password above.mysql -u $dbusername -p
Your system must have the correct JDK installed on all the nodes of the cluster. HDP requires Oracle JDK 1.6 update 31.
Use the following instructions to manually install JDK 1.6 update 31:
Check the version. From a terminal window, type:
java -version
(Optional) Uninstall the Java package if the JDK version is less than v1.6 update 31.
rpm -qa | grep java yum remove {java-1.*}
(Optional) Verify that the default Java package is uninstalled.
which java
Download the Oracle 64-bit JDK (jdk-6u31-linux-x64.bin) from the Oracle download site:
wget http://www.oracle.com/technetwork/java/javasebusiness/downloads/java-archive-downloads-javase6-419409.html#jdk-6u31-oth-JPR
Change directory to the location where you downloaded the JDK and run the install.
mkdir /usr/jdk1.6.0_31 cd /usr/jdk1.6.0_31 chmod u+x $JDK_download_directory/jdk-6u31-linux-x64.bin ./$JDK_download_directory/jdk-6u31-linux-x64.bin
Create symbolic links (symlinks) to the JDK.
mkdir /usr/java ln -s /usr/jdk1.6.0_31/jdk1.6.0_31 /usr/java/default ln -s /usr/java/default/bin/java /usr/bin/java
Set up your environment to define
JAVA_HOME
to put the Java Virtual Machine and the Java compiler on your path.export JAVA_HOME=/usr/java/default export PATH=$JAVA_HOME/bin:$PATH
HDP is certified and supported when running on virtual or cloud platforms (for example, VMware vSphere or Amazon Web Services EC2) as long as the respective guest operating system (OS) is supported by HDP and any issues detected on these platforms are reproducible on the same supported OS installed on bare metal.
See Operating Systems Requirements for the list of supported operating systems for HDP.