Hortonworks Data Platform

Installing HDP Manually

2013-01-14


Contents

1. Getting Ready to Install
1. Understand the Basics
2. Meet Minimum System Requirements
2.1. Hardware Recommendations
2.2. Operating Systems Requirements
2.3. Software Requirements
2.4. Database Requirements
2.5. JDK Requirements
3. Configure the Remote Repository
4. Decide on Deployment Type
5. Collect Information
6. Prepare the Environment
6.1. Enable NTP on the Cluster
6.2. Check DNS
6.3. Disable SELinux
7. [Optional] Configure the Local Repositories
8. Install the Java Development Kit
9. Create Service Users and Groups
10. Download Companion Files
11. Define Environment Parameters
11.1. Define Users and Groups
11.2. Define Directories
2. Installing HDFS and MapReduce
1. Set Default File and Directory Permissions
2. Install the Hadoop RPMs
3. Install Compression Libraries
3.1. Install Snappy
3.2. Install LZO
4. Create Directories
4.1. Create the NameNode Directories
4.2. Create the SecondaryNameNode Directories
4.3. Create the DataNode and MapReduce Local Directories
4.4. Create the Log and PID Directories
3. Setting Up the Hadoop Configuration
1. Extract the Core Hadoop Configuration Files
2. Modify the Configuration Files
3. Copy the Configuration Files
4. Validating the Core Hadoop Installation
1. Format and Start HDFS
2. Smoke Test HDFS
3. Start MapReduce
4. Smoke Test MapReduce
5. Installing Apache Pig
1. Install the Pig RPMs
2. Set Up Configuration Files
2.1. Extract the Pig Configuration Files
2.2. Copy the Configuration Files
3. Validate the Installation
3.1. Smoke Test Pig
6. Installing Apache Hive and Apache HCatalog
1. Install the Hive and HCatalog RPMs
2. Set Directories and Permissions
2.1. Create Log Directories
3. Set Up the Hive/HCatalog Configuration Files
3.1. Extract the Hive/HCatalog Configuration Files
3.2. Modify the Configuration Files
3.3. Copy the Configuration Files
4. Create Directories on HDFS
5. Download The Database Connector
6. Validate the Installation
7. Installing WebHCat
1. Install the WebHCat RPMs
2. Set Directories and Permissions
3. Modify WebHCat Config Files
4. Set Up the WebHCat Configuration Files
4.1. Extract the WebHCat Configuration Files
4.2. Copy the Configuration Files
5. Set Up the HDFS User and Prepare WebHCat Directories On HDFS
6. Validate the Installation
8. Installing Apache Oozie
1. Install the Oozie RPMs
1.1. Optional: Add Database Connector JAR Files
2. Set Directories and Permissions
2.1. Create Log Directories
3. Set Up the Oozie Configuration Files
3.1. Extract the Oozie Configuration Files
3.2. Modify the Configuration Files
3.3. Copy the Configuration Files
4. Initialize Database Schema
5. Install the Oozie Sharelib in Hadoop HDFS
6. Validate the Installation
9. Installing HBase and ZooKeeper
1. Install the HBase and ZooKeeper RPMs
2. Set Directories and Permissions
2.1. Create Log and PID Directories
3. Set Up the Configuration Files
3.1. Extract the HBase/ZooKeeper Configuration Files
3.2. Modify the Configuration Files
3.3. Copy the Configuration Files
4. Validate the Installation
4.1. Start HBase and ZooKeeper
4.2. Smoke Test HBase and ZooKeeper
10. Installing Apache Sqoop
1. Install the Sqoop RPMs
2. Optional: Download Database Connector
3. Set Up the Sqoop Configuration
3.1. Extract the Sqoop Configuration Files
3.2. Copy the Configuration Files
4. Validate the Installation
11. Installing Ganglia
1. Install the Ganglia RPMs
2. Install the Configuration Files
2.1. Extract the Ganglia Configuration Files
2.2. Copy the Configuration Files
2.3. Set Up Ganglia Hosts
2.4. Set Up Configurations
2.5. Set Up Hadoop Metrics
3. Validate the Installation
3.1. Start the Ganglia Server
3.2. Start Ganglia Monitoring on All Hosts
3.3. Confirm that Ganglia is Running
12. Installing Nagios
1. Install the Nagio RPMs
2. Install the Configuration Files
2.1. Extract the Nagios Configuration Files
2.2. Copy the Configuration Files
2.3. Set the Nagios Admin Password
2.4. Set the Nagios Admin Email Contact Address
2.5. Register the Hadoop Configuration Files
2.6. Set Hosts
2.7. Set Host Groups
2.8. Set Services
2.9. Set Status
3. Validate the Installation
3.1. Start Nagios
3.2. Confirm Nagios is Running
3.3. Test Nagios Services
3.4. Test Nagios Access
3.5. Test Nagios Alerts
13. Setting Up Security for Manual Installs
1. Preparing Kerberos
1.1. Kerberos Overview
1.2. Installing and Configuring the KDC
1.3. Creating the Database and Setting Up the First Administrator
1.4. Creating Service Principals and Keytab Files for HDP
1.5. Providing the jce-6 Security Jars
2. Configuring HDP
2.1. Configuration Overview
2.2. Creating Mappings Between Principals and OS Service Usernames
2.3. Adding Security Information to Configuration Files

loading table of contents...