Installing HDP Manually
Copyright © 2012-2014 Hortonworks, Inc.
 | Except where otherwise noted, this document is licensed under Creative Commons Attribution ShareAlike 3.0 License |
Hortonworks Data Platform (HDP) and any of its components are not anticipated to be combined with any hardware, software or data, except as expressly recommended in this documentation.
2014-04-22
Abstract
The Hortonworks Data Platform, powered by Apache Hadoop, is a massively scalable and 100% open source platform for storing, processing and analyzing large volumes of data. It is designed to deal with data from many sources and formats in a very quick, easy and cost-effective manner. The Hortonworks Data Platform consists of the essential set of Apache Hadoop projects including MapReduce, Hadoop Distributed File System (HDFS), HCatalog, Pig, Hive, HBase, Zookeeper and Ambari. Hortonworks is the major contributor of code and patches to many of these projects. These projects have been integrated and tested as part of the Hortonworks Data Platform release process and installation and configuration tools have also been included.
Unlike other providers of platforms built using Apache Hadoop, Hortonworks contributes 100% of our code back to the Apache Software Foundation. The Hortonworks Data Platform is Apache-licensed and completely open source. We sell only expert technical support, training and partner-enablement services. All of our technology is, and will remain free and open source.
Please visit the Hortonworks Data Platform page for more information on Hortonworks technology. For more information on Hortonworks services, please visit either the Support or Training page. Feel free to Contact Us directly to discuss your specific needs.
Contents
- 1. Getting Ready to Install
- 1. Meet Minimum System Requirements
- 2. Configure the Remote Repositories
- 3. Decide on Deployment Type
- 4. Collect Information
- 5. Prepare the Environment
- 6. Download Companion Files
- 7. Define Environment Parameters
- 8. [Optional] Create System Users and Groups
- 9. Determine HDP Memory Configuration Settings
- 10. Allocate Adequate Log Space for HDP
- 2. Installing HDFS and YARN
- 3. Setting Up the Hadoop Configuration
- 4. Validating the Core Hadoop Installation
- 5. Installing Zookeeper
- 6. Installing HBase
- 7. Installing Phoenix
- 8. Installing Apache Pig
- 9. Installing Apache Hive and Apache HCatalog
- 10. Installing and Configuring Apache Tez
- 11. Installing WebHCat
- 12. Installing Apache Oozie
- 13. Installing Hue
- 14. Installing Apache Sqoop
- 15. Installing Mahout
- 16. Installing and Configuring Flume in HDP
- 17. Installing and Configuring Storm
- 18. Installing Accumulo
- 19. Installing Falcon
- 20. Installing Knox
- 21. Installing Ganglia
- 22. Installing Nagios
- 1. Install the Nagios RPMs
- 2. Install the Configuration Files
- 2.1. Extract the Nagios Configuration Files
- 2.2. Create the Nagios Directories
- 2.3. Copy the Configuration Files
- 2.4. Set the Nagios Admin Password
- 2.5. Set the Nagios Admin Email Contact Address
- 2.6. Register the Hadoop Configuration Files
- 2.7. Set Hosts
- 2.8. Set Host Groups
- 2.9. Set Services
- 2.10. Set Status
- 2.11. Add Templeton Status and Check TCP Wrapper Commands
- 3. Validate the Installation
- 23. Setting Up Security for Manual Installs
- 1. Preparing Kerberos
- 2. Configuring HDP
- 3. Configure secure HBase and ZooKeeper
- 3.1. Configure HBase Master
- 3.2. Create JAAS configuration files
- 3.3. Start HBase and ZooKeeper services
- 3.4. Configure secure client side access for HBase
- 3.5. Optional: Configure client-side operation for secure operation - Thrift Gateway
- 3.6. Optional: Configure client-side operation for secure operation - REST Gateway
- 3.7. Configure HBase for Access Control Lists (ACL)
- 4. Setting up One-Way Trust with Active Directory
- 24. Upgrade from HDP 1.3 to HDP 2.1 Manually
- 1. Getting Ready to Upgrade
- 2. Upgrade Hadoop
- 3. Migrate the HDP Configurations
- 4. Create Local Directories
- 5. Start HDFS
- 6. Upgrade ZooKeeper
- 7. Upgrade HBase
- 8. Upgrade Hive and HCatalog
- 9. Upgrade Oozie
- 10. Upgrade WebHCat (Templeton)
- 11. Upgrade Pig
- 12. Upgrade Sqoop
- 13. Upgrade Flume
- 14. Upgrade Mahout
- 15. Upgrade Hue
- 16. Finalize Upgrade
- 17. Install New HDP 2.1 Services
- 25. Upgrade from HDP 2.0 to HDP 2.1 Manually
- 1. Getting Ready to Upgrade
- 2. Upgrade Hadoop
- 3. Start HDFS
- 4. Upgrade ZooKeeper
- 5. Upgrade HBase
- 6. Upgrade Hive and HCatalog
- 7. Upgrade Oozie
- 8. Upgrade WebHCat (Templeton)
- 9. Upgrade Pig
- 10. Upgrade Sqoop
- 11. Upgrade Flume
- 12. Upgrade Mahout
- 13. Upgrade Hue
- 14. Finalize Upgrade
- 15. Install New HDP 2.1 Services
- 26. Uninstalling HDP
List of Tables
- 1.1. Define Directories for Core Hadoop
- 1.2. Define Directories for Ecosystem Components
- 1.3. Define Users and Groups for Systems
- 1.4. Typical System Users and Groups
- 9.1. Hive-Related Configuration Parameters
- 10.1. Tez Configuration Parameters
- 13.1. Hue Browser Support
- 13.2. Dependencies on the HDP components
- 16.1. Flume 1.4.0 Dependencies
- 22.1. Host Group Parameters
- 22.2. Core and Monitoring Hosts
- 22.3. Ecosystem Hosts
- 23.1. Service Principals
- 23.2. Service Keytab File Names
- 23.3. core-site.xml
- 23.4. core-site.xml
- 23.5. hdfs-site.xml
- 23.6. mapred-site.xml
- 23.7. hbase-site.xml
- 23.8. hive-site.xml
- 23.9. oozie-site.xml
- 23.10. webhcat-site.xml
- 24.1. Hive Metastore Database Backup and Rstore
- 24.2. Oozie Metastore Database Backup and Restore
- 24.3. HDP 1.3.2 Hadoop Core Site (core-site.xml)
- 24.4. HDP 1.3.2 Hadoop Core Site (hdfs-site.xml)
- 24.5. HDP 1.3.2 Configs now in Capacity Scheduler for HDP 2.x (mapred-site.xml)
- 24.6. HDP 1.3.2 Configs now in Capacity Scheduler for HDP 2.x (capacity-scheduler.xml)
- 24.7. HDP 1.3.2 Configs and HDP 2.x for hadoop-env.sh
- 25.1. Hive Metastore Database Backup and Rstore
- 25.2. Oozie Metastore Database Backup and Restore
