Installing HDP Manually
Copyright © 2012, 2013 Hortonworks, Inc.
 | Except where otherwise noted, this document is licensed under Creative Commons Attribution ShareAlike 3.0 License |
Hortonworks Data Platform (HDP) and any of its components are not anticipated to be combined with any hardware, software or data, except as expressly recommended in this documentation.
2013-10-22
Abstract
The Hortonworks Data Platform, powered by Apache Hadoop, is a massively scalable and 100% open source platform for storing, processing and analyzing large volumes of data. It is designed to deal with data from many sources and formats in a very quick, easy and cost-effective manner. The Hortonworks Data Platform consists of the essential set of Apache Hadoop projects including MapReduce, Hadoop Distributed File System (HDFS), HCatalog, Pig, Hive, HBase, Zookeeper and Ambari. Hortonworks is the major contributor of code and patches to many of these projects. These projects have been integrated and tested as part of the Hortonworks Data Platform release process and installation and configuration tools have also been included.
Unlike other providers of platforms built using Apache Hadoop, Hortonworks contributes 100% of our code back to the Apache Software Foundation. The Hortonworks Data Platform is Apache-licensed and completely open source. We sell only expert technical support, training and partner-enablement services. All of our technology is, and will remain free and open source.
Please visit the Hortonworks Data Platform page for more information on Hortonworks technology. For more information on Hortonworks services, please visit either the Support or Training page. Feel free to Contact Us directly to discuss your specific needs.
Contents
- 1. Getting Ready to Install
- 1. Understand the Basics
- 2. Meet Minimum System Requirements
- 3. Configure the Remote Repositories
- 4. Decide on Deployment Type
- 5. Collect Information
- 6. Prepare the Environment
- 7. [Optional] Install MySQL
- 8. Download Companion Files
- 9. Define Environment Parameters
- 10. [Optional] Create System Users and Groups
- 2. Installing HDFS and YARN
- 3. Setting Up the Hadoop Configuration
- 4. Validating the Core Hadoop Installation
- 5. Installing HBase and Zookeeper
- 6. Installing Apache Pig
- 7. Installing Apache Hive and Apache HCatalog
- 8. Installing WebHCat
- 9. Installing Apache Oozie
- 10. Installing Hue
- 11. Installing Apache Sqoop
- 12. Installing Mahout
- 13. Installing and Configuring Flume in HDP
- 14. Installing Ganglia
- 15. Installing Nagios
- 1. Install the Nagios RPMs
- 2. Install the Configuration Files
- 2.1. Extract the Nagios Configuration Files
- 2.2. Create the Nagios Directories
- 2.3. Copy the Configuration Files
- 2.4. Set the Nagios Admin Password
- 2.5. Set the Nagios Admin Email Contact Address
- 2.6. Register the Hadoop Configuration Files
- 2.7. Set Hosts
- 2.8. Set Host Groups
- 2.9. Set Services
- 2.10. Set Status
- 3. Validate the Installation
- 16. Manual Install Appendix: Tarballs
- 17. Upgrade HDP Manually
- 1. Getting Ready to Upgrade
- 2. Upgrade Hadoop
- 3. Migrate the HDP Configurations
- 4. Create Local Directories
- 5. Start HDFS
- 6. Upgrade ZooKeeper
- 7. Upgrade HBase
- 8. Upgrade Hive and HCatalog
- 9. Upgrade Oozie
- 10. Upgrade WebHCat (Templeton)
- 11. Upgrade Pig
- 12. Upgrade Sqoop
- 13. Upgrade Flume
- 14. Upgrade Mahout
- 15. Upgrade Hue
- 18. Setting Up Security for Manual Installs
- 19. Uninstalling HDP
List of Tables
- 1.1. Define Directories for Core Hadoop
- 1.2. Define Directories for Ecosystem Components
- 1.3. Define Users and Groups for Systems
- 1.4. Typical System Users and Groups
- 10.1. Hue Browser Support
- 10.2. Dependencies on the HDP components
- 15.1. Host Group Parameters
- 15.2. Core and Monitoring Hosts
- 15.3. Ecosystem Hosts
- 16.1. RHEL/CentOS 5
- 16.2. RHEL/CentOS 6
- 16.3. SLES 11
- 16.4. Ubuntu 12.04
- 17.1. Hive Metastore Database Backup and Rstore
- 17.2. Oozie Metastore Database Backup and Restore
- 17.3. HDP 1.3.2 Hadoop Core Site (core-site.xml)
- 17.4. HDP 1.3.2 Hadoop Core Site (hdfs-site.xml)
- 17.5. HDP 1.3.2 Configs now in Capacity Scheduler for HDP 2.x (mapred-site.xml)
- 17.6. HDP 1.3.2 Configs now in Capacity Scheduler for HDP 2.x (capacity-scheduler.xml)
- 17.7. HDP 1.3.2 Configs and HDP 2.x for hadoop-env.sh
- 18.1. Service Principals
- 18.2. Service Keytab File Names
- 18.3. core-site.xml
- 18.4. hdfs-site.xml
- 18.5. mapred-site.xml
- 18.6. hbase-site.xml
- 18.7. hive-site.xml
- 18.8. oozie-site.xml
- 18.9. templeton-site.xml
