Cloudera Docs
»
2.6.2
»
Command Line Installation
Command Line Installation
Also available as:
Contents
1. Preparing to Manually Install HDP
Meeting Minimum System Requirements
Hardware Recommendations
Operating System Requirements
Software Requirements
JDK Requirements
Manually Installing Oracle JDK 1.7 or 1.8
Manually Installing OpenJDK 1.7
Manually Installing the JCE
Metastore Database Requirements
Metastore Database Prerequisites
Installing and Configuring PostgreSQL
Installing PostgreSQL on RHEL, CentOS, and Oracle Linux
Installing PostgreSQL on SUSE Linux Enterprise Server (SLES)
Installing PostgreSQL on Ubuntu and Debian
Installing and Configuing MariaDB
Installing MariaDB on RHEL and CentOS
Installing and Configuring MySQL
Installing MySQL on RHEL and CentOS
SUSE Linux Enterprise Server (SLES)
Ubuntu/Debian
Configuring Oracle as the Metastore Database
Virtualization and Cloud Platforms
Configuring Remote Repositories
Deciding on a Deployment Type
Collect Information
Prepare the Environment
Enable NTP on Your Cluster
Disable SELinux
Disable IPTables
Download Companion Files
Define Environment Parameters
Creating System Users and Groups
Determining HDP Memory Configuration Settings
Running the YARN Utility Script
Calculating YARN and MapReduce Memory Requirements
Configuring NameNode Heap Size
Allocating Adequate Log Space for HDP
Downloading the HDP Maven Artifacts
2. Installing Apache ZooKeeper
Install the ZooKeeper Package
Securing ZooKeeper with Kerberos (optional)
Securing ZooKeeper Access
ZooKeeper Configuration
YARN Configuration
HDFS Configuration
Set Directories and Permissions
Set Up the Configuration Files
Start ZooKeeper
3. Installing HDFS, YARN, and MapReduce
Set Default File and Directory Permissions
Install the Hadoop Packages
Install Compression Libraries
Install Snappy
Install LZO
Create Directories
Create the NameNode Directories
Create the SecondaryNameNode Directories
Create DataNode and YARN NodeManager Local Directories
Create the Log and PID Directories
HDFS Logs
Yarn Logs
HDFS Process
Yarn Process ID
JobHistory Server Logs
JobHistory Server Process ID
Symlink Directories with hdp-select
4. Setting Up the Hadoop Configuration
5. Validating the Core Hadoop Installation
Format and Start HDFS
Smoke Test HDFS
Configure YARN and MapReduce
Start YARN
Start MapReduce JobHistory Server
Smoke Test MapReduce
6. Deploying HDP In Production Data Centers With Firewalls
Terminology
Mirroring or Proxying
Considerations for choosing a Mirror or Proxy solution
Recommendations for Deploying HDP
Detailed Instructions for Creating Mirrors and Proxies
Option I - Mirror server has no access to the Internet
Option II - Mirror server has temporary or continuous access to the Internet
Set up a trusted proxy server
7. Installing Apache HBase
Install the HBase Package
Set Directories and Permissions
Set Up the Configuration Files
Add Configuration Parameters for Bulk Load Support
Validate the Installation
Starting the HBase Thrift and REST Servers
8. Installing Apache Phoenix
Installing the Phoenix Package
Configuring HBase for Phoenix
Configuring Phoenix to Run in a Secure Cluster
Validating the Phoenix Installation
Troubleshooting Phoenix
9. Installing and Configuring Apache Tez
Prerequisites
Installing the Tez Package
Configuring Tez
Setting Up Tez for the Tez UI
Setting Up Tez for the Tez UI
Deploying the Tez UI
Configuring the Timeline Server URL and Resource Manager UI URL
Hosting the UI in Tomcat
Hosting the UI Using a Standalone Webserver
Additional Steps for the Application Timeline Server
Creating a New Tez View Instance
Validating the Tez Installation
Troubleshooting
10. Installing Apache Hive and Apache HCatalog
Installing the Hive-HCatalog Package
Setting Up the Hive/HCatalog Configuration Files
HDP-Utility script
Configure Hive and HiveServer2 for Tez
Hive-on-Tez Configuration Parameters
Examples of Hive-Related Configuration Properties:
Using Hive-on-Tez with Capacity Scheduler
Setting Up the Database for the Hive Metastore
Setting up RDBMS for use with Hive Metastore
Enabling Tez for Hive Queries
Disabling Tez for Hive Queries
Configuring Tez with the Capacity Scheduler
Validating Hive-on-Tez Installation
Installing Apache Hive LLAP
LLAP Prerequisites
Preparing to Install LLAP
Installing LLAP on an Unsecured Cluster
Installing LLAP on a Secured Cluster
Prerequisites
Installing LLAP on a Secured Cluster
Validating the Installation on a Secured Cluster
Stopping the LLAP Service
Tuning LLAP for Performance
11. Installing Apache Pig
Install the Pig Package
Validate the Installation
12. Installing Apache WebHCat
Install the WebHCat Package
Upload the Pig, Hive and Sqoop tarballs to HDFS
Set Directories and Permissions
Modify WebHCat Configuration Files
Set Up HDFS User and Prepare WebHCat Directories
Validate the Installation
13. Installing Apache Oozie
Install the Oozie Package
Set Directories and Permissions
Set Up the Oozie Configuration Files
For Derby
For MySQL
For PostgreSQL
For Oracle
Configure Your Database for Oozie
Set up the Sharelib
Validate the Installation
Stop and Start Oozie
14. Installing Apache Ranger
Installation Prerequisites
Installing Policy Manager
Install the Ranger Policy Manager
Install the Ranger Policy Administration Service
Start the Ranger Policy Administration Service
Configuring the Ranger Policy Administration Authentication Mode
Configuring Ranger Policy Administration High Availability
Installing UserSync
Using the LDAP Connection Check Tool
LDAP Connection Check Tool Parameters
Input Properties
Discovery of UserSync Properties
Discovery of Authentication Properties
Retrieval of Users and Groups
Output Directory Content
Other UserSync Related Properties
Assumptions
Sample input.properties File
Install UserSync and Start the Service
Installing Ranger Plug-ins
Installing the Ranger HDFS Plug-in
Installing the Ranger YARN Plug-in
Installing the Ranger Kafka Plug-in
Installing the Ranger HBase Plug-in
Installing the Ranger Hive Plug-in
Installing the Ranger Knox Plug-in
Installing the Ranger Storm Plug-in
Installing Ranger in a Kerberized Environment
Creating Keytab and Principals
Before You Begin
Prepare Ranger Admin
Prepare Ranger Lookup
Prepare Ranger Usersync
Prepare Ranger Tagsync
Installing Ranger Services
Prerequisites
Install Ranger Admin
Install Ranger Usersync
Install Ranger Tagsync
Install Ranger KMS
Manually Installing and Enabling the Ranger Plug-ins
Install and Enable Ranger HDFS Plug-in
Install and Enable Ranger Hive Plug-in
Install and Enable Ranger HBase Plug-in
Install and Enable Ranger YARN Plug-in
Install and Enable Ranger Knox Plug-in
Install and Enable Ranger Storm Plug-in
Install and Enable Ranger Kafka Plug-in
Verifying the Installation
15. Installing Hue
Before You Begin
Configure HDP to Support Hue
Install the Hue Packages
Configure Hue to Communicate with the Hadoop Components
Configure the Web Server
Configure Hadoop
Configure Hue for Databases
Using Hue with Oracle
Using Hue with MySQL
Using Hue with PostgreSQL
Start, Stop, and Restart Hue
Validate the Hue Installation
16. Installing Apache Sqoop
Install the Sqoop Package
Set Up the Sqoop Configuration
Validate the Sqoop Installation
17. Installing Apache Mahout
Install Mahout
Validate Mahout
18. Installing and Configuring Apache Flume
Installing Flume
Configuring Flume
Starting Flume
19. Installing and Configuring Apache Storm
Install the Storm Package
Configure Storm
Configure a Process Controller
(Optional) Configure Kerberos Authentication for Storm
(Optional) Configuring Authorization for Storm
Validate the Installation
20. Installing and Configuring Apache Spark
Spark Prerequisites
Installing Spark
Configuring Spark
(Optional) Starting the Spark Thrift Server
(Optional) Configuring Dynamic Resource Allocation
(Optional) Installing and Configuring Livy
Installing Livy
Configuring Livy
Starting, Stopping, and Restarting Livy
Granting Livy the Ability to Impersonate
(Optional) Configuring Zeppelin to Interact with Livy
Validating Spark
21. Installing and Configuring Apache Spark 2
Spark 2 Prerequisites
Installing Spark 2
Configuring Spark 2
(Optional) Starting the Spark 2 Thrift Server
(Optional) Configuring Dynamic Resource Allocation
(Optional) Installing and Configuring Livy
Installing Livy
Configuring Livy
Starting, Stopping, and Restarting Livy
Granting Livy the Ability to Impersonate
(Optional) Configuring Zeppelin to Interact with Livy
Validating Spark 2
22. Installing and Configuring Apache Kafka
Install Kafka
Configure Kafka
Validate Kafka
23. Installing and Configuring Zeppelin
Installation Prerequisites
Installing the Zeppelin Package
Configuring Zeppelin
Starting, Stopping, and Restarting Zeppelin
Validating Zeppelin
Accessing the Zeppelin UI
24. Installing Apache Accumulo
Installing the Accumulo Package
Configuring Accumulo
Configuring the "Hosts" Files
Validating Accumulo
Smoke Testing Accumulo
25. Installing Apache Falcon
Installing the Falcon Package
Setting Directories and Permissions
Configuring Proxy Settings
Configuring Falcon Entities
Configuring Oozie for Falcon
Configuring Hive for Falcon
Configuring for Secure Clusters
Validate Falcon
26. Installing Apache Knox
Install the Knox Package on the Knox Server
Set up and Validate the Knox Gateway Installation
Configuring Knox Single Sign-on (SSO)
27. Installing Apache Slider
28. Setting Up Kerberos Security for Manual Installs
29. Uninstalling HDP
« Prev
Next »
Installing UserSync
In this section:
Using the LDAP Connection Check Tool
Install UserSync and Start the Service
© 2012–2021 by Cloudera, Inc.
Document licensed under the
Creative Commons Attribution ShareAlike 4.0 License
.
Cloudera.com
|
Documentation
|
Support
|
Community