HDP Documentation

  PDF Version

Command Line Installation

1. Preparing to Manually Install HDP
2. Installing Apache ZooKeeper
3. Installing HDFS, YARN, and MapReduce
4. Setting Up the Hadoop Configuration
5. Validating the Core Hadoop Installation
6. Deploying HDP In Production Data Centers With Firewalls
7. Installing Apache HBase
8. Installing Apache Phoenix
9. Installing and Configuring Apache Tez
10. Installing Apache Hive and Apache HCatalog
11. Installing Apache Pig
12. Installing Apache WebHCat
13. Installing Apache Oozie
14. Installing Apache Ranger
15. Installing Hue
16. Installing Apache Sqoop
17. Installing Apache Mahout
18. Installing and Configuring Apache Flume
19. Installing and Configuring Apache Storm
20. Installing and Configuring Apache Spark
21. Installing and Configuring Apache Spark 2
22. Installing and Configuring Apache Kafka
23. Installing and Configuring Zeppelin
24. Installing Apache Accumulo
25. Installing Apache Falcon
26. Installing Apache Knox
27. Installing Apache Slider
28. Setting Up Kerberos Security for Manual Installs
29. Uninstalling HDP

  PDF Version

Command Line Upgrade

1. Upgrade from HDP 2.5 to HDP 2.6.0 Manually
2. Upgrade from HDP 2.4 to HDP 2.6.0 Manually
3. Upgrade from HDP 2.3 to HDP 2.6.0 Manually

  PDF Version

Apache Ambari Upgrade Best Practices

1. Overview
2. Planning Your Upgrade
3. Completing the Preupgrade Verifications
4. Upgrading Your System
5. Completing the Postupgrade Verifications

  PDF Version

Data Governance

1. HDP Data Governance
2. Installing and Configuring Apache Atlas Using Ambari
3. Searching and Viewing Entities
4. Working with Atlas Tags
5. Apache Atlas REST API

  PDF Version

Security

1. HDP Security Overview
2. Authentication
3. Configuring Authorization in Hadoop
4. Data Protection: Wire Encryption
5. Auditing in Hadoop
6. ACLs on HDFS
7. Data Protection: HDFS Encryption
8. Running DataNodes as Non-Root
9. Addendum

  PDF Version

Data Access

1. What's New in Data Access for HDP 2.6
2. Data Warehousing with Apache Hive
3. Enabling Efficient Execution with Apache Pig and Apache Tez
4. Managing Metadata Services with Apache HCatalog
5. Persistent Read/Write Data Access with Apache HBase
6. Orchestrating SQL and APIs with Apache Phoenix
7. Real-Time Data Analytics with Druid

  PDF Version

Apache Hive Performance Tuning

1. Optimizing an Apache Hive Data Warehouse
2. Hive LLAP on Your Cluster
3. Best Practices Prior to Tuning Performance
4. Connectivity and Admission Control
5. Using the Cost-Based Optimizer to Enhance Performance
6. Optimizing the Hive Execution Engine
7. Maximizing Storage Resources
8. Debugging Performance Issues

  PDF Version

Apache Storm Component Guide

1. Analyzing Streams of Data with Apache Storm
2. Installing Apache Storm
3. Configuring Apache Storm for a Production Environment
4. Developing Apache Storm Applications
5. Moving Data Into and Out of Apache Storm Using Spouts and Bolts
6. Packaging Storm Topologies
7. Deploying and Managing Apache Storm Topologies
8. Monitoring and Debugging an Apache Storm Topology
9. Tuning an Apache Storm Topology

  PDF Version

Apache Spark Component Guide

1. Analyzing Data with Apache Spark
2. Installing Spark
3. Configuring Spark
4. Running Spark
5. Submitting Spark Applications Through Livy
6. Running PySpark in a Virtual Environment
7. Automating Spark Jobs with Oozie Spark Action
8. Developing Spark Applications
9. Using Spark from R: SparkR
10. Tuning Spark

  PDF Version

Apache Zeppelin Component Guide

1. Overview
2. Installing Apache Zeppelin
3. Configuring Zeppelin
4. Using Zeppelin
5. Configuring Zeppelin Security
6. Stopping the Zeppelin Server and Livy Server

  PDF Version

Cluster Planning

1. Hardware Recommendations for Apache Hadoop
2. File System Partitioning Recommendations

  PDF Version

Apache Hadoop High Availability

1. High Availability for Hive Metastore
2. Deploying Multiple HiveServer2 Instances for High Availability
3. HiveServer2 High Availability via ZooKeeper
4. High Availability for HBase
5. Namenode High Availability
6. Resource Manager High Availability
7. Apache Ranger High Availability

  PDF Version

Apache YARN Resource Management

1. Capacity Scheduler
2. CGroups
3. Using CPU Scheduling to Allocate Resources
4. Log Aggregation for Long-running Applications
5. Node Labels
6. Running Applications on YARN Using Slider
7. Running Multiple MapReduce Versions Using the YARN Distributed Cache
8. Timeline Server
9. Using the YARN REST APIs to Manage Applications
10. Work-Preserving Restart
11. Using the YARN CLI to View Logs for Running Applications

  PDF Version

HDFS Administration

1. Accidental Deletion Protection
2. ACLs on HDFS
3. Archival Storage
4. Backing Up HDFS Metadata
5. Balancing in HDFS
6. Centralized Cache Management in HDFS
7. Configuring HDFS Compression
8. Configuring Rack Awareness On HDP
9. Customizing HDFS
10. G1GC Garbage Collector Configuration (Technical Preview)
11. Hadoop Archives
12. HDFS DataNode Monitoring
13. JMX Metrics APIs for HDFS Daemons
14. Memory as Storage (Technical Preview)
15. Running DataNodes as Non-Root
16. Short Circuit Local Reads On HDFS
17. WebHDFS Administrator Guide

  PDF Version

HDFS NFS Gateway User Guide

1. HDFS NFS Gateway User Guide

  PDF Version

Administration

1. Decommissioning Slave Nodes
2. Manually Adding Slave Nodes to an HDP Cluster
3. Using DistCp to Copy Files

  PDF Version

Reference

1. Configuring Ports
2. Controlling HDP Services Manually
3. Deploying HDP In Production Data Centers With Firewalls
4. Hadoop Service Accounts
5. Supported Database Matrix for the Hortonworks Data Platform

  PDF Version

Workflow Management

1. Workflow Manager Basics
2. Content Roadmap for Workflow Manager
3. Quick Start
4. Designing Workflows Using the Design Component
5. Monitoring Jobs Using the Dashboard
6. Sample ETL Use Case
7. Workflow Parameters
8. Settings Menu Parameters
9. Job States
10. Workflow Manager Files

  PDF Version

Data Movement and Integration

1. What's New in the Data Movement and Integration Guide
2. HDP Data Movement and Integration
3. Data Management and Falcon Overview
4. Prerequisite to Installing or Upgrading Falcon
5. Considerations for Using Falcon
6. Configuring for High Availability
7. Creating Falcon Entity Definitions
8. Mirroring Data with Falcon
9. Replicating Data with Falcon
10. Mirroring Data with HiveDR in a Secure Environment
11. Enabling Mirroring and Replication with Azure Cloud Services
12. Using Advanced Falcon Features
13. Using Apache Sqoop to Transfer Bulk Data
14. Using HDP for Workflow and Scheduling With Oozie
15. Using Apache Flume for Streaming
16. Troubleshooting
17. Appendix

  PDF Version

Apache Flume Component Guide

1. Feature Updates
2. Introduction
3. Using Apache Flume for Streaming

  PDF Version

Apache Kafka Component Guide

1. Building a High-Throughput Messaging System with Apache Kafka
2. Apache Kafka Concepts
3. Installing Kafka
4. Configuring Kafka for a Production Environment
5. Mirroring Data Between Clusters: Using the MirrorMaker Tool
6. Creating a Kafka Topic
7. Developing Kafka Producers and Consumers

  PDF Version

Cloud Data Access

1. About This Guide
2. Introducing the Cloud Storage Connectors
3. Working with Amazon S3
4. Working with ADLS
5. Working with WASB
6. Working with Google Cloud Storage (Technical Preview)
7. Accessing Cloud Data in Hive
8. Accessing Cloud Data in Spark
9. Copying Cloud Data with Hadoop

  PDF Version

As of December 2017, Hortonworks HDB is no longer available as part of new service subscriptions.

Hortonworks HDB (Powered by Apache HAWQ)

1. Hortonworks HDB (Powered by Apache HAWQ)

  PDF Version

Teradata Connector User Guide

1. What's New in Hortonworks Connector for Teradata
2. Hortonworks Connector for Teradata

  PDF Version

Hortonworks Data Platform Search (Apache Solr)

1. Introduction
2. HDP Search 3.0
3. HDP Search 2.2.10 - Deprecated
4. Upgrading HDP Search