Validating Spark
To validate the Spark installation, run the Spark jobs described in the Running Sample Spark 1.x Applications section of the Spark Component Guide.
- 1. Preparing to Manually Install HDP
- Meeting Minimum System Requirements
- Virtualization and Cloud Platforms
- Configuring Remote Repositories
- Deciding on a Deployment Type
- Collect Information
- Prepare the Environment
- Download Companion Files
- Define Environment Parameters
- Creating System Users and Groups
- Determining HDP Memory Configuration Settings
- Configuring NameNode Heap Size
- Allocating Adequate Log Space for HDP
- Downloading the HDP Maven Artifacts
- 2. Installing Apache ZooKeeper
- 3. Installing HDFS, YARN, and MapReduce
- 4. Setting Up the Hadoop Configuration
- 5. Validating the Core Hadoop Installation
- 6. Deploying HDP In Production Data Centers With Firewalls
- 7. Installing Apache HBase
- 8. Installing Apache Phoenix
- 9. Installing and Configuring Apache Tez
- 10. Installing Apache Hive and Apache HCatalog
- Installing the Hive-HCatalog Package
- Setting Up the Hive/HCatalog Configuration Files
- Setting Up the Database for the Hive Metastore
- Setting up RDBMS for use with Hive Metastore
- Enabling Tez for Hive Queries
- Disabling Tez for Hive Queries
- Configuring Tez with the Capacity Scheduler
- Validating Hive-on-Tez Installation
- Installing Apache Hive LLAP
- LLAP Prerequisites
- Preparing to Install LLAP
- Installing LLAP on an Unsecured Cluster
- Installing LLAP on a Secured Cluster
- Stopping the LLAP Service
- Tuning LLAP for Performance
- 11. Installing Apache Pig
- 12. Installing Apache WebHCat
- 13. Installing Apache Oozie
- 14. Installing Apache Ranger
- Installation Prerequisites
- Installing Policy Manager
- Installing UserSync
- Installing Ranger Plug-ins
- Installing Ranger in a Kerberized Environment
- Verifying the Installation
- 15. Installing Hue
- 16. Installing Apache Sqoop
- 17. Installing Apache Mahout
- 18. Installing and Configuring Apache Flume
- 19. Installing and Configuring Apache Storm
- 20. Installing and Configuring Apache Spark
- 21. Installing and Configuring Apache Spark 2
- 22. Installing and Configuring Apache Kafka
- 23. Installing and Configuring Zeppelin
- 24. Installing Apache Accumulo
- 25. Installing Apache Falcon
- 26. Installing Apache Knox
- 27. Installing Apache Slider
- 28. Setting Up Kerberos Security for Manual Installs
- 29. Uninstalling HDP