Setting Up Knox for WebHDFS HA
This chapter describes how to set up the Knox Gateway for WebHDFS HA (high availability).
To set up Knox for WebHDFS HA:
- 1. HDP Security Overview
- 2. Authentication
- Enabling Kerberos Authentication Using Ambari
- Configuring Ambari Authentication with LDAP or AD
- Configuring LDAP Authentication in Hue
- Enabling the LDAP Backend
- Enabling User Authentication with Search Bind
- Setting the Search Base to Find Users and Groups
- Specifying the URL of the LDAP Server
- Specifying LDAPS and StartTLS Support
- Specifying Bind Credentials for LDAP Searches
- Synchronizing Users and Groups
- Setting Search Bind Authentication and Importing Users and Groups
- Setting LDAP Users' Filter
- Setting an LDAP Groups Filter
- Setting Multiple LDAP Servers
- Advanced Security Options for Ambari
- Enabling SPNEGO Authentication for Hadoop
- Setting Up Kerberos Authentication for Non-Ambari Clusters
- Preparing Kerberos
- Configuring HDP for Kerberos
- Creating Mappings Between Principals and UNIX Usernames
- Adding Security Information to Configuration Files
- Configuring HBase and ZooKeeper
- Configure HBase Master
- Create JAAS configuration files
- Start HBase and ZooKeeper services
- Configure secure client side access for HBase
- Optional: Configure client-side operation for secure operation - Thrift Gateway
- Optional: Configure client-side operation for secure operation - REST Gateway
- Configure HBase for Access Control Lists (ACL)
- Configuring Hue
- Configuring Phoenix Query Server
- Setting up One-Way Trust with Active Directory
- Configuring Proxy Users
- Perimeter Security with Apache Knox
- Apache Knox Gateway Overview
- Configuring the Knox Gateway
- Defining Cluster Topologies
- Configuring a Hadoop Server for Knox
- Mapping the Internal Nodes to External URLs
- Configuring Authentication
- Configuring Identity Assertion
- Configuring Service Level Authorization
- Audit Gateway Activity
- Gateway Security
- Setting Up Knox for WebHDFS HA
- Knox CLI Testing Tools
- 3. Configuring Authorization in Hadoop
- Installing Ranger Using Ambari
- Overview
- Installation Prerequisites
- Ranger Installation
- Start the Installation
- Customize Services
- Complete the Ranger Installation
- Advanced Usersync Settings
- Configuring Ranger for LDAP SSL
- Setting up Database Users Without Sharing DBA Credentials
- Updating Ranger Admin Passwords
- Enabling Ranger Plugins
- Ranger Plugins - Kerberos Overview
- Using Ranger to Provide Authorization in Hadoop
- Opening and Closing the Ranger Console
- Console Operations Summary
- Configuring Services
- Policy Management
- Users/Groups and Permissions Administration
- Reports Administration
- Special Requirements for High Availability Environments
- Adding a New Component to Apache Ranger
- Developing a Custom Authorization Module
- Apache Ranger Public REST API
- Installing Ranger Using Ambari
- 4. Data Protection: Wire Encryption
- Enabling RPC Encryption
- Enabling Data Transfer Protocol
- Enabling SSL: Understanding the Hadoop SSL Keystore Factory
- Creating and Managing SSL Certificates
- Enabling SSL for HDP Components
- Enable SSL for WebHDFS, MapReduce Shuffle, and YARN
- Enable SSL for HttpFS
- Enable SSL on Oozie
- Enable SSL on the HBase REST Server
- Enable SSL on the HBase Web UI
- Enable SSL on HiveServer2
- Enable SSL for Kafka Clients
- Enable SSL for Accumulo
- SPNEGO setup for WebHCat
- Configure SSL for Hue
- Configure SSL for Knox
- Securing Phoenix
- Set Up SSL for Ambari
- Configure Ambari Ranger SSL
- Configure Non-Ambari Ranger SSL
- Connecting to SSL-Enabled Components
- 5. Auditing in Hadoop
- 6. Data Protection: HDFS Encryption
- Ranger KMS Administration Guide
- HDFS "Data at Rest" Encryption
- HDFS Encryption Overview
- Configuring and Starting the Ranger Key Management Service (Ranger KMS)
- Configuring and Using HDFS Data at Rest Encryption
- Configuring HDP Services for HDFS Encryption
- Appendix: Creating an HDFS Admin User