The purpose of authentication in Hadoop, as in other systems, is simply to prove that a user or service is who he or she claims to be.

Typically, authentication in enterprises is managed through a single distributed system, such as a Lightweight Directory Access Protocol (LDAP) directory. LDAP authentication consists of straightforward username/password services backed by a variety of storage systems, ranging from file to database.

A common enterprise-grade authentication system is Kerberos. Kerberos provides strong security benefits including capabilities that render intercepted authentication packets unusable by an attacker. It virtually eliminates the threat of impersonation by never sending a user's credentials in cleartext over the network.

Several components of the Hadoop ecosystem are converging to use Kerberos authentication with the option to manage and store credentials in LDAP or AD. For example, Microsoft's Active Directory (AD) is an LDAP directory that also provides Kerberos authentication for added security.

Before you use this guide to configure Kerberos on your cluster, ensure you have a working KDC (MIT KDC or Active Directory), set up. You can then use Cloudera Manager's Kerberos wizard to automate several aspects of Kerberos configuration on your cluster.

Continue reading:

• Kerberos Principals and Keytabs
• Why Use Cloudera Manager to Implement Hadoop Security?
• Enabling Kerberos Authentication Using Cloudera Manager
• Viewing and Regenerating Kerberos Principals
• Configuring LDAP Group Mappings
• Mapping Kerberos Principals to Short Names
• Troubleshooting Kerberos Security Issues
• Known Kerberos Issues in Cloudera Manager
• Appendix A - Manually Configuring Kerberos Using Cloudera Manager
• Appendix B - Set up a Cluster-dedicated MIT KDC and Default Domain for the Hadoop Cluster
• Appendix C - Hadoop Users in Cloudera Manager