Hadoop Users (user:group) and Kerberos Principals

During the Cloudera Manager installation process, several Linux user accounts and groups are created by default. These are listed in the table below. Integrating the cluster to use Kerberos for authentication requires creating Kerberos principals and keytabs for these user accounts.

Table 1. Users and Groups
Component (Version) Unix User ID Groups Functionality
Apache Atlas atlas atlas, hadoop Apache Atlas by default has atlas as user and group. It is configurable
Apache Flink flink flink The Flink Dashboard runs as this user.
Apache HBase hbase hbase The Master and the RegionServer processes run as this user.
Apache HBase Indexer hbase hbase The indexer servers are run as this user.
Apache HDFS hdfs hdfs, hadoop The NameNode and DataNodes run as this user, and the HDFS root directory as well as the directories used for edit logs should be owned by it.

Apache Hive

Hive on Tez

hive hive The HiveServer2 process and the Hive Metastore processes run as this user.A user must be defined for Hive access to its Metastore DB (for example, MySQL or Postgres) but it can be any identifier and does not correspond to a Unix uid. This is javax.jdo.option.ConnectionUserName in hive-site.xml.
Apache Impala impala impala, hive Impala services run as this user.
Apache Kafka kafka kafka Kafka brokers, mirrorMaker, and Connect workers run as this user.
Apache Knox knox knox Apache Knox Gateway Server runs as this user
Apache Kudu kudu kudu Kudu services run as this user.
Apache Livy livy livy The Livy Server process runs as this user
Apache NiFi nifi nifi Runs as the nifi user
Apache NiFi Registry nifiregistry nifiregistry Runs as the nifiregistry user
Apache Oozie oozie oozie The Oozie service runs as this user.
Apache Ozone hdfs hdfs, hadoop Ozone Manager, Storage Container Manager (SCM), Recon and Ozone Datanodes run as this user.
Apache Parquet ~ ~ No special users.
Apache Phoenix phoenix phoenix The Phoenix Query Server runs as this user
Apache Ranger ranger ranger, hadoop Ranger Admin, Usersync and Tagsync services by default have ranger as user and ranger, hadoop as groups. It is configurable.
Apache Ranger KMS kms kms Ranger KMS runs with kms user and group. It is configurable.
Apache Ranger Raz rangerraz ranger Ranger Raz runs with rangerraz user and is part of the ranger group.
Apache Ranger RMS rangerrms ranger Ranger RMS runs with rangerrms user and is part of the ranger group.
Apache Solr solr solr The Solr processes run as this user.
Apache Spark spark spark The Spark History Server process runs as this user.
Apache Sqoop sqoop sqoop This user is only for the Sqoop1 Metastore, a configuration option that is not recommended.
Apache YARN yarn yarn, hadoop Without Kerberos, all YARN services and applications run as this user. The LinuxContainerExecutor binary is owned by this user for Kerberos.
Apache Zeppelin zeppelin zeppelin The Zeppelin Server process runs as this user
Apache ZooKeeper zookeeper zookeeper The ZooKeeper processes run as this user. It is not configurable.
Cloudera Manager (all versions) cloudera-scm cloudera-scm Clusters managed by Cloudera Manager run Cloudera Manager Server, monitoring roles, and other Cloudera Server processes as cloudera-scm. Requires keytab file named cmf.keytab because name is hard-coded in Cloudera Manager.
Cruise Control cruisecontrol hadoop The Cruise Control process runs as this user.
HttpFS httpfs httpfs The HttpFS service runs as this user. See HttpFS authentication for instructions on how to generate the merged httpfs-http.keytab file.
Hue hue hue Hue services run as this user.
Hue Load Balancer apache apache The Hue Load balancer has a dependency on the apache2 package that uses the apache user name. Cloudera Manager does not run processes using this user ID.
Key Trustee Server keytrustee keytrustee The Key Trustee Server service runs as this user.
Schema Registry schemaregistry hadoop The Schema Registry process runs as this user.
Streams Messaging Manager streamsmsgmgr streamsmsgmgr The Streams Messaging Manager processes runs as this user.
Streams Replication Manager streamsrepmgr streamsrepmgr The Streams Replication Manager processes runs as this user.

Keytabs and Keytab File Permissions

Linux user accounts, such as hdfs, are mapped to the username portion of the Kerberos principal names, as follows:
username/host.example.com@EXAMPLE.COM
For example, the Kerberos principal for Apache Hive would be:
hive/host.example.com@EXAMPLE.COM

Keytabs that contain multiple principals are merged automatically from individual keytabs by Cloudera Manager. If you do not use Cloudera Manager, you must merge the keytabs manually.

For example, for Filename (*.keytab), the Atlas keytab filename would be atlas.keytab, HBase would be hbase.keytab, and Cloudera Manager would be cmf.keytab and scm.keytab.

If you are providing your own keytabs, you will not use the Keytab File Owner:Group values listed in the tables; instead ensure that the Cloudera Manager user (cloudera-scm) has read access to the keytabs that you are creating.

Keytab File Owner:Group matters when Cloudera Manager starts a role. For example, Cloudera Manager starts the role "DataNode"". Cloudera Manager launches the DataNode process as a user (here, "hdfs"). Because that process needs to access the HDFS keytab, Cloudera Manager puts the HDFS keytab in the DataNode's process directory, and the keytab is given the owner:group that is listed in the table. Thus, the DataNode process properly owns the keytab file.

The tables below lists the usernames to use for Kerberos principal names, for clusters managed by Cloudera Manager.

Apache Atlas

Role: atlas-ATLAS_SERVER
Kerberos Principals
atlas
Filename (*.keytab)
atlas
Keytab File Owner:Group
atlas:atlas
File Permission (octal)
600

Apache Flink

Role: flink
Kerberos Principals
flink
Filename (*.keytab)
flink
Keytab File Owner:Group
flink:flink
File Permission (octal)
600

Apache HBase

Role: hbase-HBASETHRIFTSERVER
Kerberos Principals
hbase, HTTP
Filename (*.keytab)
hbase, HTTP
Keytab File Owner:Group
hbase:hbase
File Permission (octal)
600
Role: hbase-REGIONSERVER
Kerberos Principals
hbase, HTTP
Filename (*.keytab)
hbase, HTTP
Keytab File Owner:Group
hbase:hbase
File Permission (octal)
600
Role: hbase-HBASERESTSERVER
Kerberos Principals
hbase, HTTP
Filename (*.keytab)
hbase, HTTP
Keytab File Owner:Group
hbase:hbase
File Permission (octal)
600
Role: hbase-MASTER
Kerberos Principals
hbase, HTTP
Filename (*.keytab)
hbase, HTTP
Keytab File Owner:Group
hbase:hbase
File Permission (octal)
600

Apache HBase indexer

Role: ks_indexer-HBASE_INDEXER
Kerberos Principals
hbase, HTTP
Filename (*.keytab)
hbase
Keytab File Owner:Group
hbase:hbase
File Permission (octal)
600

Apache HDFS

Role: hdfs-NAMENODE
Kerberos Principals
hdfs, HTTP
Filename (*.keytab)
hdfs
Keytab File Owner:Group
hdfs:hdfs
File Permission (octal)
600
Role: hdfs-DATANODE
Kerberos Principals
hdfs, HTTP
Filename (*.keytab)
hdfs
Keytab File Owner:Group
hdfs:hdfs
File Permission (octal)
600
Role: hdfs-SECONDARYNAMENODE
Kerberos Principals
hdfs, HTTP
Filename (*.keytab)
hdfs
Keytab File Owner:Group
hdfs:hdfs
File Permission (octal)
600

Apache Hive, Hive on Tez

Role: hive-HIVESERVER2
Kerberos Principals
hive
Filename (*.keytab)
hive
Keytab File Owner:Group
hive:hive
File Permission (octal)
600
Role: hive-HIVEMETASTORE
Kerberos Principals
hive
Filename (*.keytab)
hive
Keytab File Owner:Group
cloudera-scm:cloudera-scm
File Permission (octal)
600

Apache Impala

Role: impala-STATESTORE
Kerberos Principals
impala, HTTP
Filename (*.keytab)
impala
Keytab File Owner:Group
impala:impala
File Permission (octal)
600
Role: impala-CATALOGSERVER
Kerberos Principals
impala, HTTP
Filename (*.keytab)
impala
Keytab File Owner:Group
impala:impala
File Permission (octal)
600
Role: impala-IMPALAD
Kerberos Principals
impala, HTTP
Filename (*.keytab)
impala
Keytab File Owner:Group
impala:impala
File Permission (octal)
600

Apache Kafka

Role: kafka-KAFKA_BROKER
Kerberos Principals
kafka
Filename (*.keytab)
kafka
Keytab File Owner:Group
kafka:kafka
File Permission (octal)
600
Role: kafka-KAFKA_MIRROR_MAKER
Kerberos Principals
kafka_mirror_maker
Filename (*.keytab)
kafka
Keytab File Owner:Group
kafka:kafka
File Permission (octal)
600
Role: kafka-KAFKA_CONNECT
Kerberos Principals
kafka
Filename (*.keytab)
kafka
Keytab File Owner:Group
kafka:kafka
File Permission (octal)
600

Apache Knox

Role: knox-KNOX_GATEWAY
Kerberos Principals
knox, HTTP
Filename (*.keytab)
hbase
Keytab File Owner:Group
knox:knox
File Permission (octal)
600

Apache Kudu

Role: kudu-KUDU_MASTER
Kerberos Principals
kudu
Filename (*.keytab)
kudu
Keytab File Owner:Group
kudu:kudu
File Permission (octal)
600
Role: kudu-KUDU_TSERVER
Kerberos Principals
kudu
Filename (*.keytab)
kudu
Keytab File Owner:Group
kudu:kudu
File Permission (octal)
600

Apache Livy

Role: livy-LIVY_SERVER
Kerberos Principals
livy
Filename (*.keytab)
livy
Keytab File Owner:Group
livy:livy
File Permission (octal)
600

Apache NiFi

Role: nifi
Kerberos Principals
nifi, HTTP
Filename (*.keytab)
nifi
Keytab File Owner:Group
nifi:nifi
File Permission (octal)
600

Apache NiFi Registry

Role: nifiregistry
Kerberos Principals
nifiregistry, HTTP
Filename (*.keytab)
nifiregistry
Keytab File Owner:Group
nifiregistry:nifiregistry
File Permission (octal)
600

Apache Oozie

Role: oozie-OOZIE_SERVER
Kerberos Principals
oozie, HTTP
Filename (*.keytab)
oozie
Keytab File Owner:Group
oozie:oozie
File Permission (octal)
600

Apache Ozone

Role: ozone-OZONE_MANAGER
Kerberos Principals
om, HTTP
Filename (*.keytab)
ozone
Keytab File Owner:Group
hdfs:hdfs
File Permission (octal)
600
Role: ozone-STORAGE_CONTAINER_MANAGER
Kerberos Principals
scm, HTTP
Filename (*.keytab)
ozone
Keytab File Owner:Group
hdfs:hdfs
File Permission (octal)
600
Role: ozone-OZONE_DATANODE
Kerberos Principals
dn, HTTP
Filename (*.keytab)
ozone
Keytab File Owner:Group
hdfs:hdfs
File Permission (octal)
600
Role: ozone-OZONE_RECON
Kerberos Principals
recon, HTTP
Filename (*.keytab)
ozone
Keytab File Owner:Group
hdfs:hdfs
File Permission (octal)
600
Role: ozone-S3_GATEWAY
Kerberos Principals
HTTP
Filename (*.keytab)
ozone
Keytab File Owner:Group
hdfs:hdfs
File Permission (octal)
600

Apache Phoenix

Role: phoenix-PHOENIX_QUERY_SERVER
Kerberos Principals
phoenix, HTTP
Filename (*.keytab)
phoenix
Keytab File Owner:Group
phoenix:phoenix
File Permission (octal)
600

Apache Ranger

Role: ranger-RANGER_ADMIN
Kerberos Principals
rangeradmin, rangerlookup, HTTP
Filename (*.keytab)
ranger
Keytab File Owner:Group
ranger:ranger
File Permission (octal)
600
Role: ranger-RANGER_USERSYNC
Kerberos Principals
rangerusersync
Filename (*.keytab)
ranger
Keytab File Owner:Group
ranger:ranger
File Permission (octal)
600
Role: ranger-RANGER_TAGSYNC
Kerberos Principals
rangertagsync
Filename (*.keytab)
ranger
Keytab File Owner:Group
ranger:ranger
File Permission (octal)
600

Apache Ranger KMS

Role: ranger-RANGER_TAGSYNC
Kerberos Principals
rangerkms, HTTP
Filename (*.keytab)
ranger_kms
Keytab File Owner:Group
kms:kms
File Permission (octal)
600

Apache Ranger Raz

Role: ranger-RANGER_RAZ
Kerberos Principals
rangerraz, HTTP
Filename (*.keytab)
rangerraz
Keytab File Owner:Group
ranger:rangerraz
File Permission (octal)
600

Apache Ranger RMS

Role: ranger-RANGER_RMS
Kerberos Principals
rangerrms
Filename (*.keytab)
rangerrms
Keytab File Owner:Group
ranger:rangerrms
File Permission (octal)
600

Apache Solr

Role: solr-SOLR_SERVER
Kerberos Principals
solr, HTTP
Filename (*.keytab)
solr
Keytab File Owner:Group
solr:solr
File Permission (octal)
600

Apache Spark

Role: spark_on_yarn-SPARK_YARN_HISTORY_SERVER
Kerberos Principals
spark
Filename (*.keytab)
spark
Keytab File Owner:Group
spark:spark
File Permission (octal)
600

Apache YARN

Role: yarn-NODEMANAGER
Kerberos Principals
yarn, HTTP
Filename (*.keytab)
yarn
Keytab File Owner:Group
yarn:hadoop
File Permission (octal)
644
Role: yarn-RESOURCEMANAGER
Kerberos Principals
yarn, HTTP
Filename (*.keytab)
yarn
Keytab File Owner:Group
yarn:hadoop
File Permission (octal)
600
Role: yarn-JOBHISTORY
Kerberos Principals
mapred
Filename (*.keytab)
mapred
Keytab File Owner:Group
yarn:hadoop
File Permission (octal)
600

Apache Zeppelin

Role: zeppelin-ZEPPELIN_SERVER
Kerberos Principals
zeppelin, HTTP
Filename (*.keytab)
zeppelin
Keytab File Owner:Group
zeppelin:zeppelin
File Permission (octal)
600

Apache ZooKeeper

Role: zookeeper-server
Kerberos Principals
zookeeper
Filename (*.keytab)
zookeeper
Keytab File Owner:Group
zookeeper:zookeeper
File Permission (octal)
600

Cloudera Management

Role: cloudera-mgmt-REPORTSMANAGER
Kerberos Principals
hdfs
Filename (*.keytab)
headlamp
Keytab File Owner:Group
cloudera-scm:cloudera-scm
File Permission (octal)
600
Role: cloudera-mgmt-SERVICEMONITOR
Kerberos Principals
hue
Filename (*.keytab)
cmon
Keytab File Owner:Group
cloudera-scm:cloudera-scm
File Permission (octal)
600
Role: cloudera-mgmt-ACTIVITYMONITOR
Kerberos Principals
hue
Filename (*.keytab)
cmon
Keytab File Owner:Group
cloudera-scm:cloudera-scm
File Permission (octal)
600

Cloudera Manager

Kerberos Principals
cloudera-scm, HTTP
Filename (*.keytab)
cmf, scm
Keytab File Owner:Group
cloudera-scm:cloudera-scm
File Permission (octal)
600

CruiseControl

Role: cruise_control-CRUISE_CONTROL_SERVER
Kerberos Principals
cruisecontrol, kafka, HTTP
Filename (*.keytab)
cruise_control
Keytab File Owner:Group
cruisecontrol:hadoop
File Permission (octal)
600

HttpFS

Role: hdfs-HTTPFS
Kerberos Principals
httpfs, HTTP
Filename (*.keytab)
httpfs
Keytab File Owner:Group
httpfs:httpfs
File Permission (octal)
600

Hue

Role: hue-KT_RENEWER
Kerberos Principals
hue
Filename (*.keytab)
hue
Keytab File Owner:Group
hue:hue
File Permission (octal)
600

Schema Registry

Role: schemaregistry-SCHEMA_REGISTRY_SERVER
Kerberos Principals
schemaregistry, HTTP
Filename (*.keytab)
schemaregistry
Keytab File Owner:Group
schemaregistry:hadoop
File Permission (octal)
600

Streams Messaging Manager

Role: streams_messaging_manager-STREAMS_MESSAGING_MANAGER_SERVER
Kerberos Principals
streamsmsgmgr, HTTP
Filename (*.keytab)
streams_messaging_manager
Keytab File Owner:Group
streamsmsgmgr:streamsmsgmgr
File Permission (octal)
600

Streams Replication Manager

Role: streams_replication_manager-STREAMS_REPLICATION_MANAGER_DRIVER
Kerberos Principals
streamsrepmgr
Filename (*.keytab)
streams_replication_manager
Keytab File Owner:Group
streamsrepmgr:streamsrepmgr
File Permission (octal)
600
Role: streams_replication_manager-STREAMS_REPLICATION_MANAGER_SERVICE
Kerberos Principals
streamsrepmgr
Filename (*.keytab)
streams_replication_manager
Keytab File Owner:Group
streamsrepmgr:streamsrepmgr
File Permission (octal)
600