Hadoop Users (user:group) and Kerberos Principals
During the Cloudera Manager installation process, several Linux user accounts and groups are created by default. These are listed in the table below. For details of these local Linux user accounts, see the Overview page of this section. Integrating the cluster to use Kerberos for authentication requires creating Kerberos principals and keytabs for these user accounts.
Component (Version) | Unix User ID | Groups | Functionality |
---|---|---|---|
Apache Atlas | atlas | atlas, hadoop | Apache Atlas by default has atlas as user and group. It is configurable |
Apache Flink | flink | flink | The Flink Dashboard runs as this user. |
Apache HBase | hbase | hbase | The Master and the RegionServer processes run as this user. |
Apache HBase Indexer | hbase | hbase | The indexer servers are run as this user. |
Apache HDFS | hdfs | hdfs, hadoop | The NameNode and DataNodes run as this user, and the HDFS root directory as well as the directories used for edit logs should be owned by it. |
Apache Hive Hive on Tez |
hive | hive | The HiveServer2 process and the Hive Metastore processes run as this user.A user must be defined for Hive access to its Metastore DB (for example, MySQL or Postgres) but it can be any identifier and does not correspond to a Unix uid. This is javax.jdo.option.ConnectionUserName in hive-site.xml. |
Apache Impala | impala | impala, hive | Impala services run as this user. |
Apache Kafka | kafka | kafka | Kafka brokers, mirrorMaker, and Connect workers run as this user. |
Apache Knox | knox | knox | Apache Knox Gateway Server runs as this user |
Apache Kudu | kudu | kudu | Kudu services run as this user. |
Apache Livy | livy | livy | The Livy Server process runs as this user |
Apache NiFi | nifi | nifi | Runs as the nifi user |
Apache NiFi Registry | nifiregistry | nifiregistry | Runs as the nifiregistry user |
Apache Oozie | oozie | oozie | The Oozie service runs as this user. |
Apache Ozone | hdfs | hdfs, hadoop | Ozone Manager, Storage Container Manager (SCM), Recon and Ozone Datanodes run as this user. |
Apache Parquet | ~ | ~ | No special users. |
Apache Phoenix | phoenix | phoenix | The Phoenix Query Server runs as this user |
Apache Ranger | ranger | ranger, hadoop | Ranger Admin, Usersync and Tagsync services by default have ranger as user and ranger, hadoop as groups. It is configurable. |
Apache Ranger KMS | kms | kms | Ranger KMS runs with kms user and group. It is configurable. |
Apache Ranger Raz | rangerraz | ranger | Ranger Raz runs with rangerraz user and is part of the ranger group. |
Apache Ranger RMS | rangerrms | ranger | Ranger RMS runs with rangerrms user and is part of the ranger group. |
Apache Solr | solr | solr | The Solr processes run as this user. |
Apache Spark | spark | spark | The Spark History Server process runs as this user. |
Apache Sqoop | sqoop | sqoop | This user is only for the Sqoop1 Metastore, a configuration option that is not recommended. |
Apache YARN | yarn | yarn, hadoop | Without Kerberos, all YARN services and applications run as this user. The LinuxContainerExecutor binary is owned by this user for Kerberos. |
Apache Zeppelin | zeppelin | zeppelin | The Zeppelin Server process runs as this user |
Apache ZooKeeper | zookeeper | zookeeper | The ZooKeeper processes run as this user. It is not configurable. |
Cloudera Manager (all versions) | cloudera-scm | cloudera-scm | Clusters managed by Cloudera Manager run Cloudera Manager Server, monitoring roles, and other Cloudera Server processes as cloudera-scm. Requires keytab file named cmf.keytab because name is hard-coded in Cloudera Manager. |
Cruise Control | cruisecontrol | hadoop | The Cruise Control process runs as this user. |
HttpFS | httpfs | httpfs | The HttpFS service runs as this user. See HttpFS authenticationfor instructions on how to generate the merged httpfs-http.keytab file. |
Hue | hue | hue | Hue services run as this user. |
Hue Load Balancer | apache | apache | The Hue Load balancer has a dependency on the apache2 package that uses the apache user name. Cloudera Manager does not run processes using this user ID. |
Key Trustee Server | keytrustee | keytrustee | The Key Trustee Server service runs as this user. |
Schema Registry | schemaregistry | hadoop | The Schema Registry process runs as this user. |
Streams Messaging Manager | streamsmsgmgr | streamsmsgmgr | The Streams Messaging Manager processes runs as this user. |
Streams Replication Manager | streamsrepmgr | streamsrepmgr | The Streams Replication Manager processes runs as this user. |
Keytabs and Keytab File Permissions
user
accounts, such as hdfs
, are mapped to the
username
portion of the Kerberos principal names, as
follows:username/host.example.com@EXAMPLE.COM
For example, the
Kerberos principal for Apache Hive would be:
hive/host.example.com@EXAMPLE.COM
Keytabs that contain multiple principals are merged automatically from individual keytabs by Cloudera Manager. If you override a service configuration to not use the CM-provided keytab, then you must ensure that all the principals required for the given role instance on a specific host are merged together in the keytab file you deploy manually on that host.
For example, for Filename (*.keytab), the Atlas keytab filename would be
atlas.keytab
, HBase would be hbase.keytab
, and Cloudera
Manager would be cmf.keytab
and scm.keytab
.
Keytab File Owner:Group matters when Cloudera Manager starts a role. For example, Cloudera Manager starts the role "DataNode"". Cloudera Manager launches the DataNode process as a user (here, "hdfs"). Because that process needs to access the HDFS keytab, Cloudera Manager puts the HDFS keytab in the DataNode's process directory, and the keytab is given the owner:group that is listed in the table. Thus, the DataNode process properly owns the keytab file.
The tables below lists the usernames to use for Kerberos principal names, for clusters managed by Cloudera Manager.
Apache Atlas
- Role: atlas-ATLAS_SERVER
- Kerberos Principals
- atlas
- Filename (*.keytab)
- atlas
- Keytab File Owner:Group
- atlas:atlas
- File Permission (octal)
- 600
Apache Flink
- Role: flink
- Kerberos Principals
- flink
- Filename (*.keytab)
- flink
- Keytab File Owner:Group
- flink:flink
- File Permission (octal)
- 600
Apache HBase
- Role: hbase-HBASETHRIFTSERVER
- Kerberos Principals
- hbase, HTTP
- Filename (*.keytab)
- hbase, HTTP
- Keytab File Owner:Group
- hbase:hbase
- File Permission (octal)
- 600
- Role: hbase-REGIONSERVER
- Kerberos Principals
- hbase, HTTP
- Filename (*.keytab)
- hbase, HTTP
- Keytab File Owner:Group
- hbase:hbase
- File Permission (octal)
- 600
- Role: hbase-HBASERESTSERVER
- Kerberos Principals
- hbase, HTTP
- Filename (*.keytab)
- hbase, HTTP
- Keytab File Owner:Group
- hbase:hbase
- File Permission (octal)
- 600
- Role: hbase-MASTER
- Kerberos Principals
- hbase, HTTP
- Filename (*.keytab)
- hbase, HTTP
- Keytab File Owner:Group
- hbase:hbase
- File Permission (octal)
- 600
Apache HBase indexer
- Role: ks_indexer-HBASE_INDEXER
- Kerberos Principals
- hbase, HTTP
- Filename (*.keytab)
- hbase
- Keytab File Owner:Group
- hbase:hbase
- File Permission (octal)
- 600
Apache HDFS
- Role: hdfs-NAMENODE
- Kerberos Principals
- hdfs, HTTP
- Filename (*.keytab)
- hdfs
- Keytab File Owner:Group
- hdfs:hdfs
- File Permission (octal)
- 600
- Role: hdfs-DATANODE
- Kerberos Principals
- hdfs, HTTP
- Filename (*.keytab)
- hdfs
- Keytab File Owner:Group
- hdfs:hdfs
- File Permission (octal)
- 600
- Role: hdfs-SECONDARYNAMENODE
- Kerberos Principals
- hdfs, HTTP
- Filename (*.keytab)
- hdfs
- Keytab File Owner:Group
- hdfs:hdfs
- File Permission (octal)
- 600
Apache Hive, Hive on Tez
- Role: hive-HIVESERVER2
- Kerberos Principals
- hive
- Filename (*.keytab)
- hive
- Keytab File Owner:Group
- hive:hive
- File Permission (octal)
- 600
- Role: hive-HIVEMETASTORE
- Kerberos Principals
- hive
- Filename (*.keytab)
- hive
- Keytab File Owner:Group
- cloudera-scm:cloudera-scm
- File Permission (octal)
- 600
Apache Impala
- Role: impala-STATESTORE
- Kerberos Principals
- impala, HTTP
- Filename (*.keytab)
- impala
- Keytab File Owner:Group
- impala:impala
- File Permission (octal)
- 600
- Role: impala-CATALOGSERVER
- Kerberos Principals
- impala, HTTP
- Filename (*.keytab)
- impala
- Keytab File Owner:Group
- impala:impala
- File Permission (octal)
- 600
- Role: impala-IMPALAD
- Kerberos Principals
- impala, HTTP
- Filename (*.keytab)
- impala
- Keytab File Owner:Group
- impala:impala
- File Permission (octal)
- 600
Apache Kafka
- Role: kafka-KAFKA_BROKER
- Kerberos Principals
- kafka
- Filename (*.keytab)
- kafka
- Keytab File Owner:Group
- kafka:kafka
- File Permission (octal)
- 600
- Role: kafka-KAFKA_MIRROR_MAKER
- Kerberos Principals
- kafka_mirror_maker
- Filename (*.keytab)
- kafka
- Keytab File Owner:Group
- kafka:kafka
- File Permission (octal)
- 600
- Role: kafka-KAFKA_CONNECT
- Kerberos Principals
- kafka
- Filename (*.keytab)
- kafka
- Keytab File Owner:Group
- kafka:kafka
- File Permission (octal)
- 600
Apache Knox
- Role: knox-KNOX_GATEWAY
- Kerberos Principals
- knox, HTTP
- Filename (*.keytab)
- hbase
- Keytab File Owner:Group
- knox:knox
- File Permission (octal)
- 600
Apache Kudu
- Role: kudu-KUDU_MASTER
- Kerberos Principals
- kudu
- Filename (*.keytab)
- kudu
- Keytab File Owner:Group
- kudu:kudu
- File Permission (octal)
- 600
- Role: kudu-KUDU_TSERVER
- Kerberos Principals
- kudu
- Filename (*.keytab)
- kudu
- Keytab File Owner:Group
- kudu:kudu
- File Permission (octal)
- 600
Apache Livy
- Role: livy-LIVY_SERVER
- Kerberos Principals
- livy
- Filename (*.keytab)
- livy
- Keytab File Owner:Group
- livy:livy
- File Permission (octal)
- 600
Apache NiFi
- Role: nifi
- Kerberos Principals
- nifi, HTTP
- Filename (*.keytab)
- nifi
- Keytab File Owner:Group
- nifi:nifi
- File Permission (octal)
- 600
Apache NiFi Registry
- Role: nifiregistry
- Kerberos Principals
- nifiregistry, HTTP
- Filename (*.keytab)
- nifiregistry
- Keytab File Owner:Group
- nifiregistry:nifiregistry
- File Permission (octal)
- 600
Apache Oozie
- Role: oozie-OOZIE_SERVER
- Kerberos Principals
- oozie, HTTP
- Filename (*.keytab)
- oozie
- Keytab File Owner:Group
- oozie:oozie
- File Permission (octal)
- 600
Apache Ozone
- Role: ozone-OZONE_MANAGER
- Kerberos Principals
- om, HTTP
- Filename (*.keytab)
- ozone
- Keytab File Owner:Group
- hdfs:hdfs
- File Permission (octal)
- 600
- Role: ozone-STORAGE_CONTAINER_MANAGER
- Kerberos Principals
- scm, HTTP
- Filename (*.keytab)
- ozone
- Keytab File Owner:Group
- hdfs:hdfs
- File Permission (octal)
- 600
- Role: ozone-OZONE_DATANODE
- Kerberos Principals
- dn, HTTP
- Filename (*.keytab)
- ozone
- Keytab File Owner:Group
- hdfs:hdfs
- File Permission (octal)
- 600
- Role: ozone-OZONE_RECON
- Kerberos Principals
- recon, HTTP
- Filename (*.keytab)
- ozone
- Keytab File Owner:Group
- hdfs:hdfs
- File Permission (octal)
- 600
- Role: ozone-S3_GATEWAY
- Kerberos Principals
- HTTP
- Filename (*.keytab)
- ozone
- Keytab File Owner:Group
- hdfs:hdfs
- File Permission (octal)
- 600
Apache Phoenix
- Role: phoenix-PHOENIX_QUERY_SERVER
- Kerberos Principals
- phoenix, HTTP
- Filename (*.keytab)
- phoenix
- Keytab File Owner:Group
- phoenix:phoenix
- File Permission (octal)
- 600
Apache Ranger
- Role: ranger-RANGER_ADMIN
- Kerberos Principals
- rangeradmin, rangerlookup, HTTP
- Filename (*.keytab)
- ranger
- Keytab File Owner:Group
- ranger:ranger
- File Permission (octal)
- 600
- Role: ranger-RANGER_USERSYNC
- Kerberos Principals
- rangerusersync
- Filename (*.keytab)
- ranger
- Keytab File Owner:Group
- ranger:ranger
- File Permission (octal)
- 600
- Role: ranger-RANGER_TAGSYNC
- Kerberos Principals
- rangertagsync
- Filename (*.keytab)
- ranger
- Keytab File Owner:Group
- ranger:ranger
- File Permission (octal)
- 600
Apache Ranger KMS
- Role: ranger-RANGER_TAGSYNC
- Kerberos Principals
- rangerkms, HTTP
- Filename (*.keytab)
- ranger_kms
- Keytab File Owner:Group
- kms:kms
- File Permission (octal)
- 600
Apache Ranger Raz
- Role: ranger-RANGER_RAZ
- Kerberos Principals
- rangerraz, HTTP
- Filename (*.keytab)
- rangerraz
- Keytab File Owner:Group
- ranger:rangerraz
- File Permission (octal)
- 600
Apache Ranger RMS
- Role: ranger-RANGER_RMS
- Kerberos Principals
- rangerrms
- Filename (*.keytab)
- rangerrms
- Keytab File Owner:Group
- ranger:rangerrms
- File Permission (octal)
- 600
Apache Solr
- Role: solr-SOLR_SERVER
- Kerberos Principals
- solr, HTTP
- Filename (*.keytab)
- solr
- Keytab File Owner:Group
- solr:solr
- File Permission (octal)
- 600
Apache Spark
- Role: spark_on_yarn-SPARK_YARN_HISTORY_SERVER
- Kerberos Principals
- spark
- Filename (*.keytab)
- spark
- Keytab File Owner:Group
- spark:spark
- File Permission (octal)
- 600
Apache YARN
- Role: yarn-NODEMANAGER
- Kerberos Principals
- yarn, HTTP
- Filename (*.keytab)
- yarn
- Keytab File Owner:Group
- yarn:hadoop
- File Permission (octal)
- 644
- Role: yarn-RESOURCEMANAGER
- Kerberos Principals
- yarn, HTTP
- Filename (*.keytab)
- yarn
- Keytab File Owner:Group
- yarn:hadoop
- File Permission (octal)
- 600
- Role: yarn-JOBHISTORY
- Kerberos Principals
- mapred
- Filename (*.keytab)
- mapred
- Keytab File Owner:Group
- yarn:hadoop
- File Permission (octal)
- 600
Apache Zeppelin
- Role: zeppelin-ZEPPELIN_SERVER
- Kerberos Principals
- zeppelin, HTTP
- Filename (*.keytab)
- zeppelin
- Keytab File Owner:Group
- zeppelin:zeppelin
- File Permission (octal)
- 600
Apache ZooKeeper
- Role: zookeeper-server
- Kerberos Principals
- zookeeper
- Filename (*.keytab)
- zookeeper
- Keytab File Owner:Group
- zookeeper:zookeeper
- File Permission (octal)
- 600
Cloudera Management
- Role: cloudera-mgmt-REPORTSMANAGER
- Kerberos Principals
- hdfs
- Filename (*.keytab)
- headlamp
- Keytab File Owner:Group
- cloudera-scm:cloudera-scm
- File Permission (octal)
- 600
- Role: cloudera-mgmt-SERVICEMONITOR
- Kerberos Principals
- hue
- Filename (*.keytab)
- cmon
- Keytab File Owner:Group
- cloudera-scm:cloudera-scm
- File Permission (octal)
- 600
- Role: cloudera-mgmt-ACTIVITYMONITOR
- Kerberos Principals
- hue
- Filename (*.keytab)
- cmon
- Keytab File Owner:Group
- cloudera-scm:cloudera-scm
- File Permission (octal)
- 600
Cloudera Manager
- Kerberos Principals
- cloudera-scm, HTTP
- Filename (*.keytab)
- cmf, scm
- Keytab File Owner:Group
- cloudera-scm:cloudera-scm
- File Permission (octal)
- 600
CruiseControl
- Role: cruise_control-CRUISE_CONTROL_SERVER
- Kerberos Principals
- cruisecontrol, kafka, HTTP
- Filename (*.keytab)
- cruise_control
- Keytab File Owner:Group
- cruisecontrol:hadoop
- File Permission (octal)
- 600
HttpFS
- Role: hdfs-HTTPFS
- Kerberos Principals
- httpfs, HTTP
- Filename (*.keytab)
- httpfs
- Keytab File Owner:Group
- httpfs:httpfs
- File Permission (octal)
- 600
Hue
- Role: hue-KT_RENEWER
- Kerberos Principals
- hue
- Filename (*.keytab)
- hue
- Keytab File Owner:Group
- hue:hue
- File Permission (octal)
- 600
Schema Registry
- Role: schemaregistry-SCHEMA_REGISTRY_SERVER
- Kerberos Principals
- schemaregistry, HTTP
- Filename (*.keytab)
- schemaregistry
- Keytab File Owner:Group
- schemaregistry:hadoop
- File Permission (octal)
- 600
Streams Messaging Manager
- Role: streams_messaging_manager-STREAMS_MESSAGING_MANAGER_SERVER
- Kerberos Principals
- streamsmsgmgr, HTTP
- Filename (*.keytab)
- streams_messaging_manager
- Keytab File Owner:Group
- streamsmsgmgr:streamsmsgmgr
- File Permission (octal)
- 600
Streams Replication Manager
- Role: streams_replication_manager-STREAMS_REPLICATION_MANAGER_DRIVER
- Kerberos Principals
- streamsrepmgr
- Filename (*.keytab)
- streams_replication_manager
- Keytab File Owner:Group
- streamsrepmgr:streamsrepmgr
- File Permission (octal)
- 600
- Role: streams_replication_manager-STREAMS_REPLICATION_MANAGER_SERVICE
- Kerberos Principals
- streamsrepmgr
- Filename (*.keytab)
- streams_replication_manager
- Keytab File Owner:Group
- streamsrepmgr:streamsrepmgr
- File Permission (octal)
- 600