Single User Mode Requirements

In a conventional Cloudera Manager deployment, the Cloudera Manager Agent, which manages Hadoop processes on each host, runs as the root user. However, some environments restrict access to the root account.

Cloudera Manager 5.3 provides single user mode, which satisfies the requirements of such environments. In single user mode, the Cloudera Manager Agent and all the processes run by services managed by Cloudera Manager are started as a single configured user and group. Single user mode prioritizes isolation between Hadoop and the rest of the system over isolation between Hadoop processes running on the system.

Within a Cloudera Manager deployment, single user mode is global and applies to all clusters managed by that instance of Cloudera Manager.

By default, the single user is cloudera-scm and the configuration steps described in the following sections assume that user. However, other users are supported. If you choose another user, replace cloudera-scm in the following steps with the selected user, and perform the additional steps in Using a Non-default Single User.

The following sections describe limitations of single user mode and the required configuration steps for the supported installation scenarios at specific points during the installation process.

Limitations

  • Switching between conventional and single user mode is not supported.
  • Single user mode is supported for clusters running CDH 5.2 and higher.
  • NFS Gateway is not supported in single user mode.

Using a Non-default Single User

When configuring single user mode for a user other than the default (cloudera-scm), perform the following configuration steps:

  • Make the following directories writable by the single user:
    • /var/log/cloudera-scm-agent/
    • /var/lib/cloudera-scm-agent/
  • Cloudera Manager stores parcels under /opt/cloudera, which by default is owned by cloudera-scm. Do one of the following:
    • Change /opt/cloudera to be writable by the single user.
    • Change the parcel directory location to be writable by the single user:
      1. Go to Administration > Settings > Parcels.
      2. Set the Local Parcel Repository Path property.
      3. Click Save Changes.
  • For a single user username, create the process limits configuration file at /etc/security/limits.d/username.conf with the following settings: 
    username soft nofile 32768
username soft nproc 65536
username hard nofile 1048576
username hard nproc unlimited
username hard memlock unlimited
username soft memlock unlimited

Configuration Steps Before Starting Cloudera Manager Agents in Installation Paths B and C

  • If you manually install Agent packages, before starting the Agents, configure them to run as cloudera-scm by editing the file /etc/default/cloudera-scm-agent and uncommenting the line: 
    USER="cloudera-scm"
  • Configure the parcels directory. Do one of the following:
    • On each host, in the Agent configuration file /etc/cloudera-scm-agent/config.ini, set the parcel_dir property: 
      # Parcel directory.  Unpacked parcels will be stored in this directory.
# Downloaded parcels will be stored in <parcel_dir>/../parcel-cache
# parcel_dir=/opt/cloudera/parcels
      1. Click Hosts in the top navigation bar.
      2. Click the Configuration tab.
      3. Configure the value of the Parcel Directory property. The setting of the parcel_dir property in the Cloudera Manager Agent configuration file overrides this setting.
      4. Click Save Changes to commit the changes.
      5. On each host, restart the Cloudera Manager Agent: 
        $ sudo service cloudera-scm-agent restart

Configuration Steps Before Running the Installation Wizard

Before configuring a cluster to run in single user mode, the following steps must be performed on all hosts in the cluster:
  • Give the single user passwordless sudo access. You must create the user if it doesn’t exist. One common way of achieving this is to add the user to the configured sudoers group by running the command: 
    usermod -a -G sudo cloudera-scm
    or adding a new sudo configuration for the cloudera-scm group by running the command visudo and then adding the following line: 
    %cloudera-scm ALL=(ALL) NOPASSWD: ALL
  • Sudo must be configured so that /usr/sbin is in the path when running sudo. One way to achieve this is by adding the following configuration to sudoers:
    1. Edit the /etc/sudoers file using the visudo command
    2. Add this line to the configuration file: 
      Defaults secure_path = /sbin:/bin:/usr/sbin:/usr/bin
  • Set up per user limits for su prior to setting up the Agent.
    1. Edit/etc/pam.d/su.
    2. Uncomment: 
      session required pam_limits.so
  • Roles that run on Tomcat require some directories to exist in non-configurable paths. The following directories must be created and be writable by cloudera-scm:
    • HDFS (HttpFS role) - /var/lib/hadoop-httpfs
    • Oozie Server - /var/lib/oozie
    • Sqoop 2 Server - /var/lib/sqoop2
    • Solr Server - /var/lib/solr
  • Cloudera recommends that you create a prefix directory (for example, /cm) owned by cloudera-scm under which all other service directories will be placed. In single user mode, the Cloudera Manager Agent creates directories under the prefix directory with the correct ownership. If hosts have additional volumes on them that will be used for data directories Cloudera recommends creating a directory on each volume (for example, /data0/cm and /data1/cm) that is writable by cloudera-scm.

Configuration Steps Before Starting the Installation Wizard in Installation Paths B and C

Perform the following steps for the indicated scenarios:
  • Path C - Do one of the following:
    • Create and change the ownership of /var/lib/cloudera-scm-server to the single user.
    • Set the Cloudera Manager Server local storage directory to one owned by the single user:
      1. Go to Administration > Settings > Advanced.
      2. Set the Cloudera Manager Server Local Data Storage Directory property to a directory owned by the single user.
      3. Click Save Changes to commit the changes.
  • Path B and C when using already managed hosts - Configure single user mode:
    1. Go to Administration > Settings > Advanced.
    2. Check the Single User Mode checkbox.
    3. Click Save Changes to commit the changes.

Configuration Steps While Running the Installation Wizard

When configuring the first cluster in Cloudera Manager using the Installation wizard you’ll have the option to set up the cluster in single user mode. This configures the Agents to run as cloudera-scm.

During the review configuration step you confirm that all the configured paths are writable by cloudera-scm. The directories themselves don’t have to exist as long as the parent directory is writable by cloudera-scm.

Following the standard review configuration page, an additional paths configuration page shows all the configurable paths for the services that will be created in the cluster. These must also be modified to be locations writable by cloudera-scm. In most cases, the paths that need to be modified from their default locations fall under two categories:
  • Paths under /var - These are log, run, and data directories for the different services.
  • Per volume data directories - These are data directory configurations that list a directory per volume. Such configurations are used by HDFS, MapReduce, YARN and Impala.

Configuration for Secure Clusters

You must perform some additional configuration when setting up secure HDFS in single user mode: