Cloudera Docs

Cloudera Manager Agents

The Cloudera Manager Agent is a Cloudera Manager component that works with the Cloudera Manager Server to manage the processes that map to role instances.

In a Cloudera Manager managed cluster, you can only start or stop role instance processes using Cloudera Manager. Cloudera Manager uses a modified version of the open source process management tool, Supervisor. The Cloudera Manager Agent manages this through a process called cloudera-scm-supervisord, which starts processes, redirects log files, notifies of process failures, and sets the effective user ID for service processes. Cloudera Manager supports automatically restarting a crashed process. It will also flag a role instance with a bad health flag if its process crashes repeatedly right after start up.

The Agent is started by init.d at start-up. It, in turn, contacts the Cloudera Manager Server and determines which processes should be running. The Agent is monitored as part of Cloudera Manager's host monitoring. If the Agent stops heartbeating, the host is marked as having bad health.

One of the Agent's main responsibilities is to start and stop processes. When the Agent detects a new process from the Server heartbeat, the Agent creates a directory for it in /var/run/cloudera-scm-agent and unpacks the configuration. It then contacts cloudera-scm-supervisord, which starts the process.

Monitored File Systems

Cloudera Manager Agent monitors file systems to report usage and free space. The following is a list of the supported file system mount types:
  • ext2
  • ext3
  • ext4
  • xfs
  • aufs
  • tmpfs

By modifying the parameter monitored_nodev_filesystem_types in the Cloudera Manager Agent's config.ini file, you can also add NFS type mounts to the monitored list. For more information on this parameter, see Configuring Cloudera Manager Agents.

Other file system types, especially auto-mounted file systems, or other network file systems, are not supported and might prevent the operation of the Cloudera Manager Agent.

cm_processes

To enable Cloudera Manager to run scripts in subdirectories of /var/run/cloudera-scm-agent, (because /var/run is mounted noexec in many Linux distributions), Cloudera Manager mounts a tmpfs, named cm_processes, for process subdirectories.

A tmpfs defaults to a max size of 50% of physical RAM but this space is not allocated until its used, and tmpfs is paged out to swap if there is memory pressure.

The lifecycle actions of cmprocesses can be described by the following statements:

  • Created when the Agent starts up for the first time with a new cloudera-scm-supervisord process.
  • If it already exists without noexec, reused when the Agent is started using start and not recreated.
  • Remounted if Agent is started using clean_restart.
  • Unmounting and remounting cleans out the contents (since it is mounted as a tmpfs).
  • Unmounted when the host is rebooted.
  • Not unmounted when the Agent is stopped.