Altus Director Scripts

Cloudera Altus Director does a lot of work automatically, but there are often unique customizations necessary for your environment. To that end, Altus Director offers the ability to run scripts, supplied by you, at several points in the lifecycles of instances, deployments, and clusters.

Here are some reasons to have Altus Director run scripts:

  • To register custom DNS hostnames for instances
  • To add instances to an Active Directory domain
  • To install or update software packages, beyond what Altus Director normally handles
  • To configure instance networking in specialized ways

Altus Director always runs scripts with root permissions, so they have unfettered ability to make necessary changes. They are not limited in length, and do not conflict with other mechanisms that cloud providers offer for running scripts, such as EC2 user data. A script may be written in any language that is executable from the command line. Often they are shell scripts, but Python or other high-level scripting languages work as well.

Types of Scripts

This table lists the types of scripts that can be passed to Altus Director. Each type of script has a specific time when it runs, particular places where it runs, and a specific place to specify it. Also, some scripts are not available through the Altus Director user interface.

Type When it Runs Where it Runs Where to Specify In UI
Bootstrap Immediately after an instance becomes reachable On each instance for an instance template Instance template Yes
Pre-terminate Just before an instance is terminated On each instance for an instance template Instance template No
Deployment post-creation After deployment creation On the instance running Cloudera Manager Deployment template No
Cluster instance-level post-creation After cluster creation, before cluster post-creation scripts On every cluster instance Cluster template No
Cluster post-creation After cluster creation, after cluster instance-level post-creation scripts On an arbitrary cluster instance Cluster template No
Cluster pre-terminate Before cluster termination On an arbitrary cluster instance Cluster template No

Notes on Script Execution

Altus Director needs to perform a small amount of work on an instance before running any scripts:

  • Versions of Altus Director prior to 6.1 use the operating system package manager to install the GNU screen utility. If installation of screen fails, the bootstrap process fails.
  • Versions of Altus Director starting with 6.1 no longer perform screen installation. However, they do disable the requiretty setting in /etc/sudoers, if it is present.

The first scripts that Altus Director runs are bootstrap scripts. Bootstrap scripts are run when an instance is first allocated and are used to perform work related to the lifecycle of the individual instance rather than the deployment or cluster lifecycle.

Deployment post-creation scripts are run after Cloudera Manager is running and configured, but before Altus Director updates itself with any changes Cloudera Manager may independently and automatically make to its own configuration. Similarly, cluster post-creation scripts run after Cloudera Manager is configured to manage the cluster, but before Altus Director updates itself with any changes Cloudera Manager may independently and automatically make to those configurations. In both cases, information that scripts draw from Altus Director might be out of date.

Altus Director selects an arbitrary cluster instance for running cluster post-creation and pre-terminate scripts. Therefore, these scripts should perform work that applies to the cluster lifecycle as a whole, and not individual instances in the cluster, since each script runs on just one instance. If, before all scripts are complete, a chosen arbitrary instance fails, for example due to an unexpected termination, Altus Director selects a new instance and resumes script execution.

Elements of a Script

At a minimum, the content of a script must be supplied either directly or by referring to a file path local to where Altus Director runs. If necessary, a script may simply download another script from a URL and run it.

The following additional information may be optionally provided along with a script:

  • script ID: a unique identifier for the script, useful for tracing its execution in Altus Director logs
  • environment variables: a table of key-value pairs for environment variables to set before running the script

When you specify a script through the user interface, you cannot include a script ID or environment variables. Use a client configuration file or call the Altus Director API directly in order to pass a script ID or environment variables. See the topic Client Configuration File Fields below to see how to supply the additional information in a client configuration file.

Pre-defined Environment Variables

Some scripts automatically have access to several environment variables defined by Altus Director alongside any others defined explicitly in script definitions. The variables are oriented to make it easy to perform API calls to Cloudera Manager.

Variable Example Description Available To
DEPLOYMENT_HOST_PORT 203.0.113.1:7180 The host and port used to connect to the Cloudera Manager deployment. Deployment and cluster scripts
ENVIRONMENT_NAME dir_cluster Environment The name of the environment. Deployment and cluster scripts
DEPLOYMENT_NAME dir_cluster Deployment The name of the Cloudera Manager deployment. Deployment and cluster scripts
CLUSTER_NAME dir_cluster The name of the cluster. The Cloudera Manager API needs this to specify which cluster on a Cloudera Manager server to operate on. Cluster scripts
CM_USERNAME admin The username needed to authenticate to Cloudera Manager. Deployment and cluster scripts
CM_PASSWORD admin The username needed to authenticate to Cloudera Manager. Deployment and cluster scripts

Exit Code

A script should return exit code 0 if its execution should be considered successful by Altus Director. When a script returns a nonzero exit code, Altus Director retries the script up to a configurable number of times, in case the failure was due to a transient problem. When all script retries fail, Altus Director fails the process it is performing, such as bootstrapping a cluster. If a script returns a reserved exit code (configurable, default 91), Altus Director does not retry the script and fails its current process immediately.

Client Configuration File Fields

Any type of script may be specified in a cluster configuration file to be used with the Altus Director client. A pair of fields in the appropriate template is available for providing each type of script. One field is for specifying scripts by directly including their content, and the other by referencing local file paths. See the table below to identify the specific field name for direct inclusion or path reference for each type of script. Both fields may be used for one type of script, in which case the scripts included directly are executed first. Each field value is a list of scripts, and so within a field, scripts are executed in the order they are given.

A script can be specified simply by a string containing its content (for direct inclusion) or its path (for reference by path). To also specify a script ID or environment variables, use an object with fields for those elements as well as the script content or path. The following example lists two bootstrap scripts by content, one as just the script itself, another as an object with a script ID and environment variables:

bootstrapScripts: ["""#!/bin/sh
echo 'Hello World!'
exit 0
""",
  {
    id: bootstrapScript2,
    env {
      KEY1: VALUE1
      KEY2: VALUE2
    },
    content: "echo The values are $KEY1 and $KEY2"
  }
]

See the reference configuration files for more examples.

Type Template Field for Direct Inclusion Field for Path
Bootstrap Instance bootstrapScripts bootstrapScriptsPaths
Pre-terminate Instance preTerminateScripts preTerminateScriptsPaths
Deployment post-creation Deployment postCreateScripts postCreateScriptsPaths
Cluster instance-level post-creation Cluster instancePostCreateScripts instancePostCreateScriptsPaths
Cluster post-creation Cluster postCreateScripts postCreateScriptsPaths
Cluster pre-terminate Cluster preTerminateScripts preTerminateScriptsPaths

Where Scripts Run on an Instance

By default, Altus Director uploads each script to the /tmp directory of the instance where it is to run. However, some security standards prohibit execution of anything residing in the /tmp directory. Use the configuration property lp.ssh.tmpPath in the Altus Director server's application.properties file to set a different directory for Altus Director to use for running scripts. See Setting Altus Director Properties for information on where the file is located and details on how to set properties.

Setting this configuration property changes the execution location for all scripts run by Altus Director, not just those you supply, and for scripts in all clusters, deployments, and instances managed by Altus Director. Ensure that the directory exists on all instances launched by Altus Director. Note that a bootstrap script cannot be used to create the directory, since the directory existence is a precondition for running any scripts. Instead, use a custom image with the directory already in place, or some other mechanism.