Altus Director Scripts
Cloudera Altus Director does a lot of work automatically, but there are often unique customizations necessary for your environment. To that end, Altus Director offers the ability to run scripts, supplied by you, at several points in the lifecycles of instances, deployments, and clusters.
Here are some reasons to have Altus Director run scripts:
- To register custom DNS hostnames for instances
- To add instances to an Active Directory domain
- To install or update software packages, beyond what Altus Director normally handles
- To configure instance networking in specialized ways
Altus Director always runs scripts with root permissions, so they have unfettered ability to make necessary changes. They are not limited in length, and do not conflict with other mechanisms that cloud providers offer for running scripts, such as EC2 user data. A script may be written in any language that is executable from the command line. Often they are shell scripts, but Python or other high-level scripting languages work as well.
Types of Scripts
This table lists the types of scripts that can be passed to Altus Director. Each type of script has a specific time when it runs, particular places where it runs, and a specific place to specify it. Also, some scripts are not available through the Altus Director user interface.
Type | When it Runs | Where it Runs | Where to Specify | In UI |
---|---|---|---|---|
Bootstrap | Immediately after an instance becomes reachable | On each instance for an instance template | Instance template | Yes |
Pre-terminate | Just before an instance is terminated | On each instance for an instance template | Instance template | No |
Deployment post-creation | After deployment creation | On the instance running Cloudera Manager | Deployment template | No |
Cluster instance-level post-creation | After cluster creation, before cluster post-creation scripts | On every cluster instance | Cluster template | No |
Cluster post-creation | After cluster creation, after cluster instance-level post-creation scripts | On an arbitrary cluster instance | Cluster template | No |
Cluster pre-terminate | Before cluster termination | On an arbitrary cluster instance | Cluster template | No |
Notes on Script Execution
Altus Director needs to perform a small amount of work on an instance before running any scripts:
- Versions of Altus Director prior to 6.1 use the operating system package manager to install the GNU screen utility. If installation of screen fails, the bootstrap process fails.
- Versions of Altus Director starting with 6.1 no longer perform screen installation. However, they do disable the requiretty setting in /etc/sudoers, if it is present.
The first scripts that Altus Director runs are bootstrap scripts. Bootstrap scripts are run when an instance is first allocated and are used to perform work related to the lifecycle of the individual instance rather than the deployment or cluster lifecycle.
Deployment post-creation scripts are run after Cloudera Manager is running and configured, but before Altus Director updates itself with any changes Cloudera Manager may independently and automatically make to its own configuration. Similarly, cluster post-creation scripts run after Cloudera Manager is configured to manage the cluster, but before Altus Director updates itself with any changes Cloudera Manager may independently and automatically make to those configurations. In both cases, information that scripts draw from Altus Director might be out of date.
Altus Director selects an arbitrary cluster instance for running cluster post-creation and pre-terminate scripts. Therefore, these scripts should perform work that applies to the cluster lifecycle as a whole, and not individual instances in the cluster, since each script runs on just one instance. If, before all scripts are complete, a chosen arbitrary instance fails, for example due to an unexpected termination, Altus Director selects a new instance and resumes script execution.
Elements of a Script
At a minimum, the content of a script must be supplied either directly or by referring to a file path local to where Altus Director runs. If necessary, a script may simply download another script from a URL and run it.
The following additional information may be optionally provided along with a script:
- script ID: a unique identifier for the script, useful for tracing its execution in Altus Director logs
- environment variables: a table of key-value pairs for environment variables to set before running the script
When you specify a script through the user interface, you cannot include a script ID or environment variables. Use a client configuration file or call the Altus Director API directly in order to pass a script ID or environment variables. See the topic Client Configuration File Fields below to see how to supply the additional information in a client configuration file.
Pre-defined Environment Variables
Some scripts automatically have access to several environment variables defined by Altus Director alongside any others defined explicitly in script definitions. The variables are oriented to make it easy to perform API calls to Cloudera Manager.
Variable | Example | Description | Available To |
---|---|---|---|
DEPLOYMENT_HOST_PORT | 203.0.113.1:7180 | The host and port used to connect to the Cloudera Manager deployment. | Deployment and cluster scripts |
ENVIRONMENT_NAME | dir_cluster Environment | The name of the environment. | Deployment and cluster scripts |
DEPLOYMENT_NAME | dir_cluster Deployment | The name of the Cloudera Manager deployment. | Deployment and cluster scripts |
CLUSTER_NAME | dir_cluster | The name of the cluster. The Cloudera Manager API needs this to specify which cluster on a Cloudera Manager server to operate on. | Cluster scripts |
CM_USERNAME | admin | The username needed to authenticate to Cloudera Manager. | Deployment and cluster scripts |
CM_PASSWORD | admin | The username needed to authenticate to Cloudera Manager. | Deployment and cluster scripts |
Exit Code
A script should return exit code 0 if its execution should be considered successful by Altus Director. When a script returns a nonzero exit code, Altus Director retries the script up to a configurable number of times, in case the failure was due to a transient problem. When all script retries fail, Altus Director fails the process it is performing, such as bootstrapping a cluster. If a script returns a reserved exit code (configurable, default 91), Altus Director does not retry the script and fails its current process immediately.
Client Configuration File Fields
Any type of script may be specified in a cluster configuration file to be used with the Altus Director client. A pair of fields in the appropriate template is available for providing each type of script. One field is for specifying scripts by directly including their content, and the other by referencing local file paths. See the table below to identify the specific field name for direct inclusion or path reference for each type of script. Both fields may be used for one type of script, in which case the scripts included directly are executed first. Each field value is a list of scripts, and so within a field, scripts are executed in the order they are given.
A script can be specified simply by a string containing its content (for direct inclusion) or its path (for reference by path). To also specify a script ID or environment variables, use an object with fields for those elements as well as the script content or path. The following example lists two bootstrap scripts by content, one as just the script itself, another as an object with a script ID and environment variables:
bootstrapScripts: ["""#!/bin/sh echo 'Hello World!' exit 0 """, { id: bootstrapScript2, env { KEY1: VALUE1 KEY2: VALUE2 }, content: "echo The values are $KEY1 and $KEY2" } ]
See the reference configuration files for more examples.
Type | Template | Field for Direct Inclusion | Field for Path |
---|---|---|---|
Bootstrap | Instance | bootstrapScripts | bootstrapScriptsPaths |
Pre-terminate | Instance | preTerminateScripts | preTerminateScriptsPaths |
Deployment post-creation | Deployment | postCreateScripts | postCreateScriptsPaths |
Cluster instance-level post-creation | Cluster | instancePostCreateScripts | instancePostCreateScriptsPaths |
Cluster post-creation | Cluster | postCreateScripts | postCreateScriptsPaths |
Cluster pre-terminate | Cluster | preTerminateScripts | preTerminateScriptsPaths |
Where Scripts Run on an Instance
By default, Altus Director uploads each script to the /tmp directory of the instance where it is to run. However, some security standards prohibit execution of anything residing in the /tmp directory. Use the configuration property lp.ssh.tmpPath in the Altus Director server's application.properties file to set a different directory for Altus Director to use for running scripts. See Setting Altus Director Properties for information on where the file is located and details on how to set properties.
Setting this configuration property changes the execution location for all scripts run by Altus Director, not just those you supply, and for scripts in all clusters, deployments, and instances managed by Altus Director. Ensure that the directory exists on all instances launched by Altus Director. Note that a bootstrap script cannot be used to create the directory, since the directory existence is a precondition for running any scripts. Instead, use a custom image with the directory already in place, or some other mechanism.