Creating a Pre-Deployed Worker Host
Complete the steps below to create a pre-deployed worker host.
- Instantiate a virtual machine image (an AMI, if you are using Amazon Web Services) based on a supported operating system and start the virtual machine. See the documentation for your virtualization environment for details.
- Download the parcels required for the worker host from the public parcel repository, or from a repository that you have created and save them to a temporary directory. See Cloudera Manager 7 Download Information.
- From the same location where you downloaded the parcels, download the
parcel_name.parcel.sha1
file for each parcel. - Calculate and compare the sha1 of the downloaded parcel to ensure that the parcel
was downloaded correctly. For example:
sha1sum KAFKA-2.0.2-1.2.0.2.p0.5-el6.parcel | awk '{print $1}' > KAFKA-2.0.2-1.2.0.2.p0.5-el6.parcel.sha diff KAFKA-2.0.2-1.2.0.2.p0.5-el6.parcel.sha1 KAFKA-2.0.2-1.2.0.2.p0.5-el6.parcel.sha
- Unpack the parcel:
- Create the following directories:
/opt/cloudera/parcels
/opt/cloudera/parcel-cache
- Set the ownership for the two directories you just created so that they are owned by the username that the Cloudera Manager agent runs as.
- Set the permissions for each directory using the following
command:
Note that the contents of these directories will be publicly available and can be safely marked as world-readable.chmod 755 directory
- Running as the same user that runs the Cloudera Manager agent, extract the
contents of the parcel from the temporary directory using the following command:
tar -zxvf parcelfile -C /opt/cloudera/parcels/
- Add a symbolic link from the product name of each parcel to the
/opt/cloudera/parcels
directory.For example, to link/opt/cloudera/parcels/CDH-6.0.0-1.cdh6.0.0.p0.309038
to/opt/cloudera/parcels/CDH
, use the following command:ln -s /opt/cloudera/parcels/CDH-6.0.0-1.cdh6.0.0.p0.309038 /opt/cloudera/parcels/CDH
- Mark the parcels to not be deleted by the Cloudera Manager agent on start up
by adding a
.dont_delete
marker file (this file has no contents) to each subdirectory in the/opt/cloudera/parcels
directory. For example:touch /opt/cloudera/parcels/CDH/.dont_delete
- Create the following directories:
- Verify the file
exists:
You should see output similar to the following:ls -l /opt/cloudera/parcels/parcelname
ls -al /opt/cloudera/parcels/CDH total 100 drwxr-xr-x 9 root root 4096 Sep 14 14:53 . drwxr-xr-x 9 root root 4096 Sep 14 06:34 .. drwxr-xr-x 2 root root 4096 Sep 12 06:39 bin -rw-r--r-- 1 root root 0 Sep 14 14:53 .dont_delete drwxr-xr-x 26 root root 4096 Sep 12 05:10 etc drwxr-xr-x 4 root root 4096 Sep 12 05:04 include drwxr-xr-x 2 root root 69632 Sep 12 06:44 jars drwxr-xr-x 37 root root 4096 Sep 12 06:39 lib drwxr-xr-x 2 root root 4096 Sep 12 06:39 meta drwxr-xr-x 5 root root 4096 Sep 12 06:39 share
- Install the Cloudera Manager agent. If you have not already done so, Step 1: Configure a Repository for Cloudera Manager.
- Create an image of the worker host. See the documentation for your virtualization environment for details.