Creating a Pre-Deployed Worker Host

Complete the steps below to create a pre-deployed worker host.

  1. Instantiate a virtual machine image (an AMI, if you are using Amazon Web Services) based on a supported operating system and start the virtual machine. See the documentation for your virtualization environment for details.
  2. Download the parcels required for the worker host from the public parcel repository, or from a repository that you have created and save them to a temporary directory. See Cloudera Manager 7 Download Information.
  3. From the same location where you downloaded the parcels, download the parcel_name.parcel.sha1 file for each parcel.
  4. Calculate and compare the sha1 of the downloaded parcel to ensure that the parcel was downloaded correctly. For example:
    sha1sum KAFKA-2.0.2-1.2.0.2.p0.5-el6.parcel | awk '{print $1}' > KAFKA-2.0.2-1.2.0.2.p0.5-el6.parcel.sha
    
    diff KAFKA-2.0.2-1.2.0.2.p0.5-el6.parcel.sha1 KAFKA-2.0.2-1.2.0.2.p0.5-el6.parcel.sha
    
  5. Unpack the parcel:
    1. Create the following directories:
      • /opt/cloudera/parcels
      • /opt/cloudera/parcel-cache
    2. Set the ownership for the two directories you just created so that they are owned by the username that the Cloudera Manager agent runs as.
    3. Set the permissions for each directory using the following command:
      chmod 755 directory
      Note that the contents of these directories will be publicly available and can be safely marked as world-readable.
    4. Running as the same user that runs the Cloudera Manager agent, extract the contents of the parcel from the temporary directory using the following command:
      tar -zxvf parcelfile -C /opt/cloudera/parcels/
    5. Add a symbolic link from the product name of each parcel to the /opt/cloudera/parcels directory.
      For example, to link /opt/cloudera/parcels/CDH-6.0.0-1.cdh6.0.0.p0.309038 to /opt/cloudera/parcels/CDH, use the following command:
      ln -s /opt/cloudera/parcels/CDH-6.0.0-1.cdh6.0.0.p0.309038 /opt/cloudera/parcels/CDH
    6. Mark the parcels to not be deleted by the Cloudera Manager agent on start up by adding a .dont_delete marker file (this file has no contents) to each subdirectory in the /opt/cloudera/parcels directory. For example:
      touch /opt/cloudera/parcels/CDH/.dont_delete
  6. Verify the file exists:
    ls -l /opt/cloudera/parcels/parcelname
    You should see output similar to the following:
    ls -al /opt/cloudera/parcels/CDH
    total 100
    drwxr-xr-x  9 root root  4096 Sep 14 14:53 .
    drwxr-xr-x  9 root root  4096 Sep 14 06:34 ..
    drwxr-xr-x  2 root root  4096 Sep 12 06:39 bin
    -rw-r--r--  1 root root     0 Sep 14 14:53 .dont_delete
    drwxr-xr-x 26 root root  4096 Sep 12 05:10 etc
    drwxr-xr-x  4 root root  4096 Sep 12 05:04 include
    drwxr-xr-x  2 root root 69632 Sep 12 06:44 jars
    drwxr-xr-x 37 root root  4096 Sep 12 06:39 lib
    drwxr-xr-x  2 root root  4096 Sep 12 06:39 meta
    drwxr-xr-x  5 root root  4096 Sep 12 06:39 share
    
  7. Install the Cloudera Manager agent. If you have not already done so, Step 1: Configure a Repository for Cloudera Manager.
  8. Create an image of the worker host. See the documentation for your virtualization environment for details.