Configuring a Local Parcel Repository

You can create a parcel repository for Cloudera Manager either by hosting an internal Web repository or by manually copying the repository files to the Cloudera Manager Server host for distribution to Cloudera Manager Agent hosts.

Loading Filters ... 6.1 6.0.1 6.0.0 5.16 5.15 5.14 5.13 5.12 5.11 5.10 5.9 5.8 5.7 5.6 5.5 5.4 5.3 5.2 5.1 5.0 6.1 6.0.1 6.0.0 5.16 5.15 5.14 5.13 5.12 5.11 5.10 5.9 5.8 5.7 5.6 5.5 5.4 5.3 5.2 5.1 5.0

Using an Internally Hosted Remote Parcel Repository

The following sections describe how to use an internal Web server to host a parcel repository:

Setting Up a Web Server

To host an internal repository, you must install or use an existing Web server on an internal host that is reachable by the Cloudera Manager host, and then download the repository files to the Web server host. The examples on this page use Apache HTTP Server as the Web server. If you already have a Web server in your organization, you can skip to Downloading and Publishing the Parcel Repository.

  1. Install Apache HTTP Server:
    RHEL / CentOS
    sudo yum install httpd
    SLES
    sudo zypper install httpd
    Debian
    sudo apt-get install httpd
  2. Edit the Apache HTTP Server configuration file (/etc/httpd/conf/httpd.conf by default) to add or edit the following line in the <IfModule mime_module> section:
        AddType application/x-gzip .gz .tgz .parcel

    If the <IfModule mime_module> section does not exist, you can add it in its entirety as follows:

    <IfModule mime_module>
        #
        # TypesConfig points to the file containing the list of mappings from
        # filename extension to MIME-type.
        #
        TypesConfig /etc/mime.types
    
        #
        # AddType allows you to add to or override the MIME configuration
        # file specified in TypesConfig for specific file types.
        #
        #AddType application/x-gzip .tgz
        #
        # AddEncoding allows you to have certain browsers uncompress
        # information on the fly. Note: Not all browsers support this.
        #
        #AddEncoding x-compress .Z
        #AddEncoding x-gzip .gz .tgz
        #
        # If the AddEncoding directives above are commented-out, then you
        # probably should define those extensions to indicate media types:
        #
        AddType application/x-compress .Z
        AddType application/x-gzip .gz .tgz .parcel
    
        #
        # AddHandler allows you to map certain file extensions to "handlers":
        # actions unrelated to filetype. These can be either built into the server
        # or added with the Action directive (see below)
        #
        # To use CGI scripts outside of ScriptAliased directories:
        # (You will also need to add "ExecCGI" to the "Options" directive.)
        #
        #AddHandler cgi-script .cgi
    
        # For type maps (negotiated resources):
        #AddHandler type-map var
    
        #
        # Filters allow you to process content before it is sent to the client.
        #
        # To parse .shtml files for server-side includes (SSI):
        # (You will also need to add "Includes" to the "Options" directive.)
        #
        AddType text/html .shtml
        AddOutputFilter INCLUDES .shtml
    </IfModule>
  3. Start Apache HTTP Server:
    RHEL 7
    sudo systemctl start httpd
    RHEL 6 or lower
    sudo service httpd start
    SLES 12, Ubuntu 16.04, Debian 8
    sudo systemctl start apache2
    SLES 11, Ubuntu 14.04, Debian 7 or lower
    sudo service apache2 start

Downloading and Publishing the Parcel Repository

  1. Download manifest.json and the parcel files for the product you want to install:
    CDH 6
    Apache Impala, Apache Kudu, Apache Spark 2, and Cloudera Search are included in the CDH parcel. To download the files for the latest CDH 6.1 release, run the following commands on the Web server host:
    sudo mkdir -p /var/www/html/cloudera-repos
    sudo wget --recursive --no-parent --no-host-directories https://username:password@archive.cloudera.com/p/cdh6/6.1.1/parcels/ -P /var/www/html/cloudera-repos
    sudo wget --recursive --no-parent --no-host-directories https://archive.cloudera.com/gplextras6/6.1.1/parcels/ -P /var/www/html/cloudera-repos
    sudo chmod -R ugo+rX /var/www/html/cloudera-repos/cdh6
    sudo chmod -R ugo+rX /var/www/html/cloudera-repos/gplextras6

    If you want to create a repository for a different CDH 6 release, replace 6.1.1 with the CDH 6 version that you want. For more information, see CDH 6 Download Information.

    CDH 5
    Impala, Kudu, Spark 1, and Search are included in the CDH parcel. To download the files for a CDH release (CDH 5.14.4 in this example), run the following commands on the Web server host:
    sudo mkdir -p /var/www/html/cloudera-repos
    sudo wget --recursive --no-parent --no-host-directories https://username:password@archive.cloudera.com/p/cdh5/parcels/5.14.4/ -P /var/www/html/cloudera-repos
    sudo wget --recursive --no-parent --no-host-directories https://username:password@archive.cloudera.com/p/gplextras5/parcels/5.14.4/ -P /var/www/html/cloudera-repos
    sudo chmod -R ugo+rX /var/www/html/cloudera-repos/cdh5
    sudo chmod -R ugo+rX /var/www/html/cloudera-repos/gplextras5

    If you want to create a repository for a different CDH release, replace 5.14.4 with the CDH version that you want. For more information, see CDH Download Information.

    Apache Accumulo for CDH
    To download the files for an Accumulo release for CDH (Accumulo 1.9.2 in this example), run the following commands on the web server host:
    sudo mkdir -p /var/www/html/cloudera-repos
    sudo wget --recursive --no-parent --no-host-directories https://archive.cloudera.com/accumulo6/6.1.0/parcels/-P /var/www/html/cloudera-repos
    sudo chmod -R ugo+rX /var/www/html/cloudera-repos/accumulo6

    If you want to create a repository for Accumulo 1.6.0 instead, replace 1.7.2 with 1.6.0.

    CDS Powered By Apache Spark 2 for CDH
    To download the files for a CDS release for CDH (CDS 2.3.0.cloudera3 in this example), run the following commands on the Web server host:
    sudo mkdir -p /var/www/html/cloudera-repos
    sudo wget --recursive --no-parent --no-host-directories https://username:password@archive.cloudera.com/p/spark2/parcels/2.3.0.cloudera3/ -P /var/www/html/cloudera-repos
    sudo chmod -R ugo+rX /var/www/html/cloudera-repos/spark2

    If you want to create a repository for a different CDS release, replace 2.3.0.cloudera3 with the CDS version that you want. For more information, see CDS Powered By Apache Spark Version Information.

    Cloudera Navigator Key Trustee Server
    Go to the Key Trustee Server download page. Select Parcels from the CHOOSE DOWNLOAD TYPE drop-down menu, and click DOWNLOAD NOW. This downloads the Key Trustee Server parcels and manifest.json files in a .tar.gz file. Copy the file to your Web server, and extract the files with the tar xvfz filename.tar.gz command. This example uses Key Trustee Server 5.14.0:
    sudo mkdir -p /var/www/html/cloudera-repos/keytrustee-server
    sudo tar xvfz /path/to/keytrustee-server-5.14.0-parcels.tar.gz -C /var/www/html/cloudera-repos/keytrustee-server --strip-components=1
    sudo chmod -R ugo+rX /var/www/html/cloudera-repos/keytrustee-server
    Cloudera Navigator Key Trustee KMS and HSM KMS
    Go to the Key Trustee KMS download page. Select Parcels from the CHOOSE DOWNLOAD TYPE drop-down menu, and click DOWNLOAD NOW. This downloads the Key Trustee KMS parcels and manifest.json files in a .tar.gz file. Copy the file to your Web server, and extract the files with the tar xvfz filename.tar.gz command. This example uses Key Trustee KMS 5.14.0:
    sudo mkdir -p /var/www/html/cloudera-repos/keytrustee-kms
    sudo tar xvfz /path/to/keytrustee-kms-5.14.0-parcels.tar.gz -C /var/www/html/cloudera-repos/keytrustee-kms --strip-components=1
    sudo chmod -R ugo+rX /var/www/html/cloudera-repos/keytrustee-kms
    Sqoop Connectors
    To download the parcels for a Sqoop Connector release, run the following commands on the Web server host. This example uses the latest available Sqoop Connectors:
    sudo mkdir -p /var/www/html/cloudera-repos
    sudo wget --recursive --no-parent --no-host-directories https://username:password@archive.cloudera.com/p/sqoop-connectors/parcels/latest/ -P /var/www/html/cloudera-repos
    sudo chmod -R ugo+rX /var/www/html/cloudera-repos/sqoop-connectors

    If you want to create a repository for a different Sqoop Connector release, replace latest with the Sqoop Connector version that you want. You can see a list of versions in the parcels parent directory.

  2. Visit the Repository URL http://<Web_server>/cloudera-repos/ in your browser and verify the files you downloaded are present. If you do not see anything, your Web server may have been configured to not show indexes.

Configuring Cloudera Manager to Use an Internal Remote Parcel Repository

  1. Use one of the following methods to open the parcel settings page:
    • Navigation bar
      1. Click the parcel icon in the top navigation bar or click Hosts and click the Parcels tab.
      2. Click the Configuration button.
    • Menu
      1. Select Administration > Settings.
      2. Select Category > Parcels.
  2. In the Remote Parcel Repository URLs list, click the addition symbol to open an additional row.
  3. Enter the path to the parcel. For example: http://<web_server>/cloudera-parcels/cdh6/6.1.1/
  4. Enter a Reason for change, and then click Save Changes to commit the changes.

Using a Local Parcel Repository

To use a local parcel repository, complete the following steps:
  1. Open the Cloudera Manager Admin Console and navigate to the Parcels page.
  2. Select Configuration and verify that you have a Local Parcel Repository path set. By default, the directory is /opt/cloudera/parcel-repo.
  3. Remove any Remote Parcel Repository URLs you are not using, including ones that point to Cloudera archives.
  4. Add the parcel you want to use to the local parcel repository directory that you specified. For instructions on downloading parcels, see Downloading and Publishing the Parcel Repository above.
  5. In the command line, navigate to the local parcel repository directory.
  6. Create a SHA1 hash for the parcel you added and save it to a file named parcel_name.parcel.sha.
    For example, the following command generates a SHA1 hash for the parcel CDH-6.1.0-1.cdh6.1.0.p0.770702-el7.parcel:
    sha1sum CDH-6.1.0-1.cdh6.1.0.p0.770702-el7.parcel | awk '{ print $1 }' > CDH-6.1.0-1.cdh6.1.0.p0.770702-el7.parcel.sha
  7. Change the ownership of the parcel and hash files to cloudera-scm:
    sudo chown -R cloudera-scm:cloudera-scm /opt/cloudera/parcel-repo/*
  8. In the Cloudera Manager Admin Console, navigate to the Parcels page.
  9. Click Check for New Parcels and verify that the new parcel appears.
  10. Download, distribute, and activate the parcel.