Cloudera Docs » 2.3.4 » Upgrading HDP Manually
Upgrading HDP Manually
Also available as:
PDF
loading table of contents...

​Configure and Validate Apache Falcon

[Note]Note

In HDP-2.3.4, if authorization is enabled (for example, in the startup.properties file with *.falcon.security.authorization.enabled=true") then Access Control List (ACL) is mandated for all entities.

Upgrade Apache Falcon after you have upgraded HDFS, Hive, Oozie, and Pig. Stop Oozie jobs while running Falcon.

[Important]Important

We recommend upgrading Falcon from HDP 2.1 to HDP 2.2 and then upgrading from HDP 2.2 to HDP 2.3.

Upgrade Falcon from HDP 2.1 to HDP 2.2

  1. Upgrade Falcon from HDP 2.1 to HDP 2.2.

    • RHEL/CentOS/Oracle 6

      1. Download the HDP 2.2.9 hdp.repo file:

        wget -nv http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.2.9.0/hdp.repo -0 /etc/yum.repos.d/hdp.repo

      2. Install the HDP 2.2.9 version of Falcon:

        yum install "falcon"

    • SLES 11 SP 1

      1. Download the HDP 2.2.9 hdp.repo file:

        wget -nv http://public-repo-1.hortonworks.com/HDP/sles11sp1/2.x/updates/2.2.9.0/hdp.repo -O /etc/zypp/repos.d/hdp.repo

      2. Install the HDP 2.2.9 version of Falcon:

        zypper install "falcon"

    • SLES 11 SP3/SP4

      1. Download the HDP 2.2.9 hdp.repo file:

        http://public-repo-1.hortonworks.com/HDP/suse11sp3/2.x/updates/2.2.9.0/hdp.repo -O /etc/zypp/repos.d/hdp.repo

      2. Install the HDP 2.2.9 version of Falcon:

        zypper install "falcon"

    • Ubuntu 12

      1. Download the HDP 2.2.9 hdp.repo file:

        wget http://public-repo-1.hortonworks.com/HDP/ubuntu12/2.x/updates/2.2.9.0/hdp.list -O /etc/apt/sources.list.d/hdp.list

      2. Install the HDP 2.2.9 version of Falcon:

        apt-get install "falcon"

    • Debian 6

      1. Download the HDP 2.2.9 hdp.repo file:

        wget -nv http://public-repo-1.hortonworks.com/HDP/debian6/2.x/updates/2.2.9.0/hdp.list -O /etc/apt/sources.list.d/hdp.list

      2. Install the HDP 2.2.9 version of Falcon:

        apt-get install "falcon"

  2. Verify start.properties. In Falcon 2.2, TLS is enabled by default. When TLS is enabled, Falcon starts on https:/<falcon_host>:15443/. You can disable TLS by adding the following line in startup.properties:

    *.falcon.enableTLS=false

  3. Verify the client.properties file. If TLS is disabled, make sure the property falcon.url is set as follows:

    falcon.url=http://localhost:15000/

  4. Install the hotfix provided by Hortonworks customer support to migrate entities in bulk.

  5. Restart Falcon.

    su - falcon -c "cd /usr/hdp/2.2.9.0-$BUILD/falcon/bin/falcon-start"

  6. Run Falcon CLI to update Falcon entities from HDP 2.1 to HDP 2.2. This script also updates the ownership and permissions of staging and working directories of cluster entitles. Run this script as user falcon.

    falcon upgrade -owner hrt_qa -group users -configDir <falcon_config_store> -backupDir/tmp/

    Where:

    • Owner and group are used to set ACL for Falcon entities.

    • configDir refers to the location where Falcon stores its entities. This directory should be the same location as the value of property *.config.store.uri in the file conf/startup.properties.

    • backupDir is the location where Falcon 2.1 entitles are copied as backup. User can restore entities from this directory.

  7. Restart Falcon.

    su - falcon -c "cd /usr/hdp/2.2.9.0-$BUILD/falcon/bin/falcon-start"

  8. Replace your configuration after upgrading. Copy /etc/falcon/conf from the template to the conf directory in falcon hosts.

  9. Check your Falcon entities. There should be no changes, but in some cases you may need to update your entities post-upgrade.

Upgrade Falcon from HDP 2.2 to HDP 2.3

  1. RHEL/CentOS/Oracle 6

    1. On all Falcon hosts, clean the yum repository.

      yum clean all

    2. Remove the HDP 2.2 Falcon component. This command uninstalls the HDP 2.2 component. It leaves the user data, and metadata, but removes your configurations:

      yum erase "falcon*"

    3. Validate that all HDP 2.2 component binaries are uninstalled:

      yum list installed | grep @HDP2.2

    4. Remove the HDP 2.2 hdp.repo file:

      rm /etc/yum.repos.d/hdp.repo

    5. Install the HDP 2.3.4..0 repo:

      • Download the hdp.repo file:

        wget -nv http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.3.4.0/hdp.repo -O /etc/yum.repos.d/hdp.repo

    SLES 11 SP 1

    1. On all Falcon hosts, clean the yum repository.

      zypper clean -all

    2. Remove the Falcon HDP 2.2 component. This command uninstalls the HDP 2.2 components. It leaves the user data, and metadata, but removes your configurations:

      zypper rm "falcon*"

    3. Validate that all HDP 2.2 component binaries are uninstalled:

      yum list installed | grep @HDP2.2

    4. Remove the HDP 2.2 hdp.repo file:

      rm /etc/zypp/repos.d/hdp.repo

    5. Download the HDP 2.3.4.0 hdp.repo file:

      wget -nv http://public-repo-1.hortonworks.com/HDP/sles11sp1/2.x/updates/2.3.4.0/hdp.repo -O /etc/zypp/repos.d/hdp.repo

    6. Install the HDP 2.3.4.0 version of the Falcon component.

      zypper install "falcon"

    SLES 11 SP3/SP4

    1. On all Falcon hosts, clean the zypper repository.

      zypper clean -all

    2. Remove HDP 2.2 Falcon component.

      zypper rm "falcon*"

    3. Validate that all HDP 2.2 component binaries are uninstalled:

      zypper search --installed-only --repo HDP-2.2.6.0

    4. Remove the HDP 2.2 hdp.repo file:

      rm /etc/zypp/repos.d/hdp.repo

    5. Download the HDP 2.3.4.0 hdp.repo file:

      http://public-repo-1.hortonworks.com/HDP/suse11sp3/2.x/updates/2.3.4.0/hdp.repo -O /etc/zypp/repos.d/hdp.repo

    6. Install the HDP 2.3.4.0 version of the Falcon component. F

      zypper install "falcon"

    Ubuntu 12

    1. On all Falcon hosts, clean the apt-get repository.

      apt-get clean -&-all

    2. Remove the HDP 2.2 Falcon component. This command uninstalls the HDP 2.2 component. It leaves the user data, and metadata, but removes your configurations:

      apt-get remove "falcon*"

    3. Validate that all HDP 2.2 component binaries are uninstalled:

      yum list installed | grep @HDP2.2

    4. Remove the HDP 2.2 hdp.repo file:

      rm /etc/apt/sources.list.d/hdp.list

    5. Download the HDP 2.3.4.0 hdp.repo file:

      wget -nv http://public-repo-1.hortonworks.com/HDP/ubuntu12/2.x/updates/2.3.4.0/hdp.list - O /etc/apt/sources.list.d/hdp.list

    6. Run an update:

      apt-get update

    7. Install the HDP 2.3.4.0 versions of the Falcon component.

      apt-get install "falcon"

    Debian 6

    1. On all Falcon hosts, clean the apt-get repository.

      apt-get clean

    2. Remove the HDP 2.2 Falcon component. This command uninstalls the HDP 2.2 component. It leaves the user data, and metadata, but removes your configurations:

      apt-get remove "falcon*"

    3. Validate that all HDP 2.2 component binaries are uninstalled:

      dpkg -l | grep "^ii" | grep hadoop

    4. Remove the HDP 2.2 hdp.repo file:

      rm /etc/apt/sources.list.d/hdp.list

    5. Download the HDP 2.3.4.0 hdp.repo file:

      wget -nv http://public-repo-1.hortonworks.com/HDP/debian6/2.x/updates/2.3.4.0/hdp.list - O /etc/apt/sources.list.d/hdp.list

    6. Run an update:

      apt-get update

    7. Install the HDP 2.3.4.0 versions of the Falcon component.

      apt-get install "falcon"

  2. Verify start.properties. In Falcon 2.3.4, TLS is enabled by default. When TLS is enabled, Falcon starts on https:/<falcon_host>:15443/. You can disable TLS by adding the following line in startup.properties:

    *.falcon.enableTLS=false

  3. Verify the client.properties file. If TLS is disabled, make sure the property falcon.url is set as follows:

    falcon.url=http://localhost:15000/

  4. Install the hotfix provided at https://hortonworks.jire.com/browse/BUG-32579.

  5. Restart Falcon.

    su - falcon -c "cd /usr/hdp/2.3.4.0-$BUILD/falcon/bin/falcon-start"

  6. Run Falcon CLI to update Falcon entities from HDP 2.1 to HDP 2.2. This script also updates the ownership and permissions of staging and working directories of cluster entitles. Run this script as user falcon.

    falcon upgrade -owner hrt_qa -group users -configDir <falcon_config_store> -backupDir/tmp/

    Where:

    • Owner and group are used to set ACL for Falcon entities.

    • configDir refers to the location where Falcon stores its entities. This directory should be the ssame location as the value of property *.config.store.uri in the file conf/startup.properties.

    • backupDir is the location where Falcon 2.1 entitles are copied as backup. User can restore entities from this directory.

  7. Restart Falcon.

    su - falcon -c "cd /usr/hdp/2.3.4.0-$BUILD/falcon/bin/falcon-start"

  8. Replace your configuration after upgrading. Copy /etc/falcon/conf from the template to the conf directory in falcon hosts.

  9. Check your Falcon entities. There should be no changes, but in some cases you may need to update your entities post-upgrade.

© 2012–2021 by Cloudera, Inc.
Document licensed under the Creative Commons Attribution ShareAlike 4.0 License.
Cloudera.com | Documentation | Support | Community