Cloudera Docs

Step 12: Complete Post-Upgrade steps for upgrades to CDP Private Cloud Base

Steps to perform after upgrading a cluster.

Loading Filters ... 7.2.4 7.1.4 7.1.3 7.1.2 7.1.1 7.0.3 6.3.3 6.3.1 6.3.0 6.2.1 6.2.0 6.1.1 6.1.0 6.0.1 6.0.0 5.16 5.15 5.14 5.13 6.3.3 6.3.2 6.2.1 6.2.0 6.1.1 6.1.0 6.0.1 6.0.0 5.16 5.15 5.14 5.13 7.1.5 7.1.4 7.1.3 7.1.2 7.1.1 7.0.3 7.2.4 7.1.4 7.1.3 7.1.2 7.1.1 7.0.3 6.3.3 6.3.1 6.3.0 6.2.1 6.2.0 6.1.1 6.1.0 6.0.1 6.0.0 5.16 5.15 5.14 5.13

Several components require additional steps after you complete the upgrade to CDP Private Cloud Base:
  • HBase See Apache HBase post-upgrade tasks
  • Apache Hive See Hive Post-Upgrade Tasks.
  • Kafka
    1. Remove the following properties from the Kafka Broker Advanced Configuration Snippet (Safety Valve) configuration property.
      • Inter.broker.protocol.version
      • log.message.format.version
    2. Save your changes.
    3. Perform a rolling restart:
      1. Select the Kafka service.
      2. Click Actions > Rolling Restart.
      3. In the pop-up dialog box, select the options you want and click Rolling Restart.
      4. Click Close once the command has finished.
  • Kudu See Upgrade Notes for Apache Kudu 1.12 / CDP 7.1
  • YARN
    • Scheduler: If you are using Fair Scheduler, you must migrate to Capacity Scheduler during the upgrade process, and once the upgrade is finished you need to manually fine tune it. For more information, see Manual configuration of scheduler properties.
    • Considering logical processors in the calculation: The yarn.nodemanager.resource.count-logical-processors-as-cores property was not present in CDH 5. In Cloudera Runtime 7.1.1, it is set to false by default, meaning that YARN does not consider logical processors in the calculation which can results in a 2x performance hit if Linux Container Executor and CGroups are enabled. To solve this issue, set yarn.nodemanager.resource.count-logical-processors-as-cores=true and restart the NodeManager.
    • NodeManager recovery: By default the yarn.nodemanager.recovery.enabled property is set to true after upgrading from CDH to CDP. If in your source CDH cluster you disable the NodeManager recovery feature, and you want to keep that setting, you manually have to disable it again in the upgraded cluster. Note, that Cloudera recommends to have this feature enabled. If you are upgrading from a CDP source cluster to a CDP target cluster, the value your set for the yarn.nodemanager.recovery.enabled property in your source cluster will be kept in your upgraded cluster.
    • Log aggregation: In order to see the history of applications that were launched before upgrade, do the following:
      1. In Cloudera Manager, navigate to YARN > Configuration > Category: Log aggregation.
      2. Set the following configurations:
        yarn.log-aggregation.TFile.remote-app-log-dir-suffix=logs

        yarn.log-aggregation.IFile.remote-app-log-dir-suffix=logs-ifile
    • Maximum capacity: Set the yarn.scheduler.capacity.<queuepath>.user-limit-factor to a value that is greater than 1. This configuration will help to grow the queue usage beyond its configured capacity till its maximum capacity configured.
    • Ranger Plugins
      The following Ranger plugins are not enabled by default after the upgrade. If these services are configured in the cluster, you will need to manually enable the plugins in order for them to use Ranger:
      The following Ranger plugins are enabled after an upgrade:
      • Atlas
      • HDFS
      • Hive
      • Hive on Tez
      • Impala
      • Kafka
  • ZooKeeper

    Ensure, that QuorumSSL (Secure ZooKeeper) is enabled only if QuorumSASL (Server to server SASL authentication) is also enabled. Note, that QuorumSSL is enabled by default if AutoTLS is enabled. If QuorumSSL is enabled without QuorumSASL, then the ZooKeeper cluster can be slow to start due to some known ZooKeeper limitations.

  • Solr – See Cloudera Search post-upgrade tasks.
  • Sentry – See Sentry to Ranger Permissions.
  • Impala – See Apache Impala changes in CDP