Migrating Apache HBase Before Upgrading to CDH 6

If you are upgrading to CDH 6 from CDH 5 and have the HBase service installed, there are several pre-upgrades steps you might need to take:

Remove PREFIX_TREE Data Block Encoding

The PREFIX_TREE data block encoding code is removed in CDH 6. This feature was previously unsupported, but starting with CDH 6, HBase clusters with PREFIX_TREE enabled will fail. Therefore, before upgrading to CDH 6, you must ensure that all data has been migrated to a supported encoding type.

You can ensure that none of you tables or snapshots use the PREFIX_TREE Data Block Encoding by running the following commands:
  • hbase pre-upgrade validate-dbe
  • hbase pre-upgrade validate-hfile
You can run this command using a given CDH 6 parcel before upgrading the cluster itself, if you download and distribute the CDH 6 parcel:
  1. Download and distribute parcels for target version of CDH 6 as described in Download and Distribute Parcel Step 1-3.

    If the downloaded parcel version is higher than the current Cloudera Manager version the following error message displayed:

    Error for parcel CDH-6.X.parcel : Parcel version 6.X is not supported by this version of Cloudera Manager. Upgrade Cloudera Manager to at least 6.X before using this version of the parcel.

  2. Ignore the error message.
  3. Run the pre-upgrade command using the CDH 6 parcel preferably on a HMaster host.
    For example: 
    $ /opt/cloudera/parcels/CDH-6.1.1-1.cdh6.1.1.p0.875250/bin/hbase pre-upgrade validate-dbe
$ /opt/cloudera/parcels/CDH-6.1.1-1.cdh6.1.1.p0.875250/bin/hbase pre-upgrade validate-hfile

If you see a “PREFIX_TREE not supported” error when running the commands, you must resolve this error by changing your Data Block Encoding type to Prefix, Diff, or Fast Diff in your source cluster.

For more information about the supported Data Block Encoding types, see Compression and Data Block Encoding in HBase.

Upgrade Co-Processor Classes

External co-processors are not automatically upgraded. There are two ways to handle co-processor upgrade:
  • Upgrade your co-processor jars manually before continuing the upgrade.
  • Temporarily unset the co-processors and continue the upgrade.

    Once they are manually upgraded, they can be reset.

Attempting to upgrade without upgrading the co-processor jars can result in unpredictable behaviour such as HBase role start failure, HBase role crashing, or even data corruption.

You can ensure that your co-processors are compatible with the upgrade by running the hbase pre-upgrade validate-cp command. You can run this command using a given CDH 6 parcel before upgrading the cluster itself, if you download and distribute the CDH 6 parcel:
  1. Download and distribute parcels for target version of CDH 6 as described in Download and Distribute Parcel Step 1-3.

    If the downloaded parcel version is higher than the current Cloudera Manager version the following error message displayed:

    Error for parcel CDH-6.X.parcel : Parcel version 6.X is not supported by this version of Cloudera Manager. Upgrade Cloudera Manager to at least 6.X before using this version of the parcel.

  2. Ignore the error message.
  3. Run the pre-upgrade command using the CDH 6 parcel preferably on a HMaster host.
    For example, you can check for co-processor compatibility on master: 
    $ /opt/cloudera/parcels/CDH-6.1.1-1.cdh6.1.1.p0.875250/bin/hbase pre-upgrade validate-cp -jar /opt/cloudera/parcels/CDH-6.1.1-1.cdh6.1.1.p0.875250/jars/ -config
    Or, you can validate every table level co-processors where the table name matches to the .* regular expression: 
    $ /opt/cloudera/parcels/CDH-6.1.1-1.cdh6.1.1.p0.875250/bin/hbase pre-upgrade validate-cp -table .*

Check the HBase Related Upgrade Checkboxes

When you are attempting to upgrade from a CDH 5 cluster to a CDH 6 cluster, checkboxes appear to ensure you have performed all the HBase related pre-upgrade migration steps.

The upgrade continues only if you check both of the following statements: