Known issues

Learn about the known issues in Cloudera Operational Database (COD), the impact or changes to the functionality, and the workaround.

ODX-1603 (COD 1.22) : COD throws a "Node Failure" issue

Problem: COD reports "Node Failure" state whenever OMID service goes down and retains the state until the OMID service is restarted from Cloudera Manager (CM).

  • If you are using OMID, restart the OMID service through CM and wait for a few minutes until COD starts displaying Available again.
  • If you are not using OMID service and your COD status is not affected by the OMID service status, you can suppress the OMID alerts directly in the CM.

Technical Service Bulletins

TSB 2022-568: HBase normalizer must be disabled for Salted Phoenix tables
When Apache Phoenix (“Phoenix”) creates a salted table, it pre-splits the table according to the number of salt regions. These regions must always be kept separate, otherwise Phoenix does not work correctly.

The HBase normalizer is not aware of this requirement, and in some cases the pre-split regions are merged automatically. This causes failure in Phoenix.

The same requirement applies when merging regions of salted tables manually: regions containing different salt keys (the first byte of the rowkey) must never be merged.

Note that either automatic or manual splitting of the regions for a salted table does not cause a problem. The problem only occurs when adjacent regions containing different salt keys are merged.

Upstream JIRA
Customers using the salted table feature of Phoenix.
Action required
  • Checking for affected tables:
    1. Run the following query from phoenix-sqlline to find the salted tables:
      select TABLE_SCHEM,TABLE_NAME from system.catalog where salt_buckets>0;
    2. From hbase shell run describe 'SCHEMA.TABLE' for each salted table, substituting the schema and table (for tables without schema, use just the table name) and look for {NORMALIZATION_ENABLED => 'true'} in the output.
  • If normalizer has been enabled on a salted Phoenix table, the HBase normalizer must be disabled manually. From HBase shell, run the following command for each salted Phoenix table:
    (If the table is in the default schema, just use 'TABLE_NAME')
Knowledge article
For the latest update on this issue, see the corresponding Knowledge article: TSB 2022-568: Hbase normalizer must be disabled for Salted Phoenix tables