Known Issues in Iceberg REST Catalog

Learn about the known issues in Iceberg REST Catalog, the impact or changes to the functionality, and the workaround.

Known Issues identified in Cloudera Runtime 7.3.1.400 SP2

CDPQE-40109: REST Catalog needs to be reconfigured, after migration DL from LD to EDL.

After migrating the Data Lake (DL) from Light Duty to Enterprise Data Lake (EDL), all previously configured REST Catalog settings are missing. These include configurations such as the Hive Metastore (HMS) REST catalog enablement flag and Knox settings.

Manually reconfigure the REST Catalog settings after Data Lake migration. This includes the following:

Reenable the REST catalog flag in HMS.
Repeat the necessary Knox configurations.

CDPQE-40110: REST Catalog needs to be reconfigured, after performing backup & restore to new DL

When performing a Data Lake backup and restore to new DL, any REST Catalog configurations made through Cloudera Manager, including enabling the Hive Metastore REST Catalog and Knox settings, are not retained in the restored Data Lake.

Manually reconfigure the REST Catalog settings after Data Lake backup or restore. This includes the following:

Reenable the REST catalog flag in HMS.
Repeat the necessary Knox configurations.

DOCS-25855: REST Catalog needs to be reconfigured after performing upgrade recovery

When a patch upgrade fails for a cluster with REST Catalog, perform an SDX Upgrade recover operation on the Data Lake as the REST Catalog settings are not retained.

Manually reconfigure the REST Catalog settings after performing a Data Lake upgrade recovery. This includes the following:

Reenable the REST catalog flag in HMS.
Repeat the necessary Knox configurations.

DOCS-26132: Data Lake Zero Downtime Upgrade (ZDU) is not supported as High Availability (HA) support not available for REST Catalog

Data Lake Zero Downtime Upgrade (ZDU) is not supported for REST Catalog. At the time of a rolling upgrade, when the HMS role is stopped, Rest APIs calls are failing with the HTTP 500 error.

None

DOCS-25891: Ranger Audits not recorded for roles or client ids

When running a Curl call or selecting a query from the Spark client via CLIENT-ID and CLIENT_SECRET, there is no audit log recorded in Apache Ranger.

Apache Ranger audits for the REST Catalog meta operations will not show up unless HMS and REST Catalog is restarted after configuring Apache Knox and the IDBroker and reenabling REST Catalog for Cloudera Data Sharing.

DOCS-25863: Rollback query on Cloudera does not pick up correct metadata file and snapshot on Snowflake

After performing a rollback for table in Apache Hive, the pointer shows the correct file in Cloudera, but Snowflake shows the last snapshot instead of the snapshot targeted by the rollback.

In Snowflake, remove the table from the current schema with DROP and recreate it.

DOCS-25892: Materialized View and Metadata Read Fails from External Client (EMR)

After you create a materialized view for an Apache Hive table and try to read it using the AWS Elastic Map Reduce (EMR) external client, the read operation fails. This is because the Iceberg REST 1.3.1 specification supported by Cloudera are not supporting external clients yet.

DOCS-25895: Compatibility and Access Solutions for Cloudera Iceberg REST Catalog with Snowflake

Cloudera complies with the current Apache Iceberg REST Catalog API specification, providing credentials in the response to the LoadTable API call. Snowflake currently does not use the credentials in the LoadTable API response. Instead, Snowflake retrieves the credentials via a separate API call.

This issue has been raised with Snowflake, and they acknowledge it and are actively working on a resolution.

Until the update is available, consider using the external volume approach suggested by Snowflake when accessing the Iceberg REST catalog from Snowflake.

CDPD-85253: Rest Catalog service should use only HMS RangerHiveAuthorizer for its command authorization

Due to the static nature of the hive ranger plugin, having a new instance of HivePlugin for REST Catalog causes overwriting of the hivePlugin reference and using it for authorization. This is caused due to the REST Catalog being embedded in the HMS and not having its separate service. You have to use hiveMetasore as an application filter in Ranger > Audits to see the audit events.

CDPD-82812: The High Availability feature is not working for Rest Catalog

The Knox topology file cdp-share-access.xml created during Cloudera Data Sharing setup cannot handle multiple Hive Metastore nodes. In case of a node failure, the healthy nodes cannot reliably take over the workload.