Cloudera Docs

Fixed Issues in Iceberg REST Catalog

Review the list of Iceberg REST Catalog issues that are resolved in Cloudera Runtime 7.3.1, its service packs and cumulative hotfixes.

Cloudera Runtime 7.3.1.1100 SP3 CHF6

CDPD-93707: Fix Rest catalog Access Token Query
7.3.1.700, 7.3.1.1100
7.3.1.400, 7.3.1.500, 7.3.1.600
Previously, the Iceberg REST Catalog constructed an Access Token URL with an incorrect parameter name for the read-only policy and included the storage scheme (such as s3a://) in the table path, which caused the token query to fail against IDBroker. This failure prevented proper downscoping of credentials, causing the tokens returned by the catalog to retain broad read/write access to all buckets, including the Data Lake, based on the administrative role. The Iceberg REST Catalog Access Token URL query has been updated to correctly strip the storage scheme from the table path and use the proper policy parameter, ensuring that the credentials returned are properly downscoped to read-only access for the specific table.
CDPD-91471: Missing metering events for Iceberg REST Catalog endpoints
7.3.1.700, 7.3.1.1100
7.3.1.400, 7.3.1.500, 7.3.1.600
Metering events for the Iceberg REST Catalog could be lost because the API call count was maintained in an in-memory counter that lacked a graceful shutdown mechanism and mistakenly incremented counts even for failed requests. During a Knox Gateway restart, unpublished counts were permanently lost, resulting in missing metering events. This issue has been resolved. The metering logic has been updated to ensure API call counts are accurately recorded for successful requests and are not lost during node restarts.
CDPD-94295: IDBroker no longer includes preceding "/" in prefix when substituting session policy contents
7.3.1.507, 7.3.1.700, 7.3.1.1100
7.3.1.500, 7.3.1.600
IDBroker correctly handles leading path separator characters in prefix values when performing substitutions in condition statements of AWS session policies used in data sharing use cases. This means that Data Shares are properly constrained to the intended contents.
CDPD-93707: Fix Rest catalog Access Token Query
7.3.1.700, 7.3.1.1100
7.3.1.400, 7.3.1.500, 7.3.1.600
Previously, the Iceberg REST Catalog constructed an Access Token URL with an incorrect parameter name for the read-only policy and included the storage scheme (such as s3a://) in the table path, which caused the token query to fail against IDBroker. This failure prevented proper downscoping of credentials, causing the tokens returned by the catalog to retain broad read/write access to all buckets, including the Data Lake, based on the administrative role. The Iceberg REST Catalog Access Token URL query has been updated to correctly strip the storage scheme from the table path and use the proper policy parameter, ensuring that the credentials returned are properly downscoped to read-only access for the specific table.

Cloudera Runtime 7.3.1.1000 SP3 CHF5

There are no new fixed issues in this release.

Cloudera Runtime 7.3.1.900 SP3 CHF4

There are no new fixed issues in this release.

Cloudera Runtime 7.3.1.706 SP3 CHF 2

CDPD-91471: Missing metering events for Iceberg REST Catalog endpoints
7.3.1.700
7.3.1.400, 7.3.1.500, 7.3.1.600
Metering events for the Iceberg REST Catalog could be lost because the API call count was maintained in an in-memory counter that lacked a graceful shutdown mechanism and mistakenly incremented counts even for failed requests. During a Knox Gateway restart, unpublished counts were permanently lost, resulting in missing metering events. This issue has been resolved. The metering logic has been updated to ensure API call counts are accurately recorded for successful requests and are not lost during node restarts.
CDPD-94295: IDBroker no longer includes preceding "/" in prefix when substituting session policy contents
7.3.1.507, 7.3.1.700
7.3.1.500, 7.3.1.600
IDBroker correctly handles leading path separator characters in prefix values when performing substitutions in condition statements of AWS session policies used in data sharing use cases. This means that Data Shares are properly constrained to the intended contents.
CDPD-93707: Fix Rest catalog Access Token Query
7.3.1.700
7.3.1.400, 7.3.1.500, 7.3.1.600
Previously, the Iceberg REST Catalog constructed an Access Token URL with an incorrect parameter name for the read-only policy and included the storage scheme (such as s3a://) in the table path, which caused the token query to fail against IDBroker. This failure prevented proper downscoping of credentials, causing the tokens returned by the catalog to retain broad read/write access to all buckets, including the Data Lake, based on the administrative role. The Iceberg REST Catalog Access Token URL query has been updated to correctly strip the storage scheme from the table path and use the proper policy parameter, ensuring that the credentials returned are properly downscoped to read-only access for the specific table.

Cloudera Runtime 7.3.1.600 SP3 CHF1

CDPD-84118: Investigate HMS service health during longevity runs
7.3.1.600
7.3.1.400, 7.3.1.500
The update introduces the use of a UserGroupInformation (UGI) cache to optimize the handling of proxy users in the Hive Metastore. The update improves performance and resource management in the Hive Metastore by reducing redundant UGI creation and ensuring proper cleanup of resources to avoid API response timeouts from memory leaks.
CDPD-85416: Enhancing HiveAuthorizer Authorization Context for Accurate Ranger Auditing in REST Catalog
7.3.1.600
7.3.1.400, 7.3.1.500
HiveAuthorizer needs the authorization context enriched to have the client_type for the auditing purposes. Currently when calls are made into Ranger HiveAuthorizer from REST Catalog, Ranger is not able to differentiate the plugins between HMS / REST catalog and hence the audit is not correctly done for the calls made from REST Catalog service. With this change, requests from the Rest catalog will appear in the Ranger audit list with the client type set to restCatalog. Without it, all such requests would show up as HiveMetaStore.
CDPD-92586: Memory leak in HMS REST Catalog
Optimizing the REST Catalog instances in Hive Metastore prevents a potential memory leak. Instead of creating new instances, existing REST catalog instances are reused. This solves CDPD-91447.

Cloudera Runtime 7.3.1.500 SP3

CDPD-85253: Rest Catalog service should only use HMS RangerHiveAuthorizer for its command authorization
7.3.1.500
7.3.1.400
The update changes the appType for the REST_CATALOG case to use HIVE_METASTORE_APP_ID instead of REST_CATALOG_APP_ID. This aligns the authorization logic for Rest Catalog with Hive Metastore. In Ranger, now you are able to filter for audit events with application ID restCatalog.

Cloudera Runtime 7.3.1.400 SP2

There are no fixed issues in this release.