Fixed Issues in Cloudera Manager 7.11.3 Cumulative hotfix 7 (CDP Private Cloud Base 7.1.9 SP1)

OPSAPS-69018: Cloudera Manager fails to support multiple SAML role values

When multiple values for the SAML role assignment attribute are returned in an assertion, Cloudera Manager only reads the first attribute value returned in an assertion list.

Since the attribute typically reflects a user’s LDAP groups, multiple values are common and can include any number of values which may or may not be mapped to roles in Cloudera Manager, in any order. This can cause authorization failures, or unexpected limited access rights in Cloudera Manager. This issue is fixed now.

OPSAPS-69709: Set Sqoop Atlas hook to send notifications synchronously

Sqoop has an Atlas hook which by default runs asynchronously to send notifications to the Atlas server. In certain cases, the Java Virtual Machine (JVM) in which Sqoop is running can shut down before the Kafka notification of the Atlas hook is sent. This can result in lost notifications.

This issue is fixed by ensuring that the notifications are synchronous.

OPSAPS-68387: Cloudera Manager UI incorrectly showing Skipped status for Hive ACID replication policy jobs when the job status was unknown

The Status column on the Cloudera Manager > Replication > Replication Policies page was incorrectly showing Skipped for Hive ACID replication policy jobs when the job status was unknown. The column now shows the Waiting for Update status for the Hive ACID replication policy jobs until the job status is confirmed.

OPSAPS-68494: Replication metric getter handles scenarios when "hive.resultset.use.unique.column.names" = "false"

Replication Metric getter failed when the hive.resultset.use.unique.column.names parameter was set to false because the resulting columns were non-unique. The Replication Metric getter now configures the hive.resultset.use.unique.column.names parameter to true during its JDBC session to override the service configuration.

OPSAPS-69978: Cruise Control capacity.py script fails on Python 3.8

Cruise Control no longer fails to start on Python 3.x when the capacity information is queried during the startup process. The script querying the capacity information is now fully compatible with Python 3.x.

OPSAPS-70269: Mismatch in ssl_enabled configuration between Cloudera Manager and Knox

The ssl.enabled property is populated in gateway-site.xml. Knox startup check script works when ssl is not enabled. This issue is fixed now.

OPSAPS-70257: Cloudera Manager upgrade fails with an error

While using CDP 7.1.6, if you try to upgrade Cloudera Manager with a version prior to 7.11.3 CHF7, the Cloudera Manager upgrade failed with the following error message:

Start RANGER_KMS-1 FAILED with Failed to start service

This issue is fixed now.

OPSAPS-70188: Conflicts field missing in ParcelInfo

Fixed an issue in parcels where conflicts field in manifest.json would mark a parcel as invalid

OPSAPS-70051: Configuration issue with the hive.server2.tez.initialize.default.sessions parameter

Cloudera Manager incorrectly sets hive.server2.tez.initialize.default.sessions to true, conflicting with its expected false value in Hive configurations.

Adjusted Cloudera Manager to align with Hive configuration, ensuring the parameter defaults correctly to false for consistency and to prevent overriding settings.

OPSAPS-70248: Optimize Impala Graceful Shutdown Initiation Time

This issue is resolved by streamlining the shutdown initiation process, reducing delays on large clusters.

OPSAPS-68906: Impala Rolling Restart Sequence for ZDU

This adjustment refines the Impala rolling restart sequence with catalog and statestore HA support, reducing Impala downtime during cluster upgrades.

OPSAPS-67641: The Next Run column for Hive ACID replication policies shows the correct message.

The Next Run column on the Cloudera Manager > Replication > Replication Policies page showed None Scheduled for recurring Hive ACID replication policy jobs, which is incorrect. The column now displays the correct message.

OPSAPS-68246: Added a parameter for ozoneReplicationResult response for Ozone replication policies

The resultMessage parameter in the ozoneReplicationResult response for Ozone replication policies in Cloudera Manager REST API shows whether the replication command completed successfully or has failed with a specific message.

OPSAPS-70157: Long-term credential-based GCS replication policies continue to work when cluster-wide IDBroker client configurations are deployed

Replication policies that use long-term GCS credentials work as expected even when cluster-wide IDBroker client configurations are configured.

OPSAPS-70422: Change the “Run as username(on source)” field during Hive external table replication policy creation

You can use a different user other than hdfs for Hive external table replication policy run to replicate from an on-premises cluster to the cloud bucket if the USE_PROXY_USER_FOR_CLOUD_TRANSFER=true key-value pair is set for the source Cloudera Manager > Clusters > Hive service > Configuration > Hive Replication Environment Advanced Configuration Snippet (Safety Valve) property. This is applicable for all external accounts other than IDBroker external account.

OPSAPS-70460: Allow white space characters in Ozone snapshot-diff parsing

Ozone incremental replication no longer fails if a changed path contains one or more space characters.

OPSAPS-70607: Peer name validation step during Iceberg replication policy creation process is updated

During the Iceberg replication policy creation process if the source cluster name is renamed, the replication policy creation process does not fail.

OPSAPS-70492: ZDU | Handling of JDK add-opens flag in YARN with Cloudera Manager

The `JDK_JAVA_OPTIONS` environment variable is now used to supply the JDK 17 related flags.

OPSAPS-70594: Ozone HttpFS gateway role is not added to Rolling Restart

This issue is now resolved by adding the Ozone HttpFS gateway role to the Rolling Restart.

OPSAPS-69859: Correct configuration propagation in Cloudera Manager for non-HA clusters

In non-HA clusters, Cloudera Manager previously failed to propagate a few configurations for Ozone Manager (OM). This resulted in errors when attempting to submit a DistCp job to YARN, causing the submission process to fail. This issue has been fixed to ensure all required configurations are propagated correctly in non-HA clusters, allowing DistCp job submissions to proceed without errors.

OPSAPS-69987: Set the decommissioning state during decommission of Ozone Manager and Storage Container Manager

This issue is resolved by setting the state of master roles (OM and SCM) in Ozone to decommissioned after successful decommissioning.

OPSAPS-68752: Snapshot-diff delta is incorrectly renamed/deleted twice during on-premises to cloud replication

The snapshots created during replication are deleted twice instead of once, which results in incorrect snapshot information. This issue is fixed. For more information, see Cloudera Customer Advisory 2023-715: Replication Manager may delete its snapshot information when migrating from on-prem to cloud.

OPSAPS-63193: Need to enable Atlas canary check by default

Atlas canary check was disabled because Data Hub creation fails as Data Lake Atlas service health degrades.

OPSAPS-70226: Atlas uses the Solr configuration directory available in ATLAS_PROCESS/conf/solr instead of the Cloudera Manager provided directory

Atlas uses the configuration in /var/run/cloudera-scm-agent/process/151-atlas-ATLAS_SERVER/solrconf.xml.

OPSAPS-70355: Change compression from 'gz' to 'SNAPPY' in Atlas HBase tables

Changed the compression algorithm from GZ to SNAPPY in Atlas HBase tables to reduce the compaction time.

OPSAPS-68112: Atlas diagnostic bundle should contain server log, configurations, and, if possible, heap memories

The diagnostic bundle contains server log, configurations, and heap memories in a GZ file inside the diagnostic .zip package.

OPSAPS-69921: ATLAS_OPTS environment variable is set for FIPS with JDK 11 environments to run the import script in Atlas

_JAVA_OPTIONS are populated with additional parameters as seen in the following:

java_opts = 'export _JAVA_OPTIONS="-Dcom.safelogic.cryptocomply.fips.approved_only=true ' \
'--add-modules=com.safelogic.cryptocomply.fips.core,' \
'bctls --add-exports=java.base/sun.security.provider=com.safelogic.cryptocomply.fips.core ' \
'--add-exports=java.base/sun.security.provider=bctls --module-path=/cdep/extra_jars ' \
'-Dcom.safelogic.cryptocomply.fips.approved_only=true -Djdk.tls.ephemeralDHKeySize=2048 ' \
'-Dorg.bouncycastle.jsse.client.assumeOriginalHostName=true -Djdk.tls.trustNameService=true" '

OPSAPS-70299: Added optional Run As User option for hbase initial snapshot export on the source cluster

Added runAsUser query parameter to the clusters/{cluster-name}/services/{service-name}/snapshots/hbase/remote endpoint. When creating an on-premises to cloud HBase replication policies with Perform Initial Snapshot option, this appears as the Export snapshot user field in the Create HBase replication policy wizard in Cloudera Replication Manager.

When a user is specified in the runAsUser parameter, the YARN application that exports the HBase snapshot gets submitted by the hbase user impersonating the specified runAsUser.

To ensure the YARN application succeeds, the hbase user must be allowed to impersonate the runAsUser in HDFS by configuring the required properties and values in the HDFS core_site_safety_valve.

For example, if you want to allow the impersonation from any host and if the runAsUser is in the repl user group, you can set the following key-value pairs in Cloudera Manager > Clusters > [***CORE SETTINGS***] > Configuration > Cluster-wide Advanced Configuration Snippet (Safety Valve) for core-site.xml:

hadoop.proxyuser.hbase.groups = repl
hadoop.proxyuser.hbase.hosts = *