Cloudera Docs

Known Issues in Apache Ozone

Learn about the known issues in Ozone, the impact or changes to the functionality, and the workaround.

In the CDP Private Cloud Base 7.1.7 SP2, a manual process is available to renew the expiring internal Ozone certificates. However, this is failing for Ozone Managers.
You can manually renew the Ozone certificates selectively for Ozone Managers using the Ozone Internal SSL certificate expiration KB article or the Ozone Internal SSL certificate expiration for versions 7.1.8 CHF2 and lower documentation.
HDDS-5341: Container report processing is single threaded.
This is a known issue in 7.1.7 SP2. Workaround is not available for SP2. This has been fixed by HOTFIX-5211 (CHF10). This will be fixed in SP2 CHF1
None.
HDDS-6277 and HDDS-6278: Improve memory foot print for listStatus.
This is a known issue in 7.1.7 SP2. Workaround is not available for SP2. This has been fixed by HOTFIX-5211 (CHF10). This will be fixed in SP2 CHF1
None.
HDDS-6357: RenameKey request has memory leak.
This is a known issue in 7.1.7 SP2. Workaround is not available for SP2. This has been fixed by HOTFIX-5211 (CHF10). This will be fixed in SP2 CHF1
None.
HDDS-6974: Container report processing in Recon is single threaded.
This is a known issue in 7.1.7 SP2. Workaround is not available for SP2. This has been fixed by HOTFIX-5211 (CHF10). This will be fixed in SP2 CHF1
None.
CDPD-48742: Secure clusters with Ozone service installed and upgraded from CDP 7.1.6 or earlier versions to CDP 7.1.7 or later versions fail to run container related operations. These operations are available for administrators with the help of the sub-commands of `ozone admin container` command. The failure itself is not visible Secure clusters with Ozone service installed and upgraded from CDP 7.1.6 or earlier versions to CDP 7.1.7 or later versions fail to run container related operations. These operations are available for administrators with the help of the sub-commands of `ozone admin container` command. The failure itself is not visible in the command output, but the underlying operations within the system are failing, the error is reported in the leader DataNode log of the container's Pipeline for example as: 2023-01-18 02:47:00,194 ERROR org.apache.hadoop.hdds.security.x509.certificate.client.DNCertificateClient: Error while signing the stream java.security.InvalidKeyException: Wrong key usage.
The cause of the problem is that SCM uses an inappropriate SSL certificate to sign the container token, and the token signature validation fails on the DataNode.
Note that other operations are not affected by this issue, but internally there will be a burden on Ozone if the system is left without a fix.
To resolve the problem, Storage Container Manager High Availability has to be enabled after the upgrade is finished. As the operations to enable Storage Container Manager High Availability are pretty involved with Ozone internals, if you are upgrading Ozone from CDP 7.1.6 or earlier to CDP 7.1.7 or later please contact our support team in order to go over this exercise.
CDPD-48292: Under certain circumstances Ozone SCM can crash (it happens literally always in our system test run on this non-HA config)
Restart Ozone SCM.
OPSAPS-60721: Ozone SCM Primordial Node ID is a required field which needs to be specified with one of the SCM hostnames during Ozone HA installation. In Cloudera Manager this field is not mandatory during Ozone deployment, this can cause end users continue further with installation which causes startup to fail in Ozone services.
Make sure during ozone HA installation Ozone SCM Primordial Node ID is specified with one of the SCM hostname.
OPSAPS-61253: Host recommission without starting the roles leaves Ozone DataNode in a decommissioned state in Cloudera Manager in case the recommission is initiated without starting the roles on the host to be recommissioned.
In this case a cluster administrator must recommission it from the Ozone > Instances page manually in order to recommission it properly.
CDPD-15268:Uploading a key using the S3 Multi-part upload API into an Ozone encryption zone (TDE-enabled bucket) is not currently supported. The key upload will fail with an exception.
None
CDPD-15602: Creating or deleting keys with a trailing forward slash (/) in the name is not supported via the Ozone shell or the S3 REST API. Such keys are internally treated as directories by the Ozone service for compatibility with the Hadoop filesystem interface. This will be supported in a later release of CDP.
You can create or delete keys via the Hadoop Filesystem interface, either programmatically or via the filesystem Hadoop shell. For example, `ozone fs -rmdir <dir>`.
CDPD-21837:
Adding new Ozone Manager (OM) role instances to an existing cluster will cause the cluster to behave erratically. It can possibly cause split-brain between the Ozone Managers or crash them.
Adding new OM roles to an existing cluster is currently not supported and there is no workaround.
OPSAPS-59647:
Ozone has an optional role where it can deploy a pre-configured Prometheus instance. This prometheus instance's default port '9090' conflicts with HBase Thrift Server's port. Hence, one of the components will fail to start if they are on the same host.

The prometheus port is a directly editable field on the CM UI, with the name 'ozone.prometheus.http-port'. This can be changed to a non conflicting port.

CDPD-24321:
On a secure cluster with Kerberos enabled, the Recon dashboard shows a value of zero for volumes, buckets, and keys.
  • Enable kerberos authentication for HTTP web consoles, if not already enabled, by configuring the ozone.security.http.kerberos.enabled property on Cloudera Manager.
  • Add om/_HOST@REALM,recon/_HOST@REALM to ozone.administrators as an advanced configuration snippet by configuring the Ozone Service Advanced Configuration Snippet (Safety Valve) for ozone-conf/ozone-site.xml property on Cloudera Manager.