Scanner keyvalue size / partial results

Issue: In 2.2 clusters, setting "hbase.client.scanner.max.result.size" differently on the client side or server side might result in missed data from large scans because the scanner might wrongly assume region is exhausted before the region boundary is reached. Additionally, the result size calculation might be different if cell level tags are used even if "hbase.client.scanner.max.result.size" is the same between 2.2 and 2.3 clusters. This issue also affects rolling upgrades between 2.2 and 2.3 clusters, where a 2.2 client might miss data in a large scan when rolling upgrading or talking to 2.3 servers. As a protection, 2.3 does not set "hbase.client.scanner.max.result.size" by default.

Workaround: Do not use partial scan results to be returned (by not setting "hbase.client.scanner.max.result.size") is recommended.

Meta WALs are not split or cleared

Issue: In case a meta region gets moved from one region server to another region server, the write ahead logs for the meta table might be left in the HDFS directory for the regionserver if that particular regionserver fails. This might leave the WAL files in that directory until they are removed manually, and also may result in the previous regionservers to be listed as "dead regionservers" in the master UI indefinitely. However, this is harmless and does not cause any data loss other operational issues in HBase.

Issue: Upgrade to HDP 2.2.6 breaks usersync.

Details: Ranger in HDP 2.2.6 introduced authentication between usersync and the admin REST API. If you upgrade to HDP 2.2.6, there will be a previous version of /etc/ranger/usersync/conf/unixauthservice.properties and {{ /etc/ranger/admin/conf/security-applicationContext.xml}} in place. Because setup.sh treats these files as templates, it expects certain base content to be there in order to generate correct configurations. It looks like there is new configuration in the default templates (the ones in /usr/hdp/2.2.6.0-2800/etc/ranger/...) that will not get pulled into the live configs. This results in usersync pulling 401 errors every time it tries to add/modify a user.

Workaround:

HDFS-5215

HDFS-9038

Issue: Spark: Fails to find table

Details: Spark + Kerberos + yarn-cluster mode + remote hive meta store for SparkSQL does not work.

Workaround: Try this with Spark + Kerberos + yarn-client mode for SparkSQL.

HBASE-13250

HBASE-14445

Local fetch task does not work with UDTF

Workaround: Disable fetch task conversion if using a UDTF that forwards rows during GenericUDTF.close().

Ranger Admin failed to start after the stack upgrade from HDP 2.2.6 to HDP 2.2.9 in AMBARI 2.1.2.1

Issue: If Ranger service is not stopped prior to upgrade (using the old version) and started after upgrade (using the new version), you will encounter issue starting ranger service.

Workaround: Change the permission for stop-ranger-admin.sh file (from root to user running ranger service). The service should start up fine.

Issue: When Ranger is upgraded from version HDP 2.6.0.2.2.8.0-3150 to HDP-2.2.9.0-3350. After upgrade Ranger restart fails with error of missing property in xa_system.properties file: 2015-11-20 09:30:45,268 [localhost-startStop-1] ERROR org.springframework.web.context.ContextLoader (ContextLoader.java:307) - Context initialization failed org.springframework.beans.factory.BeanDefinitionStoreException: Invalid bean definition with name 'defaultDataSource' defined in ServletContext resource [/META-INF/applicationContext.xml]: Could not resolve placeholder 'jdbc.maxIdleTimeExcessConnections' in string value "${jdbc.maxIdleTimeExcessConnections}"

Workaround: Add properties below given properties in xa_system.properties and Retry to start Ranger:

jdbc.maxIdleTimeExcessConnections=300

jdbc.testConnectionOnCheckin=true

This should resolve the issue.

Kafka file system support

Issue: Encrypted file systems such as SafenetFS are not supported for Kafka. Index file corruption can occur.

For more information, see: Install Kafka.

Impact of LDAP Channel Binding and LDAP signing changes in Microsoft Active Directory

Microsoft has introduced changes in LDAP Signing and LDAP Channel Binding to increase the security for communications between LDAP clients and Active Directory domain controllers. These optional changes will have an impact on how 3rd party products integrate with Active Directory using the LDAP protocol.

Workaround

Disable LDAP Signing and LDAP Channel Binding features in Microsoft Active Directory if they are enabled

For more information on this issue, see the corresponding Knowledge article: TSB-2021 405: Impact of LDAP Channel Binding and LDAP signing changes in Microsoft Active Directory

CVE-2020-9492 Hadoop filesystem bindings (ie: webhdfs) allows credential stealing

WebHDFS clients might send SPNEGO authorization header to remote URL without proper verification. A maliciously crafted request can trigger services to send server credentials to a webhdfs path (ie: webhdfs://…) for capturing the service principal

For more information on this issue, see the corresponding Knowledge article: TSB-2021 406: CVE-2020-9492 Hadoop filesystem bindings (ie: webhdfs) allows credential stealing

KMS Load Balancing Provider Fails to invalidate Cache on Key Delete

For more information on this issue, see the corresponding Knowledge article: TSB 2020-434: KMS Load Balancing Provider Fails to invalidate Cache on Key Delete

Corruption of HBase data stored with MOB feature

For more information on this issue, see the corresponding Knowledge article: TSB 2021-465: Corruption of HBase data stored with MOB feature on upgrade from CDH 5 and HDP 2

CVE-2021-27905: Apache Solr SSRF vulnerability with the Replication handler

The Apache Solr ReplicationHandler (normally registered at "/replication" under a Solr core) has a "masterUrl" (also "leaderUrl" alias) parameter. The “masterUrl” parameter is used to designate another ReplicationHandler on another Solr core to replicate index data into the local core. To help prevent the CVE-2021-27905 SSRF vulnerability, Solr should check these parameters against a similar configuration used for the "shards" parameter.

For more information on this issue, see the corresponding Knowledge article: TSB 2021-497: CVE-2021-27905: Apache Solr SSRF vulnerability with the Replication handler

HBase MOB data loss

HBase tables with the MOB feature enabled may encounter problems which result in data loss.

For more information on this issue, see the corresponding Knowledge article: TSB 2021-512: HBase MOB data loss