Release Notes
Also available as:
PDF

HBase

This release provides HBase 1.1.2 with no additional Apache patches.

HDP 2.6.3 provided HBase 1.1.2 and the following Apache patches:

  • HBASE-9393: Hbase does not closing a closed socket resulting in many CLOSE_WAIT.

  • HBASE-11013: Clone Snapshots on Secure Cluster Should provide option to apply Retained User Permissions.

  • HBASE-14469: Fix some comment, validation and logging around memstore lower limit configuration.

  • HBASE-14584: TestNamespacesInstanceModel fails on jdk8.

  • HBASE-14644: Region in transition metric is broken (Huaxiang Sun).

  • HBASE-14663: HStore::close does not honor config hbase.rs.evictblocksonclose.

  • HBASE-15160: Put back HFile's HDFS op latency sampling code and add metrics for monitoring (Yu Li and Enis Soztutar) HBASE-18184 Add hbase-hadoop2-compat jar as MapReduce dependency.

  • HBASE-15198: RPC client not using Codec and CellBlock for puts by default.

  • HBASE-15203: Reduce garbage created by path.toString() during Checksum verification.

  • HBASE-15214: Valid mutate Ops fail with RPC Codec in use and region moves across.

  • HBASE-15433: SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception (Jianwei Cui).

  • HBASE-16700: Allow for coprocessor whitelisting.

  • HBASE-16886: hbase-client: scanner with reversed=true and small=true gets no result.

  • HBASE-17002: JMX metrics and some UI additions for space quotas.

  • HBASE-17003: Documentation updates for space quotas.

  • HBASE-17127: Locate region should fail fast if underlying Connection already closed.

  • HBASE-17215: Separate small/large file delete threads in HFileCleaner to accelerate archived hfile cleanup speed.

  • HBASE-17290: Potential loss of data for replication of bulk loaded hfiles.

  • HBASE-17557: HRegionServer#reportRegionSizesForQuotas() should respond to UnsupportedOperationException.

  • HBASE-17602: Reduce some quota chore periods/delays.

  • HBASE-17612: [C++] Set client version info in RPC header.

  • HBASE-17794: Swap "violation" for "snapshot" where appropriate.

  • HBASE-17854: Use StealJobQueue in HFileCleaner.

  • HBASE-17955: Various reviewboard improvements to space quota work.

  • HBASE-17977: Enable the MasterSpaceQuotaObserver by default.

  • HBASE-17978: Ensure superusers can circumvent actions restricted by space quota violations.

  • HBASE-17981: Consolidate the space quota shell commands.

  • HBASE-18024: HRegion#initializeRegionInternals should not re-create .hregioninfo file when the region directory no longer exists.

  • HBASE-18061: [C++] Fix retry logic in multi-get calls (Sudeep Sunthankar).

  • HBASE-18078: [C++] Harden RPC by handling various communication abnormalities.

  • HBASE-18086: Create native client which creates load on selected cluster.

  • HBASE-18093: Overloading the meaning of 'enabled' in Quota Manager to indicate either quota disabled or quota manager not ready is not good (Stephen Yuan Jiang).

  • HBASE-18150: Do not call FSUtils.setVersion() and FSUtils.checkVersion() when using checkRootDir() to check hbase.wal.dir.

  • HBASE-18168: NoSuchElementException when rolling the log (Allan Yang).

  • HBASE-18204: [C++] Rpc connection close and reconnecting.

  • HBASE-18338: [C++] Implement RpcTestServer (Xiaobing Zhou).

  • HBASE-18371: [C++] Update folly and wangle dependencies.

  • HBASE-18400: [C++] ConnectionId Equals/Hash should consider service_name (Xiaobing Zhou).

  • HBASE-18407: [C++] make Configuration::Set/GetBool work for both true/false and 1/0 (Xiaobing Zhou).

  • HBASE-18459: [C++] Fix Segfault in location-cache.

  • HBASE-18466: [C++] Support handling exception in RpcTestServer.

  • HBASE-18471: The DeleteFamily cell is skipped when StoreScanner seeks to next column.

  • HBASE-18507: [C++] Support for MultiPuts in AsyncBatchRpcRetryingCaller class (Sudeep Sunthankar).

  • HBASE-18536: [C++] Add fault injection infra.

  • HBASE-18537: [C++] Improvements to load-client.

  • HBASE-18564: [C++] Problems compiling with GCC.

  • HBASE-18565: [C++] Fix deadlock in AsyncScanRetryingCaller and other RPCs.

  • HBASE-18568: Correct metric of numRegions.

  • HBASE-18576: [C++] Add ping for RPC test.

  • HBASE-18578: [C++] Add pause for RPC test.

  • HBASE-18579: Enable core dump by default for docker.

  • HBASE-18603: buck complains about Build target path containing double slash.

  • HBASE-18614: Setting BUCKET_CACHE_COMBINED_KEY to false disables stats on RS UI.

  • HBASE-18629: Enhance ChaosMonkeyRunner with interruptibility.

  • HBASE-18631: Allow ChaosMonkey properties to be specified in hbase-site.

  • HBASE-18646: LogRollMasterProcedureManager: make procedure timeout, thread pool size configurable.

  • HBASE-18665: ReversedScannerCallable invokes getRegionLocations incorrectly.

  • HBASE-18679: Add a null check around the result of getCounters() in ITBLL.

  • HBASE-18720: [C++] Update wangle dependency to pick up the new release with Apache License v2.

  • HBASE-18725: [C++] Install header files as well as library.

  • HBASE-18727: [C++] Don't link JNI/JVM stuff into libHbaseClient.

  • HBASE-18728: [C++] The library should be named libHBaseClient, instead of libHbaseClient.

  • HBASE-18743: HFiles in use by a table which has the same name and namespace with a default table cloned from snapshot may be deleted when that snapshot and default table are deleted (wenbang).

  • HBASE-18757: Fix Improper bitwise & in BucketCache offset calculation.

  • HBASE-18777: Create one table for subtests in multi-retry-test.

  • HBASE-18802: [C++] Remove hbase-macros.h.

  • HBASE-18891: BUG-89216 Upgrade netty-all dependency.

HDP 2.6.2 provides HBase 1.1.2 and the following Apache patches:

  • HBASE-13706: CoprocessorClassLoader should not exempt Hive classes.

  • HBASE-14425: In Secure ZooKeeper cluster superuser will not have sufficient permission if multiple values are configured in hbase.superuser.

  • HBASE-14492: Increase REST server header buffer size from 8k to 64k.

  • HBASE-14734: Prevent BindException when setting up MiniKdc. Port for kdc service gets selected in the constructor, but we bind to it later in MiniKdc.start()-->MiniKdc.initKDCServer() --> KdcServer.start(). In meantime, some other service can capture the port which results in BindException. The solution here is to catch the exception and retry.

  • HBASE-15177: Reduce garbage created under high load.

  • HBASE-15645: hbase.rpc.timeout is not used in operations of HTable.

  • HBASE-16345: RpcRetryingCallerWithReadReplicas#call() should catch some RegionServer Exceptions.

  • HBASE-16420: Fix source incompatibility of Table interface.

  • HBASE-16488: Starting namespace and quota services in master startup asynchronizely.

  • HBASE-16767: Mob compaction needs to clean up files - addendum removes createMobRefKeyValue().

  • HBASE-16767: Mob compaction needs to clean up files in /hbase/mobdir/.tmp.

  • HBASE-16812: Clean up the locks in MOB.

  • HBASE-16841: Data loss in MOB files after cloning a snapshot and deleting that snapshot.

  • HBASE-17005: Improve log message in MobFileCache.

  • HBASE-17095: The ClientSimpleScanner keeps retrying if the hfile is corrupt or cannot found.

  • HBASE-17151: New API to create HFile.Reader without instantiating block cache.

  • HBASE-17241: Avoid compacting already compacted mob files with _del files.

  • HBASE-17547: TableCatelog doesn't support multiple columns from Single Column family.

  • HBASE-17922: Clean TestRegionServerHostname for hadoop3..

  • HBASE-18312: Ineffective handling of FileNotFoundException in FileLink.tryOpen().

  • HBASE-18358: Backport HBASE-18099 'FlushSnapshotSubprocedure should wait for concurrent Region#flush() to finish'.

HDP 2.6.1 provided HBase 1.1.2 and the following Apache patches:

  • HBASE-17287: Master becomes a zombie if filesystem object closes.

  • HBASE-17501: guard against NPE while reading FileTrailer and HFileBlock.

  • HBASE-17779: disable_table_replication returns misleading message and does not turn off replication.

  • HBASE-17785: RSGroupBasedLoadBalancer fails to assign new table regions when cloning snapshot.

  • HBASE-17861: Regionserver down when checking the permission of staging dir if hbase.rootdir is on S3.

  • HBASE-18036: Data locality is not maintained after cluster restart or SSH.

  • HBASE-18067: Allow default FORMATTER for shell put/get commands.

  • HBASE-18075: Support non-latin table names and namespaces.

HDP 2.6.0 provided HBase 1.1.2 and the following Apache patches:

  • BUG-45027:/BUG-45361 Fix hostname used with Kerberos on Dual-Homed clusters..

  • HBASE-13318: RpcServer.getListenerAddress should handle when the accept channel is closed (Andrew Purtell).

  • HBASE-13829: Add more ThrottleType (Guanghao Zhang).

  • HBASE-14123: HBase Backup/Restore Phase 2.

  • HBASE-14140: HBase Backup Admin API.

  • HBASE-14148: Web UI Framable Page.

  • HBASE-14172: Upgrade existing thrift binding using thrift 0.9.3 compiler (Josh Elser).

  • HBASE-14258: Make region_mover.rb script case insensitive with regard to hostname.

  • HBASE-14269: FuzzyRowFilter omits certain rows when multiple fuzzy key exist.

  • HBASE-14283: Reverse scan doesn’t work with HFile inline index/bloom blocks.

  • HBASE-14302: TableSnapshotInputFormat should not create back references when restoring snapshot.

  • HBASE-14307: Incorrect use of positional read api in HFileBlock (Chris Nauroth).

  • HBASE-14313: After a Connection sees ConnectionClosingException it never recovers (Elliott Clark).

  • HBASE-14407: NotServingRegion: hbase region closed forever (Shuaifeng Zhou).

  • HBASE-14449: Rewrite deadlock prevention for concurrent connection close.

  • HBASE-14535: Integration test for rpc connection concurrency / deadlock testing (Enis Soztutar).

  • HBASE-14536: Balancer & SSH interfering with each other leading to unavailability.

  • HBASE-14598: ByteBufferOutputStream grows its HeapByteBuffer beyond JVM limitations (Ian Friedman).

  • HBASE-14635: Fix flaky test TestSnapshotCloneIndependence.

  • HBASE-14655: Narrow the scope of doAs() calls to region observer notifications for compaction.

  • HBASE-14667: HBaseFsck constructors have diverged (Matteo).

  • HBASE-14680: Two configs for snapshot timeout and better defaults.

  • HBASE-14712: Increase MasterProcWALs clean up granularity (Matteo Bertozzi).

  • HBASE-14805: status should show the master in shell (Enis).

  • HBASE-14818: user_permission does not list namespace permissions (li xiang).

  • HBASE-14843: TestWALProcedureStore.testLoad is flakey (Matteo Bertozz).

  • HBASE-14850: C++ client implementation (TECH PREVIEW. DO NOT DEPLOY IN PROD).

  • HBASE-14906: Improvements on FlushLargeStoresPolicy (Yu Li).

  • HBASE-14947: WALProcedureStore improvements (Matteo Bertozzi).

  • HBASE-14963: Remove Guava dependency from HBase client code (ddas).

  • HBASE-15019: Replication stuck when HDFS is restarted (Matteo).

  • HBASE-15100: Master WALProcs still never clean up (Matteo Bertozzi).

  • HBASE-15125: HBaseFsck's adoptHdfsOrphan function creates region with wrong end key boundary (chenrongwei).

  • HBASE-15136: Explore different queuing behaviors while busy.

  • HBASE-15142: Procedure v2 - Basic WebUI listing the procedures (Matteo).

  • HBASE-15144: Procedure v2 - Web UI displaying Store state.

  • HBASE-15146: Don't block on Reader threads.

  • HBASE-15221: Reload the cache on re-tried puts in HTableMultiplexer and adds a close() method to HTableMultiplexer.

  • HBASE-15232: Handle region location cache mgmt in AsyncProcess for multi()'s.

  • HBASE-15292: Refined ZooKeeperWatcher to prevent ZooKeeper's callback while construction (Hiroshi Ikeda).

  • HBASE-15306: Make RPC call queue length dynamically configurable.

  • HBASE-15377: Per-RS Get metric is time based, per-region metric is size-based (Heng Chen).

  • HBASE-15378: Scanner cannot handle heartbeat message with no results (Phil Yang).

  • HBASE-15465: userPermission returned by getUserPermission() for the selected namespace does not have namespace set (li xiang).

  • HBASE-15518: Add per-table metrics.

  • HBASE-15519: Add per-user metrics.

  • HBASE-15636: hard coded wait time out value in HBaseTestingUtility#waitUntilAllRegionsAssigned might cause test failure (Stephen Yuan Jiang).

  • HBASE-15752: ClassNotFoundException is encountered when custom WAL codec is not found in WALPlayer job.

  • HBASE-15767: upgrade httpclient to 4.3.6.

  • HBASE-15808: Reduce potential bulk load intermediate space usage and waste (Jerry He).

  • HBASE-15817: Backup history should mention the type (full or incremental) of the backup.

  • HBASE-15842: SnapshotInfo should display ownership information.

  • HBASE-15850: Localize the configuration change in testCheckTableLocks to reduce flakiness of TestHBaseFsck test suite (Stephen Yuan Jiang).

  • HBASE-15858: Some region server group shell commands don't work.

  • HBASE-15861: Add support for table sets in restore operation .

  • HBASE-15862: Backup - Delete- Restore does not restore deleted data.

  • HBASE-15873: ACL for snapshot restore / clone is not enforced.

  • HBASE-15880: RpcClientImpl#tracedWriteRequest incorrectly closes HTrace span (Mikhail).

  • HBASE-15884: NPE in StoreFileScanner#skipKVsNewerThanReadpoint during reverse scan (Sergey Soldatov).

  • HBASE-15899: HBase incremental restore should handle namespaces properly.

  • HBASE-15904: Use comma as separator for list of tables in BackupInfo.

  • HBASE-15925: compat-module maven variable not evaluated (Sean Busbey).

  • HBASE-15928: hbase backup delete command does not remove backup root dir from hdfs.

  • HBASE-15931: Add log for long-running tasks in AsyncProcess (Yu Li).

  • HBASE-15933: NullPointerException may be thrown from SimpleRegionNormalizer#getRegionSize().

  • HBASE-15953: HBase backup set list command does not list any existing sets.

  • HBASE-15971: Regression: Random Read/WorkloadC slower in 1.x than 0.98 (Stack).

  • HBASE-15972: hbase backup set command should not accept non-existing table.

  • HBASE-16006: FileSystem should be obtained from specified path in WALInputFormat#getSplits().

  • HBASE-16007: Job's Configuration should be passed to TableMapReduceUtil#addDependencyJars() in WALPlayer.

  • HBASE-16012: Major compaction can't work due to obsolete scanner read point in RegionServer (Guanghao Zhang).

  • HBASE-16016: Fix overflow deadline value in AssignmentManager#waitForAssignment (Stephen Yuan Jiang).

  • HBASE-16023: Fastpath for the FIFO rpcscheduler.

  • HBASE-16037: Backup/Restore: Make automatic mode default one (Vladimir Rodionov).

  • HBASE-16045: endtime argument for VerifyReplication was incorrectly specified in usage.

  • HBASE-16048: Tag InternalScanner with LimitedPrivate(HBaseInterfaceAudience.COPROC).

  • HBASE-16052: Improve HBaseFsck Scalability (Ben Lau).

  • HBASE-16056: Procedure v2 - fix master crash for FileNotFound (Matteo Bertozzi).

  • HBASE-16059: Region normalizer fails to trigger merge action where one of the regions is empty.

  • HBASE-16062: Improper error handling in WAL Reader/Writer creation (Vladimir Rodionov).

  • HBASE-16064: delete backup command shows HDFS permission error when deleting the intended backup.

  • HBASE-16065: hbase backup set describe command does not inform if the set does not exist (Vladimir Rodionov).

  • HBASE-16088: hbase restore gives a misleading message if the table doesn't exist.

  • HBASE-16132: Scan does not return all the result when regionserver is busy (binlijin).

  • HBASE-16237: Blocks for hbase:meta table are not cached in L1 cache.

  • HBASE-16267: Remove commons-httpclient dependency from hbase-rest module.

  • HBASE-16284: Unauthorized client can shutdown the cluster.

  • HBASE-16288: HFile intermediate block level indexes might recurse forever creating multi TB files.

  • HBASE-16314: Retry on table snapshot failure.

  • HBASE-16315: RegionSizeCalculator prints region names as binary without escapes.

  • HBASE-16319: Fix TestCacheOnWrite after HBASE-16288 (Andrew Purtell).

  • HBASE-16350: Undo server abort from HBASE-14968.

  • HBASE-16358: Backup create command: log success/failure with backup id.

  • HBASE-16359: NullPointerException in RSRpcServices.openRegion().

  • HBASE-16367: Race between master and region server initialization may lead to premature server abort.

  • HBASE-16377: ServerName check is ineffective in region_mover.rb.

  • HBASE-16429: FSHLog: deadlock if rollWriter called when ring buffer filled with appends.

  • HBASE-16604: Scanner retries on IOException can cause the scans to miss data.

  • HBASE-16675: Average region size may be incorrect.

  • HBASE-16678: MapReduce jobs do not update counters from ScanMetrics.

  • HBASE-16721: Concurrency issue in WAL unflushed seqId tracking.

  • HBASE-16750: Updating asciidoctor plugin version.

  • HBASE-16824: Writer.flush() can be called on already closed streams in WAL roll.

  • HBASE-16931: Setting cell's seqId to zero in compaction flow might cause RS down.

  • HBASE-16985: TestClusterId failed due to wrong hbase rootdir.

  • HBASE-17292: Add observer notification before bulk loaded hfile is moved to region directory.

  • HBASE-5291: Add Kerberos HTTP SPNEGO authentication support to HBase web consoles.