HBase
This release provides HBase 1.1.2 with no additional Apache patches.
HDP 2.6.3 provided HBase 1.1.2 and the following Apache patches:
HBASE-9393: Hbase does not closing a closed socket resulting in many CLOSE_WAIT.
HBASE-11013: Clone Snapshots on Secure Cluster Should provide option to apply Retained User Permissions.
HBASE-14469: Fix some comment, validation and logging around memstore lower limit configuration.
HBASE-14584: TestNamespacesInstanceModel fails on jdk8.
HBASE-14644: Region in transition metric is broken (Huaxiang Sun).
HBASE-14663: HStore::close does not honor config hbase.rs.evictblocksonclose.
HBASE-15160: Put back HFile's HDFS op latency sampling code and add metrics for monitoring (Yu Li and Enis Soztutar) HBASE-18184 Add hbase-hadoop2-compat jar as MapReduce dependency.
HBASE-15198: RPC client not using Codec and CellBlock for puts by default.
HBASE-15203: Reduce garbage created by path.toString() during Checksum verification.
HBASE-15214: Valid mutate Ops fail with RPC Codec in use and region moves across.
HBASE-15433: SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception (Jianwei Cui).
HBASE-16700: Allow for coprocessor whitelisting.
HBASE-16886: hbase-client: scanner with reversed=true and small=true gets no result.
HBASE-17002: JMX metrics and some UI additions for space quotas.
HBASE-17003: Documentation updates for space quotas.
HBASE-17127: Locate region should fail fast if underlying Connection already closed.
HBASE-17215: Separate small/large file delete threads in HFileCleaner to accelerate archived hfile cleanup speed.
HBASE-17290: Potential loss of data for replication of bulk loaded hfiles.
HBASE-17557: HRegionServer#reportRegionSizesForQuotas() should respond to UnsupportedOperationException.
HBASE-17602: Reduce some quota chore periods/delays.
HBASE-17612: [C++] Set client version info in RPC header.
HBASE-17794: Swap "violation" for "snapshot" where appropriate.
HBASE-17854: Use StealJobQueue in HFileCleaner.
HBASE-17955: Various reviewboard improvements to space quota work.
HBASE-17977: Enable the MasterSpaceQuotaObserver by default.
HBASE-17978: Ensure superusers can circumvent actions restricted by space quota violations.
HBASE-17981: Consolidate the space quota shell commands.
HBASE-18024: HRegion#initializeRegionInternals should not re-create .hregioninfo file when the region directory no longer exists.
HBASE-18061: [C++] Fix retry logic in multi-get calls (Sudeep Sunthankar).
HBASE-18078: [C++] Harden RPC by handling various communication abnormalities.
HBASE-18086: Create native client which creates load on selected cluster.
HBASE-18093: Overloading the meaning of 'enabled' in Quota Manager to indicate either quota disabled or quota manager not ready is not good (Stephen Yuan Jiang).
HBASE-18150: Do not call FSUtils.setVersion() and FSUtils.checkVersion() when using checkRootDir() to check hbase.wal.dir.
HBASE-18168: NoSuchElementException when rolling the log (Allan Yang).
HBASE-18204: [C++] Rpc connection close and reconnecting.
HBASE-18338: [C++] Implement RpcTestServer (Xiaobing Zhou).
HBASE-18371: [C++] Update folly and wangle dependencies.
HBASE-18400: [C++] ConnectionId Equals/Hash should consider service_name (Xiaobing Zhou).
HBASE-18407: [C++] make Configuration::Set/GetBool work for both true/false and 1/0 (Xiaobing Zhou).
HBASE-18459: [C++] Fix Segfault in location-cache.
HBASE-18466: [C++] Support handling exception in RpcTestServer.
HBASE-18471: The DeleteFamily cell is skipped when StoreScanner seeks to next column.
HBASE-18507: [C++] Support for MultiPuts in AsyncBatchRpcRetryingCaller class (Sudeep Sunthankar).
HBASE-18536: [C++] Add fault injection infra.
HBASE-18537: [C++] Improvements to load-client.
HBASE-18564: [C++] Problems compiling with GCC.
HBASE-18565: [C++] Fix deadlock in AsyncScanRetryingCaller and other RPCs.
HBASE-18568: Correct metric of numRegions.
HBASE-18576: [C++] Add ping for RPC test.
HBASE-18578: [C++] Add pause for RPC test.
HBASE-18579: Enable core dump by default for docker.
HBASE-18603: buck complains about Build target path containing double slash.
HBASE-18614: Setting BUCKET_CACHE_COMBINED_KEY to false disables stats on RS UI.
HBASE-18629: Enhance ChaosMonkeyRunner with interruptibility.
HBASE-18631: Allow ChaosMonkey properties to be specified in hbase-site.
HBASE-18646: LogRollMasterProcedureManager: make procedure timeout, thread pool size configurable.
HBASE-18665: ReversedScannerCallable invokes getRegionLocations incorrectly.
HBASE-18679: Add a null check around the result of getCounters() in ITBLL.
HBASE-18720: [C++] Update wangle dependency to pick up the new release with Apache License v2.
HBASE-18725: [C++] Install header files as well as library.
HBASE-18727: [C++] Don't link JNI/JVM stuff into libHbaseClient.
HBASE-18728: [C++] The library should be named libHBaseClient, instead of libHbaseClient.
HBASE-18743: HFiles in use by a table which has the same name and namespace with a default table cloned from snapshot may be deleted when that snapshot and default table are deleted (wenbang).
HBASE-18757: Fix Improper bitwise & in BucketCache offset calculation.
HBASE-18777: Create one table for subtests in multi-retry-test.
HBASE-18802: [C++] Remove hbase-macros.h.
HBASE-18891: BUG-89216 Upgrade netty-all dependency.
HDP 2.6.2 provides HBase 1.1.2 and the following Apache patches:
HBASE-13706: CoprocessorClassLoader should not exempt Hive classes.
HBASE-14425: In Secure ZooKeeper cluster superuser will not have sufficient permission if multiple values are configured in hbase.superuser.
HBASE-14492: Increase REST server header buffer size from 8k to 64k.
HBASE-14734: Prevent BindException when setting up MiniKdc. Port for kdc service gets selected in the constructor, but we bind to it later in MiniKdc.start()-->MiniKdc.initKDCServer() --> KdcServer.start(). In meantime, some other service can capture the port which results in BindException. The solution here is to catch the exception and retry.
HBASE-15177: Reduce garbage created under high load.
HBASE-15645: hbase.rpc.timeout is not used in operations of HTable.
HBASE-16345: RpcRetryingCallerWithReadReplicas#call() should catch some RegionServer Exceptions.
HBASE-16420: Fix source incompatibility of Table interface.
HBASE-16488: Starting namespace and quota services in master startup asynchronizely.
HBASE-16767: Mob compaction needs to clean up files - addendum removes createMobRefKeyValue().
HBASE-16767: Mob compaction needs to clean up files in /hbase/mobdir/.tmp.
HBASE-16812: Clean up the locks in MOB.
HBASE-16841: Data loss in MOB files after cloning a snapshot and deleting that snapshot.
HBASE-17005: Improve log message in MobFileCache.
HBASE-17095: The ClientSimpleScanner keeps retrying if the hfile is corrupt or cannot found.
HBASE-17151: New API to create HFile.Reader without instantiating block cache.
HBASE-17241: Avoid compacting already compacted mob files with _del files.
HBASE-17547: TableCatelog doesn't support multiple columns from Single Column family.
HBASE-17922: Clean TestRegionServerHostname for hadoop3..
HBASE-18312: Ineffective handling of FileNotFoundException in FileLink.tryOpen().
HBASE-18358: Backport HBASE-18099 'FlushSnapshotSubprocedure should wait for concurrent Region#flush() to finish'.
HDP 2.6.1 provided HBase 1.1.2 and the following Apache patches:
HBASE-17287: Master becomes a zombie if filesystem object closes.
HBASE-17501: guard against NPE while reading FileTrailer and HFileBlock.
HBASE-17779: disable_table_replication returns misleading message and does not turn off replication.
HBASE-17785: RSGroupBasedLoadBalancer fails to assign new table regions when cloning snapshot.
HBASE-17861: Regionserver down when checking the permission of staging dir if hbase.rootdir is on S3.
HBASE-18036: Data locality is not maintained after cluster restart or SSH.
HBASE-18067: Allow default FORMATTER for shell put/get commands.
HBASE-18075: Support non-latin table names and namespaces.
HDP 2.6.0 provided HBase 1.1.2 and the following Apache patches:
BUG-45027:/BUG-45361 Fix hostname used with Kerberos on Dual-Homed clusters..
HBASE-13318: RpcServer.getListenerAddress should handle when the accept channel is closed (Andrew Purtell).
HBASE-13829: Add more ThrottleType (Guanghao Zhang).
HBASE-14123: HBase Backup/Restore Phase 2.
HBASE-14140: HBase Backup Admin API.
HBASE-14148: Web UI Framable Page.
HBASE-14172: Upgrade existing thrift binding using thrift 0.9.3 compiler (Josh Elser).
HBASE-14258: Make region_mover.rb script case insensitive with regard to hostname.
HBASE-14269: FuzzyRowFilter omits certain rows when multiple fuzzy key exist.
HBASE-14283: Reverse scan doesn’t work with HFile inline index/bloom blocks.
HBASE-14302: TableSnapshotInputFormat should not create back references when restoring snapshot.
HBASE-14307: Incorrect use of positional read api in HFileBlock (Chris Nauroth).
HBASE-14313: After a Connection sees ConnectionClosingException it never recovers (Elliott Clark).
HBASE-14407: NotServingRegion: hbase region closed forever (Shuaifeng Zhou).
HBASE-14449: Rewrite deadlock prevention for concurrent connection close.
HBASE-14535: Integration test for rpc connection concurrency / deadlock testing (Enis Soztutar).
HBASE-14536: Balancer & SSH interfering with each other leading to unavailability.
HBASE-14598: ByteBufferOutputStream grows its HeapByteBuffer beyond JVM limitations (Ian Friedman).
HBASE-14635: Fix flaky test TestSnapshotCloneIndependence.
HBASE-14655: Narrow the scope of doAs() calls to region observer notifications for compaction.
HBASE-14667: HBaseFsck constructors have diverged (Matteo).
HBASE-14680: Two configs for snapshot timeout and better defaults.
HBASE-14712: Increase MasterProcWALs clean up granularity (Matteo Bertozzi).
HBASE-14805: status should show the master in shell (Enis).
HBASE-14818: user_permission does not list namespace permissions (li xiang).
HBASE-14843: TestWALProcedureStore.testLoad is flakey (Matteo Bertozz).
HBASE-14850: C++ client implementation (TECH PREVIEW. DO NOT DEPLOY IN PROD).
HBASE-14906: Improvements on FlushLargeStoresPolicy (Yu Li).
HBASE-14947: WALProcedureStore improvements (Matteo Bertozzi).
HBASE-14963: Remove Guava dependency from HBase client code (ddas).
HBASE-15019: Replication stuck when HDFS is restarted (Matteo).
HBASE-15100: Master WALProcs still never clean up (Matteo Bertozzi).
HBASE-15125: HBaseFsck's adoptHdfsOrphan function creates region with wrong end key boundary (chenrongwei).
HBASE-15136: Explore different queuing behaviors while busy.
HBASE-15142: Procedure v2 - Basic WebUI listing the procedures (Matteo).
HBASE-15144: Procedure v2 - Web UI displaying Store state.
HBASE-15146: Don't block on Reader threads.
HBASE-15221: Reload the cache on re-tried puts in HTableMultiplexer and adds a close() method to HTableMultiplexer.
HBASE-15232: Handle region location cache mgmt in AsyncProcess for multi()'s.
HBASE-15292: Refined ZooKeeperWatcher to prevent ZooKeeper's callback while construction (Hiroshi Ikeda).
HBASE-15306: Make RPC call queue length dynamically configurable.
HBASE-15377: Per-RS Get metric is time based, per-region metric is size-based (Heng Chen).
HBASE-15378: Scanner cannot handle heartbeat message with no results (Phil Yang).
HBASE-15465: userPermission returned by getUserPermission() for the selected namespace does not have namespace set (li xiang).
HBASE-15518: Add per-table metrics.
HBASE-15519: Add per-user metrics.
HBASE-15636: hard coded wait time out value in HBaseTestingUtility#waitUntilAllRegionsAssigned might cause test failure (Stephen Yuan Jiang).
HBASE-15752: ClassNotFoundException is encountered when custom WAL codec is not found in WALPlayer job.
HBASE-15767: upgrade httpclient to 4.3.6.
HBASE-15808: Reduce potential bulk load intermediate space usage and waste (Jerry He).
HBASE-15817: Backup history should mention the type (full or incremental) of the backup.
HBASE-15842: SnapshotInfo should display ownership information.
HBASE-15850: Localize the configuration change in testCheckTableLocks to reduce flakiness of TestHBaseFsck test suite (Stephen Yuan Jiang).
HBASE-15858: Some region server group shell commands don't work.
HBASE-15861: Add support for table sets in restore operation .
HBASE-15862: Backup - Delete- Restore does not restore deleted data.
HBASE-15873: ACL for snapshot restore / clone is not enforced.
HBASE-15880: RpcClientImpl#tracedWriteRequest incorrectly closes HTrace span (Mikhail).
HBASE-15884: NPE in StoreFileScanner#skipKVsNewerThanReadpoint during reverse scan (Sergey Soldatov).
HBASE-15899: HBase incremental restore should handle namespaces properly.
HBASE-15904: Use comma as separator for list of tables in BackupInfo.
HBASE-15925: compat-module maven variable not evaluated (Sean Busbey).
HBASE-15928: hbase backup delete command does not remove backup root dir from hdfs.
HBASE-15931: Add log for long-running tasks in AsyncProcess (Yu Li).
HBASE-15933: NullPointerException may be thrown from SimpleRegionNormalizer#getRegionSize().
HBASE-15953: HBase backup set list command does not list any existing sets.
HBASE-15971: Regression: Random Read/WorkloadC slower in 1.x than 0.98 (Stack).
HBASE-15972: hbase backup set command should not accept non-existing table.
HBASE-16006: FileSystem should be obtained from specified path in WALInputFormat#getSplits().
HBASE-16007: Job's Configuration should be passed to TableMapReduceUtil#addDependencyJars() in WALPlayer.
HBASE-16012: Major compaction can't work due to obsolete scanner read point in RegionServer (Guanghao Zhang).
HBASE-16016: Fix overflow deadline value in AssignmentManager#waitForAssignment (Stephen Yuan Jiang).
HBASE-16023: Fastpath for the FIFO rpcscheduler.
HBASE-16037: Backup/Restore: Make automatic mode default one (Vladimir Rodionov).
HBASE-16045: endtime argument for VerifyReplication was incorrectly specified in usage.
HBASE-16048: Tag InternalScanner with LimitedPrivate(HBaseInterfaceAudience.COPROC).
HBASE-16052: Improve HBaseFsck Scalability (Ben Lau).
HBASE-16056: Procedure v2 - fix master crash for FileNotFound (Matteo Bertozzi).
HBASE-16059: Region normalizer fails to trigger merge action where one of the regions is empty.
HBASE-16062: Improper error handling in WAL Reader/Writer creation (Vladimir Rodionov).
HBASE-16064: delete backup command shows HDFS permission error when deleting the intended backup.
HBASE-16065: hbase backup set describe command does not inform if the set does not exist (Vladimir Rodionov).
HBASE-16088: hbase restore gives a misleading message if the table doesn't exist.
HBASE-16132: Scan does not return all the result when regionserver is busy (binlijin).
HBASE-16237: Blocks for hbase:meta table are not cached in L1 cache.
HBASE-16267: Remove commons-httpclient dependency from hbase-rest module.
HBASE-16284: Unauthorized client can shutdown the cluster.
HBASE-16288: HFile intermediate block level indexes might recurse forever creating multi TB files.
HBASE-16314: Retry on table snapshot failure.
HBASE-16315: RegionSizeCalculator prints region names as binary without escapes.
HBASE-16319: Fix TestCacheOnWrite after HBASE-16288 (Andrew Purtell).
HBASE-16350: Undo server abort from HBASE-14968.
HBASE-16358: Backup create command: log success/failure with backup id.
HBASE-16359: NullPointerException in RSRpcServices.openRegion().
HBASE-16367: Race between master and region server initialization may lead to premature server abort.
HBASE-16377: ServerName check is ineffective in region_mover.rb.
HBASE-16429: FSHLog: deadlock if rollWriter called when ring buffer filled with appends.
HBASE-16604: Scanner retries on IOException can cause the scans to miss data.
HBASE-16675: Average region size may be incorrect.
HBASE-16678: MapReduce jobs do not update counters from ScanMetrics.
HBASE-16721: Concurrency issue in WAL unflushed seqId tracking.
HBASE-16750: Updating asciidoctor plugin version.
HBASE-16824: Writer.flush() can be called on already closed streams in WAL roll.
HBASE-16931: Setting cell's seqId to zero in compaction flow might cause RS down.
HBASE-16985: TestClusterId failed due to wrong hbase rootdir.
HBASE-17292: Add observer notification before bulk loaded hfile is moved to region directory.
HBASE-5291: Add Kerberos HTTP SPNEGO authentication support to HBase web consoles.