Phoenix

This release provides Phoenix 4.7.0 and the following Apache patches:

• PHOENIX-1751: Perform aggregations, sorting, etc, in the preScannerNext instead of postScannerOpen.

• PHOENIX-2714: Correct byte estimate in BaseResultIterators and expose as interface.

• PHOENIX-2724: Query with large number of guideposts is slower compared to no stats.

• PHOENIX-2855: Workaround Increment TimeRange not being serialized for HBase 1.2.

• PHOENIX-3023: Slow performance when limit queries are executed in parallel by default.

• PHOENIX-3040: Don't use guideposts for executing queries serially.

• PHOENIX-3112: Partial row scan not handled correctly.

• PHOENIX-3240: ClassCastException from Pig loader.

• PHOENIX-3452: NULLS FIRST/NULL LAST should not impact whether GROUP BY is order preserving.

• PHOENIX-3469: Incorrect sort order for DESC primary key for NULLS LAST/NULLS FIRST.

• PHOENIX-3789: Execute cross region index maintenance calls in postBatchMutateIndispensably.

• PHOENIX-3865: IS NULL does not return correct results when first column family not filtered against.

• PHOENIX-4290: Full table scan performed for DELETE with table having immutable indexes.

• PHOENIX-4373: Local index variable length key can have trailing nulls while upserting.

• PHOENIX-4466: java.lang.RuntimeException: response code 500 - Executing a spark job to connect to phoenix query server and load data.

• PHOENIX-4489: HBase Connection leak in Phoenix MR Jobs.

• PHOENIX-4525: Integer overflow in GroupBy execution.

• PHOENIX-4560: ORDER BY with GROUP BY doesn't work if there is WHERE on pk column.

• PHOENIX-4586: UPSERT SELECT doesn't take in account comparison operators for subqueries..

• PHOENIX-4588: Clone expression also if it's children have Determinism.PER_INVOCATION.

HDP 2.6.4 provided Phoenix 4.7.0 with no additional Apache patches.

HDP 2.6.3 provided Phoenix 4.7.0 and the following Apache patches:

• PHOENIX-2236: Various phoenix-spark fixes .

• PHOENIX-2290: Various phoenix-spark fixes.

• PHOENIX-2547: Various phoenix-spark fixes.

• PHOENIX-3101: Remove findbugs' annotations dependency as it is lgpl.

• PHOENIX-3406: CSV BulkLoad MR job incorrectly handle ROW_TIMESTAMP.

• PHOENIX-3427: phoenix-spark: Table undefined for tenant specific view.

• PHOENIX-3525: Follow-on improvements to automatic index rebuilds- It contains PHOENIX-3525,PHOENIX-4099.

• PHOENIX-3532: Pass tenantId parameter to PhoenixRDD when reading.

• PHOENIX-3600: Core MapReduce classes don't provide location info.

• PHOENIX-3601: PhoenixRDD doesn't expose the preferred node locations to Spark.

• PHOENIX-3792: Provide way to skip normalization of column names in phoenix-spark integration.

• PHOENIX-3797: Address issue with compacting local index stores.

• PHOENIX-3832: Local Index - Empty resultset for multi-tenant tables.

• PHOENIX-3858: Index maintenance not required for local indexes of table with immutable rows.

• PHOENIX-3947: Increase scan time out for partial index rebuild and retry only once.

• PHOENIX-3948: Enable shorter time outs for server-side index writes.

• PHOENIX-3964: Index.preWALRestore should handle index write failure.

• PHOENIX-3986: UngroupedAggregateRegionObserver.commitBatch() should set the index metadata as an attribute on every mutation.

• PHOENIX-3997: UngroupedAggregateRegionObserver.commitBatchWithHTable() should not check the memstore size and wait for flush..

• PHOENIX-4027: Mark index as disabled during partial rebuild after configurable amount of time Modification: we will never completely disable the index to avoid user to rebuild it from scratch..

• PHOENIX-4039: Increase default number of RPC retries for our index rebuild task.

• PHOENIX-4099: Follow-on improvements to automatic index rebuilds- It contains PHOENIX-3525,PHOENIX-4099.

• PHOENIX-4188: Disable inline-DTDs in Pherf XML records.

• PHOENIX-4189: Introduce a class that wraps the Map of primary key data.

• PHOENIX-4191: Categorize uncategorized integration tests.

• PHOENIX-4198: Remove need for users to have write permission to SYSTEM.CATALOG table.

• PHOENIX-4224: Automatic resending cache for HashJoin doesn't work when cache has expired on server side.

• PHOENIX-4225: Using Google cache may lead to lock up on RS side..

• PHOENIX-4261: Remove unused netty-all dependency.

HDP 2.6.2 provided Phoenix 4.7.0 and the following Apache patches:

• PHOENIX-3333: Support Spark 2.0.

• PHOENIX-3342: ORDER BY and LIMIT+OFFSET doesnt work on second column from compound key.

• PHOENIX-3481: Phoenix initialization fails for HBase 0.98.21 and beyond.

• PHOENIX-3598: Implement HTTP parameter impersonation for PQS.

• PHOENIX-3633: null pointer exception when subsquery for not exists returns empty result set.

• PHOENIX-3797: Address issue with compacting local index stores.

• PHOENIX-3895: Upgrade to Apache Avatica 1.10.0.

• PHOENIX-3960: PhoenixStorageHandler for Hive doesn't work from Spark jobs.

• PHOENIX-3994: Index RPC priority still depends on the controller factory property in hbase-site.xml.

• PHOENIX-4010: Hash Join cache may not be send to all regionservers when we have stale HBase meta cache.

• PHOENIX-4010: Hash Join cache may not be send to all regionservers when we have stale HBase meta cache(tests addendum).

• PHOENIX-4018: HashJoin may produce nulls for LHS table columns.

• PHOENIX-4042: Add hadoop metrics2-based Indexer coproc metrics.

HDP 2.6.1 provided Phoenix 4.7.0 and the following Apache patches:

• PHOENIX-3710: Cannot use lowername data table name with indextool.

• PHOENIX-3756: Handle users lacking ADMIN for the SYSTEM namespace.

• PHOENIX-3759: Dropping all local index causes NPE during compaction.

• PHOENIX-3800: NPE when doing UPSERT SELECT into salted tables.

• PHOENIX-3833: Handle users lacking ADMIN from failing in system schema creation.

• PHOENIX-3842: Turn off all BloomFilter for Phoenix tables (Lars Hofhansl).

• PHOENIX-3843: Improve logging for UNION ALL errors.

HDP 2.6.0 provides Phoenix 4.7.0 and the following Apache patches:

• PHOENIX-86: UDF testing in phoenix is failing with NoSuchMethodError.

• PHOENIX-117: Phoenix is failing to establish connection on secure envs with NoClassDefFoundError.

• PHOENIX-118: Remove very chatty logging and add notification around cache evictions.

• PHOENIX-930: duplicated columns cause query exception and drop table exception (Junegunn Choi, Kalyan Hadoop).

• PHOENIX-1523: Make it easy to provide a tab literal as separator for CSV imports.

• PHOENIX-1754: Try to handle keytab paths on Windows filesystems.

• PHOENIX-2276: Addednum2 to fix test failures.

• PHOENIX-2276: addendum for fixing test failures.

• PHOENIX-2276: Creating index on a global view on a multi-tenant table fails with NPE.

• PHOENIX-2276: Fix test failure.

• PHOENIX-2743: HivePhoenixHandler for big-big join with predicate push down.

• PHOENIX-2748: Disable auto-commit during bulk load.

• PHOENIX-2758: Ordered GROUP BY not occurring with leading PK equality expression.

• PHOENIX-2784: phoenix-spark: Allow coercion of DATE fields to TIMESTAMP when loading DataFrames(Josh Mahonin).

• PHOENIX-2803: Initializing ToCharFunction with the List <Expression> constructor fails.

• PHOENIX-2817: Phoenix-Spark plugin doesn't work in secured env.

• PHOENIX-2820: Investigate why SortMergeJoinIT has a sort in the explain plan.

• PHOENIX-2828: Ordinality should be 1-based in UNNEST WITH ORDINALITY.

• PHOENIX-2845: Timestamp ignores nanos in literal expressions.

• PHOENIX-2856: HBaseManagedTimeTableReuseTest category isn't getting run.

• PHOENIX-2871: Workaround sqlline issue.

• PHOENIX-2878: CURRENT_TIME fails to provide correct time when projected table is using.

• PHOENIX-2886: Union ALL with Char column not present in the table in Query 1 but in Query 2 throw exception (Alicia Ying Shu).

• PHOENIX-2890: Extend IndexTool to allow incremental index rebuilds.

• PHOENIX-2894: Sort-merge join works incorrectly with DESC columns.

• PHOENIX-2898: HTable not closed in ConnectionQueryServicesImpl (Alex Araujo).

• PHOENIX-2900: Unable to find hash cache once a salted table 's first region has split (chenglei).

• PHOENIX-2905: hadoop-2.5.1 artifacts are in the dependency tree.

• PHOENIX-2908: phoenix-core depends on both antlr 3.5 and antlr 2.7.7 (William Yang).

• PHOENIX-2912: Broken IT tests after PHOENIX-2905.

• PHOENIX-2919: PreparedStatement Returns Incorrect Number of Deleted Records.

• PHOENIX-2920: Incorrect Queries on Multi-tenant tables with WHERE clause containing Row Value Constructor.

• PHOENIX-2930: Cannot resolve columns aliased to its own name.

• PHOENIX-2934: Checking a coerce expression at top level should not be necessary for Union All query (Alicia Ying Shu).

• PHOENIX-2935: IndexMetaData cache can expire when a delete and or query running on server.

• PHOENIX-2936: Missing antlr runtime on server side after PHOENIX-2908.

• PHOENIX-2940: Query the stats table and cache stats in the client.

• PHOENIX-2942: Order by incorrect for RVC.

• PHOENIX-2952: Addendum to fix test case failure (Ram).

• PHOENIX-2952: array_length return negative value (Joseph Sun).

• PHOENIX-2961: select on a composite primary key fails when DESC is set in the table.

• PHOENIX-2983: ClassCastException on auto coerce of BIGINT to DECIMAL.

• PHOENIX-2994: Coalesce function may return null.

• PHOENIX-3004: Allow configuration in hbase-site to define realms other than the server's.

• PHOENIX-3006: Fix all ScalarFunctions to implement clone(List) or <init> (List). (James Taylor & Lars Hofhansl).

• PHOENIX-3008: Prevent upgrade of existing multi-tenant table to map to namespace until we support it correctly.

• PHOENIX-3011: Fix missing apache licenses.

• PHOENIX-3013: TO_CHAR fails to handle indexed null value (Junegunn Choi).

• PHOENIX-3014: SELECT DISTINCT pk ORDER BY pk DESC gives the wrong results with salted tables..

• PHOENIX-3015: Any metadata changes may cause unpredictable result when local indexes are using.

• PHOENIX-3021: Using local index during compaction is producing NPE.

• PHOENIX-3057: Set incremental=false for sqlline-thin.

• PHOENIX-3078:. Hive storage handler does not work if phoenix.table.name contains the schema name.

• PHOENIX-3126: Tie a driver instance to a specific user (Prabhjyot Singh).

• PHOENIX-3134: Ingestion of VARBINARY data should fail when data is not base64-encoded varbinary fields bulk load difference between MR/psql and upserts.

• PHOENIX-3136: Do not relocate org.apache.calcite in queryserver/thin-client.

• PHOENIX-3149: Local index got corrupted if intermediate compaction happen during the split..

• PHOENIX-3152: Incorrect comparator in QueryOptimizer may cause IllegalArgumentException.

• PHOENIX-3161: Improve performance of automatic index rebuilding.

• PHOENIX-3164: Cache UGI instances for remote users in PQS.

• PHOENIX-3189: Perform Kerberos login before ConnectionInfo is constructed.

• PHOENIX-3199: ServerCacheClient sends cache to all regions unnecessarily (chenglei).

• PHOENIX-3223: Add `hadoop classpath` to PQS classpath when available.

• PHOENIX-3236: Problem with shading apache commons on Azure..

• PHOENIX-3241: Support both DATE and TIMESTAMP in CONVERT_TZ().

• PHOENIX-32451: Support both DATE and TIMESTAMP in CONVERT_TZ().

• PHOENIX-3246: Treat U+2002 as whitespace in parser.

• PHOENIX-3271: Distribute UPSERT SELECT across cluster.

• PHOENIX-3280: Automatic attempt to rebuild all disabled index(James Taylor).

• PHOENIX-3346: Hive PhoenixStorageHandler doesn't work well with column.

• PHOENIX-3386: PhoenixStorageHandler throws NPE if local tasks executed via child.

• PHOENIX-3387: Hive PhoenixStorageHandler fails with join on numeric fields.

• PHOENIX-3408: arithmetic/mathematical operations with Decimal columns failed in Hive with PheonixStorageHandler..

• PHOENIX-3416: Memory leak in PhoenixStorageHandler.

• PHOENIX-3422: PhoenixQueryBuilder doesn't make value string correctly for char(/varchar) column type..

• PHOENIX-3423: PhoenixObjectInspector doesn't have information on length of the column..

• PHOENIX-3486: RoundRobinResultIterator doesn't work correctly because of setting Scan's cache size inappropriately in PhoenixInputForamt.

• PHOENIX-3490: PhoenixStorageHandler doesn't handler hbase configuration properly and don't work in secured env.

• PHOENIX-3499: Enable null value for quote character for CSVBulkLoad tool.

• PHOENIX-3503: PhoenixStorageHandler doesn't work properly when execution engine of Hive is Tez..

• PHOENIX-3504: Spark integration doesn't work with decimal columns that are using default precision.

• PHOENIX-3505: Avoid NPE on close() in OrderedResultIterator.

• PHOENIX-3512: PhoenixStorageHandler makes erroneous query string when handling between clauses with date constants..

• PHOENIX-3521: Scan over local index may return incorrect result after flush & compaction.

• PHOENIX-3540: BUG-70567 Fix Time data type in Phoenix Spark integration.

• PHOENIX-3544: Fix UNSIGNED_DATE and UNSIGNED_TIME type in phoenix-spark.

• PHOENIX-3565: Set failIfNoTests=false on surefire and failsafe.

• PHOENIX-3608: KeyRange interset should return EMPTY_RANGE when one of it is NULL_RANGE(Rajeshbabu).

• PHOENIX-3609: Detect and fix corrupted local index region during compaction.

• PHOENIX-3610: Fix tableName used to get the index maintainers while creating HalfStoreFileReader for local index store.

• PHOENIX-3652: User requires all Create privilege on 'default' namespace (Ankit Singhal).

• PHOENIX-3662: PhoenixStorageHandler throws ClassCastException (by Jeongdae Kim).

• PHOENIX-3684: Handle _HOST in principal specified in JDBC url.

• PHOENIX-3686: Allow client-authentication to be disabled for PQS.

• PHOENIX-3694: Drop schema does not invalidate schema from the server cache.

• PHOENIX-3698: No-args constructor for IndexedWALEditCodec.

• PHOENIX-3708: Tests introduced in PHOENIX-3346 doesn't work well with failsafe plugin.