Hive Server 2 out of memory due to FileSystem.CACHE leak
Problem: Impersonation. By default, HiveServer2 performs query processing as the user who submitted the query. But if the following parameter is set to false, the query runs as the hiveserver2 process user.
hive.server2.enable.doAs – Impersonate the connected user, default true.
Workaround: To prevent memory leaks in unsecure mode, disable file system caches by setting the following parameters to true:
fs.hdfs.impl.disable.cache – Disable HDFS filesystem cache, default false. fs.file.impl.disable.cache – Disable local filesystem cache, default false.
MapReduce task from Hive dynamic partitioning query is killed.
Problem: When using the Hive script to create and populate the partitioned table dynamically, the following error is reported in the TaskTracker log file:
TaskTree [pid=30275,tipID=attempt_201305041854_0350_m_000000_0] is running beyond memory-limits. Current usage : 1619562496bytes. Limit : 1610612736bytes. Killing task. TaskTree [pid=30275,tipID=attempt_201305041854_0350_m_000000_0] is running beyond memory-limits. Current usage : 1619562496bytes. Limit : 1610612736bytes. Killing task. Dump of the process-tree for attempt_201305041854_0350_m_000000_0 : |- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE |- 30275 20786 30275 30275 (java) 2179 476 1619562496 190241 /usr/jdk64/jdk1.6.0_31/jre/bin/java ...
Workaround: The workaround is disable all the memory settings by setting value of the following perperties to -1 in the
mapred-site.xmlfile on the JobTracker and TaskTracker host machines in your cluster:mapred.cluster.map.memory.mb = -1 mapred.cluster.reduce.memory.mb = -1 mapred.job.map.memory.mb = -1 mapred.job.reduce.memory.mb = -1 mapred.cluster.max.map.memory.mb = -1 mapred.cluster.max.reduce.memory.mb = -1
To change these values using the UI, use the instructions provided here to update these properties.
Problem: While executing the following query:
select s, avg(d) over (partition by i order by f, b) from over100k;
the following error is reported in the Hive log file:
FAILED: SemanticException Range based Window Frame can have only 1 Sort Key
Workaround: The workaround is to use the following query:
select s, avg(d) over (partition by i order by f, b rows unbounded preceding) from over100k;
Problem: While executing the following query:
select s, i, avg(d) over (partition by s order by i) / 10.0 from over100k;
the following error is reported in the Hive log file:
NoViableAltException(15@[129:7: ( ( ( KW_AS )? identifier ) | ( KW_AS LPAREN identifier ( COMMA identifier )* RPAREN ) )?]) at org.antlr.runtime.DFA.noViableAlt(DFA.java:158) at org.antlr.runtime.DFA.predict(DFA.java:116) at org.apache.hadoop.hive.ql.parse.HiveParser_SelectClauseParser.selectItem(HiveParser_SelectClauseParser.java:2298) at org.apache.hadoop.hive.ql.parse.HiveParser_SelectClauseParser.selectList(HiveParser_SelectClauseParser.java:1042) at org.apache.hadoop.hive.ql.parse.HiveParser_SelectClauseParser.selectClause(HiveParser_SelectClauseParser.java:779) at org.apache.hadoop.hive.ql.parse.HiveParser.selectClause(HiveParser.java:30649) at org.apache.hadoop.hive.ql.parse.HiveParser.selectStatement(HiveParser.java:28851) at org.apache.hadoop.hive.ql.parse.HiveParser.regular_body(HiveParser.java:28766) at org.apache.hadoop.hive.ql.parse.HiveParser.queryStatement(HiveParser.java:28306) at org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpression(HiveParser.java:28100) at org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:1213) at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:928) at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:190) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:418) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:337) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:902) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:348) at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:446) at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:456) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:712) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:160) FAILED: ParseException line 1:53 cannot recognize input near '/' '10.0' 'from' in selection target
Workaround: The workaround is to use the following query:
select s, i, avg(d) / 10.0 over (partition by s order by i) from over100k;
Problem: While using indexes in Hive, the following error is reported:
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask
Problem: Partition in hive table that is of datatype
intis able to acceptstringentries. For example,CREATE TABLE tab1 (id1 int,id2 string) PARTITIONED BY(month string,day int) ROW FORMAT DELIMITED FIELDS TERMINATED BY ‘,’ ;
In the above example, the partition day of datatype
intcan also acceptstringentries while data insertions.Workaround: The workaround is to avoid adding
stringtointfields.Problem: In Hive 0.9, setting hive.metastore.local = true in hive-site.xml meant that the embedded metastore would ALWAYS be used regardless of the setting of hive.metastore.uris. But in Hive 0.11, hive.metastore.local is ignored when hive.metastore.uris is set (https://issues.apache.org/jira/browse/HIVE-2585). When upgrading from HDP 1.0 or HDP 1.1 to HDP 1.3.*, Hive is upgraded from 0.9 to 0.11. Therefore, the embedded metastore may no longer be used after upgrading without adjusting the hive-site.xml settings.
Workaround: To continue to use the embedded metastore after upgrading, clear the hive.metastore.uris setting in hive-site.xml.
