Home Forums Hive / HCatalog Unable to run Hive Queries in HDP2.0 with Tez

This topic contains 3 replies, has 3 voices, and was last updated by  Johnny Zhang 1 year, 3 months ago.

  • Creator
  • #27828

    I installed and configured Tez to run Hive queries according to the guideline provided in http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-
    The AMPoolService and the LazyMRAppMasters started alright. When I submitted a simple Hive query (eg. select count(*) from smalltab_txt;) it fails with an exception.

    hive (default)> select count(*) from smalltab_txt;
    Total MapReduce jobs = 1
    Launching Job 1 out of 1
    Number of reduce tasks determined at compile time: 1
    In order to change the average load for a reducer (in bytes):
    set hive.exec.reducers.bytes.per.reducer=
    In order to limit the maximum number of reducers:
    set hive.exec.reducers.max=
    In order to set a constant number of reducers:
    set mapred.reduce.tasks=
    Starting Job = job_1371670413951_0012, Tracking URL = http:///proxy/application_1371670413951_0012/
    Kill Command = //hadoop- job -kill job_1371670413951_0012
    at org.apache.tez.mapreduce.ClientServiceDelegate.invoke(ClientServiceDelegate.java:295)
    at org.apache.tez.mapreduce.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:363)
    at org.apache.tez.mapreduce.YARNRunner.getJobStatus(YARNRunner.java:526)
    at org.apache.hadoop.mapreduce.Job$1.run(Job.java:313)
    at org.apache.hadoop.mapreduce.Job$1.run(Job.java:310)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1441)
    at org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:310)
    at org.apache.hadoop.mapreduce.Job.getJobState(Job.java:342)
    at org.apache.hadoop.mapred.JobClient$NetworkedJob.getJobState(JobClient.java:308)
    at org.apache.hadoop.hive.shims.HadoopShimsSecure.isJobPreparing(HadoopShimsSecure.java:97)
    at org.apache.hadoop.hive.ql.exec.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:238)
    at org.apache.hadoop.hive.ql.exec.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:532)
    at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:454)
    at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136)
    at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:145)
    at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
    at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:47)
    Ended Job = job_1371670413951_0012 with exception ‘java.io.IOException(null)’

    The query however completes properly if I don’t configure Tez to execute the query (plain YARN+MR).

    Any help will be appreciated.

Viewing 3 replies - 1 through 3 (of 3 total)

You must be logged in to reply to this topic.

  • Author
  • #33607

    Johnny Zhang

    Hi, Ted,
    Any update on this one? I plan to try the Hive 0.11 + ORC + Tec in HDP 2.0. Wondering if the issue is a configuration issue.



    Hi Ted,
    Any update on this issue. If you need any configuration details etc. please let me know. I feel it is a basic issue that a simple query itself is failing.




    Hi Saurabh,

    Thanks for trying out HDP 2.0. We are looking into your issue and will get back to you as soon as we have an answer.


Viewing 3 replies - 1 through 3 (of 3 total)