Home Forums Hive / HCatalog Unable to run Hive Queries in HDP2.0 with Tez

This topic contains 3 replies, has 3 voices, and was last updated by  Johnny Zhang 1 year, 1 month ago.

  • Creator
    Topic
  • #27828

    I installed and configured Tez to run Hive queries according to the guideline provided in http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.0.0.2/bk_installing_manually_book/content/rpm-chap-tez.html.
    The AMPoolService and the LazyMRAppMasters started alright. When I submitted a simple Hive query (eg. select count(*) from smalltab_txt;) it fails with an exception.

    hive (default)> select count(*) from smalltab_txt;
    Total MapReduce jobs = 1
    Launching Job 1 out of 1
    Number of reduce tasks determined at compile time: 1
    In order to change the average load for a reducer (in bytes):
    set hive.exec.reducers.bytes.per.reducer=
    In order to limit the maximum number of reducers:
    set hive.exec.reducers.max=
    In order to set a constant number of reducers:
    set mapred.reduce.tasks=
    Starting Job = job_1371670413951_0012, Tracking URL = http:///proxy/application_1371670413951_0012/
    Kill Command = //hadoop-2.0.3.22/bin/hadoop job -kill job_1371670413951_0012
    java.io.IOException
    at org.apache.tez.mapreduce.ClientServiceDelegate.invoke(ClientServiceDelegate.java:295)
    at org.apache.tez.mapreduce.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:363)
    at org.apache.tez.mapreduce.YARNRunner.getJobStatus(YARNRunner.java:526)
    at org.apache.hadoop.mapreduce.Job$1.run(Job.java:313)
    at org.apache.hadoop.mapreduce.Job$1.run(Job.java:310)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1441)
    at org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:310)
    at org.apache.hadoop.mapreduce.Job.getJobState(Job.java:342)
    at org.apache.hadoop.mapred.JobClient$NetworkedJob.getJobState(JobClient.java:308)
    at org.apache.hadoop.hive.shims.HadoopShimsSecure.isJobPreparing(HadoopShimsSecure.java:97)
    at org.apache.hadoop.hive.ql.exec.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:238)
    at org.apache.hadoop.hive.ql.exec.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:532)
    at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:454)
    at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136)
    at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:145)
    at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
    at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:47)
    Ended Job = job_1371670413951_0012 with exception ‘java.io.IOException(null)’

    The query however completes properly if I don’t configure Tez to execute the query (plain YARN+MR).

    Any help will be appreciated.

Viewing 3 replies - 1 through 3 (of 3 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #33607

    Johnny Zhang
    Member

    Hi, Ted,
    Any update on this one? I plan to try the Hive 0.11 + ORC + Tec in HDP 2.0. Wondering if the issue is a configuration issue.

    Thanks,
    Johnny

    Collapse
    #28059

    Hi Ted,
    Any update on this issue. If you need any configuration details etc. please let me know. I feel it is a basic issue that a simple query itself is failing.

    Regards
    Saurabh

    Collapse
    #27868

    tedr
    Moderator

    Hi Saurabh,

    Thanks for trying out HDP 2.0. We are looking into your issue and will get back to you as soon as we have an answer.

    Ted.

    Collapse
Viewing 3 replies - 1 through 3 (of 3 total)