Unable to run Hive Queries in HDP2.0 with Tez

to create new topics or reply. | New User Registration

This topic contains 3 replies, has 3 voices, and was last updated by  Johnny Zhang 1 year, 11 months ago.

  • Creator
    Topic
  • #27828

    I installed and configured Tez to run Hive queries according to the guideline provided in http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.0.0.2/bk_installing_manually_book/content/rpm-chap-tez.html.
    The AMPoolService and the LazyMRAppMasters started alright. When I submitted a simple Hive query (eg. select count(*) from smalltab_txt;) it fails with an exception.

    hive (default)> select count(*) from smalltab_txt;
    Total MapReduce jobs = 1
    Launching Job 1 out of 1
    Number of reduce tasks determined at compile time: 1
    In order to change the average load for a reducer (in bytes):
    set hive.exec.reducers.bytes.per.reducer=
    In order to limit the maximum number of reducers:
    set hive.exec.reducers.max=
    In order to set a constant number of reducers:
    set mapred.reduce.tasks=
    Starting Job = job_1371670413951_0012, Tracking URL = http:///proxy/application_1371670413951_0012/
    Kill Command = //hadoop-2.0.3.22/bin/hadoop job -kill job_1371670413951_0012
    java.io.IOException
    at org.apache.tez.mapreduce.ClientServiceDelegate.invoke(ClientServiceDelegate.java:295)
    at org.apache.tez.mapreduce.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:363)
    at org.apache.tez.mapreduce.YARNRunner.getJobStatus(YARNRunner.java:526)
    at org.apache.hadoop.mapreduce.Job$1.run(Job.java:313)
    at org.apache.hadoop.mapreduce.Job$1.run(Job.java:310)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1441)
    at org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:310)
    at org.apache.hadoop.mapreduce.Job.getJobState(Job.java:342)
    at org.apache.hadoop.mapred.JobClient$NetworkedJob.getJobState(JobClient.java:308)
    at org.apache.hadoop.hive.shims.HadoopShimsSecure.isJobPreparing(HadoopShimsSecure.java:97)
    at org.apache.hadoop.hive.ql.exec.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:238)
    at org.apache.hadoop.hive.ql.exec.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:532)
    at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:454)
    at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136)
    at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:145)
    at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
    at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:47)
    Ended Job = job_1371670413951_0012 with exception ‘java.io.IOException(null)’

    The query however completes properly if I don’t configure Tez to execute the query (plain YARN+MR).

    Any help will be appreciated.

Viewing 3 replies - 1 through 3 (of 3 total)

You must be to reply to this topic. | Create Account

  • Author
    Replies
  • #33607

    Johnny Zhang
    Member

    Hi, Ted,
    Any update on this one? I plan to try the Hive 0.11 + ORC + Tec in HDP 2.0. Wondering if the issue is a configuration issue.

    Thanks,
    Johnny

    Collapse
    #28059

    Hi Ted,
    Any update on this issue. If you need any configuration details etc. please let me know. I feel it is a basic issue that a simple query itself is failing.

    Regards
    Saurabh

    Collapse
    #27868

    tedr
    Moderator

    Hi Saurabh,

    Thanks for trying out HDP 2.0. We are looking into your issue and will get back to you as soon as we have an answer.

    Ted.

    Collapse
Viewing 3 replies - 1 through 3 (of 3 total)
Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.