Hive / HCatalog Forum

Unable to run Hive Queries in HDP2.0 with Tez

  • #27828

    I installed and configured Tez to run Hive queries according to the guideline provided in http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.0.0.2/bk_installing_manually_book/content/rpm-chap-tez.html.
    The AMPoolService and the LazyMRAppMasters started alright. When I submitted a simple Hive query (eg. select count(*) from smalltab_txt;) it fails with an exception.

    hive (default)> select count(*) from smalltab_txt;
    Total MapReduce jobs = 1
    Launching Job 1 out of 1
    Number of reduce tasks determined at compile time: 1
    In order to change the average load for a reducer (in bytes):
    set hive.exec.reducers.bytes.per.reducer=
    In order to limit the maximum number of reducers:
    set hive.exec.reducers.max=
    In order to set a constant number of reducers:
    set mapred.reduce.tasks=
    Starting Job = job_1371670413951_0012, Tracking URL = http:///proxy/application_1371670413951_0012/
    Kill Command = //hadoop-2.0.3.22/bin/hadoop job -kill job_1371670413951_0012
    java.io.IOException
    at org.apache.tez.mapreduce.ClientServiceDelegate.invoke(ClientServiceDelegate.java:295)
    at org.apache.tez.mapreduce.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:363)
    at org.apache.tez.mapreduce.YARNRunner.getJobStatus(YARNRunner.java:526)
    at org.apache.hadoop.mapreduce.Job$1.run(Job.java:313)
    at org.apache.hadoop.mapreduce.Job$1.run(Job.java:310)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1441)
    at org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:310)
    at org.apache.hadoop.mapreduce.Job.getJobState(Job.java:342)
    at org.apache.hadoop.mapred.JobClient$NetworkedJob.getJobState(JobClient.java:308)
    at org.apache.hadoop.hive.shims.HadoopShimsSecure.isJobPreparing(HadoopShimsSecure.java:97)
    at org.apache.hadoop.hive.ql.exec.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:238)
    at org.apache.hadoop.hive.ql.exec.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:532)
    at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:454)
    at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136)
    at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:145)
    at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
    at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:47)
    Ended Job = job_1371670413951_0012 with exception ‘java.io.IOException(null)’

    The query however completes properly if I don’t configure Tez to execute the query (plain YARN+MR).

    Any help will be appreciated.

to create new topics or reply. | New User Registration

  • Author
    Replies
  • #27868
    tedr
    Moderator

    Hi Saurabh,

    Thanks for trying out HDP 2.0. We are looking into your issue and will get back to you as soon as we have an answer.

    Ted.

    #28059

    Hi Ted,
    Any update on this issue. If you need any configuration details etc. please let me know. I feel it is a basic issue that a simple query itself is failing.

    Regards
    Saurabh

    #33607
    Johnny Zhang
    Member

    Hi, Ted,
    Any update on this one? I plan to try the Hive 0.11 + ORC + Tec in HDP 2.0. Wondering if the issue is a configuration issue.

    Thanks,
    Johnny

You must be to reply to this topic. | Create Account

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.