Hive / HCatalog Forum

MapRedTask failed when executing Hive mapreduce query from beeline

  • #43747

    I have a cluster with HDP 2 installed and running correctly for a few days with the default configuration. The queries to hive work fine using the hive command line.
    The problem appears when I try to query hive from beeline or any another application using jdbc.

    The queries that do not launch any mapreduce jobs (like select *) work fine but the ones that generate mapreduce jobs fail with the following error:

    2013-11-14 10:41:55,502 WARN thrift.ThriftCLIService ( – Error fetching results:
    org.apache.hive.service.cli.HiveSQLException: Error while processing statement: FAILED: Execution Error, return code 1 from
    at org.apache.hive.service.cli.operation.SQLOperation.runInternal(
    at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(
    at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatement(
    at org.apache.hive.service.cli.CLIService.executeStatement(
    at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(
    at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(
    at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(
    at org.apache.thrift.ProcessFunction.process(
    at org.apache.thrift.TBaseProcessor.process(
    at org.apache.hive.service.auth.TUGIContainingProcessor$
    at org.apache.hive.service.auth.TUGIContainingProcessor$
    at Method)
    at org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(
    at org.apache.hive.service.auth.TUGIContainingProcessor.process(
    at org.apache.thrift.server.TThreadPoolServer$
    at java.util.concurrent.ThreadPoolExecutor.runWorker(
    at java.util.concurrent.ThreadPoolExecutor$

    From what I see this is an issue due to incorrect permissions but if I launch the hive command line with hdfs or hive user and run the same queries they work fine.

    Any ideas how to fix this?

to create new topics or reply. | New User Registration

  • Author
  • #43765
    Thejas Nair

    What user are you running hiveserver2 as ? do you see a map reduce job being launched in the RM web ui ?
    Do you have hive.server2.enable.doAs set to any value in hive-site.xml ? Do you have the hive server2 user in core-site.xml proxy user entries ?


    I have all the default values set by the Ambari installer.

    Hiveserver2 is running as user hive. Should I run the query with the same user?

    I have hive.server2.enable.doAs set to true.

    I am not exactly sure what you mean by the proxyuser in the core-site.xml. I don’t have any properties set for hiveserve2 like the ones I have for hive:


    Should I have the same for hiveserver2? which values should they be?

    Thejas Nair

    The “hive” that you are seeing in your core-site.xml *proxyuser* config is the hive user. So that looks right, as long as aws-dev-02 is the hostname of the hiveserver2 machine. You can also try changing the hostname to hadoop.proxyuser.hive.hosts to * , if there some issue with the host name mapping.

    Do you see a map reduce job being launched in the RM web UI ?
    When you tried running hive command line, did you try it from same host as the hive user ?


    I see no map reduce jobs being launched in the YARN RM console or the MR2 JobHistory list.

    You mean running the hive command line client in the aws-dev-02 host? I am running it in another host and also the beeline command line.

    Thejas Nair

    Yes, assuming aws-dev-02 is the hive server2 host, can you try running hive command line from there as hive user ?
    Can you also check the /tmp/hive/hive.log (or different file if the log dir is configured differently for hive client) to see if there is additional error information there ?


    The exception I posted earlier is taken from the hive.log.

    When I ran a mapreduce query in the hive command line in aws-dev-02 as either the hdfs or the hive user I get the following error: Cannot run program “/usr/lib/hadoop/bin/hadoop” (in directory “/home/ec2-user”): error=13, Permission denied
    at java.lang.ProcessBuilder.start(
    at java.lang.Runtime.exec(
    at java.lang.Runtime.exec(

    I usually run the queries from the hive command line with the ec2-user who belongs to the hdfs and hive groups.

    Btw, thanks for all your help Thejas.

    Thejas Nair

    were you able to fix the issue ? Is it a problem with missing home dir for this user or permissions for the bin/hadoop script?


    No, still having the same problem.

    What do you mean by missing home dir? the hive user or hdfs user should have home directories in the aws-dev-02 machine?

    Thejas Nair

    Can you use the same user as what hiveserver2 runs as and run the hive command line from same machine? If you see an error with the home dir with this user as well, can you try creating the home dir ?


    Hi, I have fixed running queries from the hive command line from the aws-dev-02 machine with users hdfs and hive.
    Also, I have realized that running queries that run actual map reduce jobs from beeline from another machine work fine, the problem is when hive transforms the mapreduce query to a local job.

    Any ideas how to fix this?


    The .log file in the /tmp/hive directory of the machine where the hiveserver2 is running shows the following stack trace: Permission denied: user=hive, access=EXECUTE, inode=”/tmp/hive-hive/hive_2013-11-18_12-47-20_879_8044219515320128885-2″:hdfs:hdfs:drwx——
    at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(
    at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkTraverse(
    at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(


    So, I have bypassed this issue by setting dfs.permissions.enabled to false.

    The problem now is that for the query I need to register a csv serde jar. I have added the property hive.aux.jars.path to the hive-site.xml and tried different ways of pointing to the jar but they all fail when running mapreduce jobs in local mode.

    Any ideas how I need to reference this jar or where can I put it?

    Thejas Nair

    By local mode, do you mean running hive command line in local mode ? (not through hive server2). If it is not related to hive server2, creating a new topic would be better.

    For the hive server2 issue, have you checked if changing the permissions of “/tmp/hive-hive/” will fix the issue (instead of disabling dfs permission issue) ?


    By local mode I mean when the the map reduce job is converted to a local map reduce job.

    I tried changing the /tmp/hive-hive permissions but it doesn’t seem to improve anything. I think the issue is originated because the local map-reduce job uses a local filesystem and it does not have the same permissions as the directory in HDFS.

You must be to reply to this topic. | Create Account

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.