The Hortonworks Community Connection is now live. A completely rebuilt Q&A forum, Knowledge Base, Code Hub and more, backed by the experts in the industry.

You will be redirected here in 10 seconds. If your are not redirected, click here to visit the new site.

The legacy Hortonworks Forum is now closed. You can view a read-only version of the former site by clicking here. The site will be taken offline on January 31,2016

Oozie Forum

Problems with Hive and Oozie

  • #40972
    Michael Peterson

    After a great deal of effort, tweaking and adjusting numerous things, I am able to run Hive actions in Oozie to do at least some things, but my overall action is still failing on the last step. To jump ahead to the punchline, here is the error message:

    LOAD DATA INPATH ‘/user/root/examples/input-data/yytable2/yytable’ INTO TABLE xmas
    2013-10-18 11:25:37,119 INFO hive.ql.parse.ParseDriver: Parse Completed
    2013-10-18 11:25:37,149 ERROR org.apache.hadoop.hive.ql.Driver: FAILED: SemanticException [Error 10028]: Line 3:17 Path is not legal ”/user/root/examples/input-data/yytable2/yytable”: Move from: hdfs:// to: hdfs:// is not valid. Please check that values for params “” and “hive.metastore.warehouse.dir” do not conflict.
    org.apache.hadoop.hive.ql.parse.SemanticException: Line 3:17 Path is not legal ”/user/root/examples/input-data/yytable2/yytable”: Move from: hdfs:// to: hdfs:// is not valid. Please check that values for params “” and “hive.metastore.warehouse.dir” do not conflict.
    at org.apache.hadoop.hive.ql.parse.LoadSemanticAnalyzer.applyConstraints(
    at org.apache.hadoop.hive.ql.parse.LoadSemanticAnalyzer.analyzeInternal(

    The background:

    The hive script is:


    CREATE EXTERNAL TABLE yytest2 (x int)
    LOCATION '/user/root/examples/input-data/yytable2/yytable';

    INSERT OVERWRITE DIRECTORY '/user/root/examples/output-data/hive2yy' SELECT * FROM yytest2;


    LOAD DATA INPATH '/user/root/examples/input-data/yytable2/yytable' INTO TABLE xmas;

    When I run this Hive script on my Hadoop system in the Hive shell it runs fine and the last Hive command (to populate the “xmas” table) goes into the Hive warehouse at /apps/hive/warehouse/xmas.

    In order to run this on Oozie I had to make numerous adjustments including following the steps here:
    and including the hive-site.xml in my hive “oozie package” which gets put into HDFS in order to run the oozie workflow.

    When the oozie job finishes it ends in an error state, but the first 4 commands in the hive script worked – both tables were created. But the final command fails (remember it works fine when I run it myself in the hive shell), with the error message at the top of this note.

    Please advise how to get this to run.

  • Author
  • #40973
    Michael Peterson

    Since you have a character limit on posts, this is part two:

    The error says that “” and “hive.metastore.warehouse.dir” conflict. I cannot find “” anywhere in my system. The hive.metastore.warehouse.dir was set to ‘/apps/hive/warehouse’, but I see that there is also a ‘/user/hive/warehouse’ HDFS dir, so I changed the property to be that and it fails with the same error message. Suggestions online say to use metatool to determine what the is, but that fails for me:

    $ /usr/lib/hive/bin/metatool -listFSRoot
    Initializing HiveMetaTool..
    HiveMetaTool:Parsing failed. Reason: Unrecognized option: -hiveconf

    I’m using a CentOS VMWare VM running Hortonworks HDP 1.3
    Oozie version:
    Hive version: 0.11 (

    Here’s the workflow.xml that runs the above script:




    Failed, error message[${wf:errorMessage(wf:lastErrorNode())}]

    The “job-xml” reference includes the hive-site.xml from /etc/hive/conf with the metastore password added.

    Michael Peterson

    This forum has no way to preview or delete your own posts. That really is quite frustrating.

    The XML did not post correctly in the previous follow-ups, so here it is again:

    Here’s the workflow.xml that runs the above script:

    &lt?xml version="1.0" encoding="UTF-8"?>
    &ltworkflow-app xmlns="uri:oozie:workflow:0.2" name="hive-wf">
    &ltstart to="hive-node"/>

    &ltaction name="hive-node">
    &lthive xmlns="uri:oozie:hive-action:0.2">
    &ltdelete path="${nameNode}/user/${wf:user()}/${examplesRoot}/output-data/hive2"/>
    &ltmkdir path="${nameNode}/user/${wf:user()}/${examplesRoot}/output-data"/>
    &ltok to="end"/>
    &lterror to="fail"/>

    &ltkill name="fail">
    &ltmessage>Failed, error message[${wf:errorMessage(wf:lastErrorNode())}]&lt/message>
    &ltend name="end"/>

    The “job-xml” reference includes the hive-site.xml from /etc/hive/conf with the metastore password added.

    Yi Zhang

    Hi Michael,

    The error message params looks odd, the property should be called in hadoop 1.x in core-site.xml.

    is the ip address for

    In oozie-site.xml have you pointed it to the correct hadoop conf directory?

    In file is the namenode pointing to the correct FQDN of the name node?


    Michael Peterson

    Hi Yi,

    Switching my nameNode and jobTracker properties to the FQDN, rather than the IP address, fixed it:

    It’s a little distressing that Hive/Hadoop is so finicky that it can’t use the IP address or FQDN interchangably (this VM has only one (virtual) NIC). In any case, thanks for your help.


The topic ‘Problems with Hive and Oozie’ is closed to new replies.

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.