The Hortonworks Community Connection is now live. A completely rebuilt Q&A forum, Knowledge Base, Code Hub and more, backed by the experts in the industry.

You will be redirected here in 10 seconds. If your are not redirected, click here to visit the new site.

The legacy Hortonworks Forum is now closed. You can view a read-only version of the former site by clicking here. The site will be taken offline on January 31,2016

Oozie Forum

Hive through Oozie on a secured cluster

  • #47986
    Tanzir
    Participant

    Hello all,
    I have a workflow which has a Hive action that loads data into tables. So far it was working fine. I want to mention that so far I was using unsecured cluster (kerberos disabled).

    But when I tried to run the same workflow on a secured cluster, I’m getting errors. It failed to find the tables as it failed to connect to Hive metastore.

    163 [main] INFO org.apache.hadoop.hive.ql.Driver – <PERFLOG method=Driver.run>
    1164 [main] INFO org.apache.hadoop.hive.ql.Driver – <PERFLOG method=TimeToSubmit>
    1164 [main] INFO org.apache.hadoop.hive.ql.Driver – <PERFLOG method=compile>
    1223 [main] INFO hive.ql.parse.ParseDriver – Parsing command: ALTER TABLE TABLE1 ADD IF NOT EXISTS PARTITION(year = 2013, month = 4, day = 8) LOCATION ‘/user/tempuser/etl/hive/table1/2013/04/08/’
    1581 [main] INFO hive.ql.parse.ParseDriver – Parse Completed
    1636 [main] INFO hive.metastore – Trying to connect to metastore with URI thrift://url:9083
    1741 [main] WARN hive.metastore – Failed to connect to the MetaStore Server…
    1741 [main] INFO hive.metastore – Waiting 1 seconds before next connection attempt.
    2741 [main] INFO hive.metastore – Trying to connect to metastore with URI thrift://url:9083
    2744 [main] WARN hive.metastore – Failed to connect to the MetaStore Server…
    2744 [main] INFO hive.metastore – Waiting 1 seconds before next connection attempt.
    3745 [main] INFO hive.metastore – Trying to connect to metastore with URI thrift://url:9083
    3749 [main] WARN hive.metastore – Failed to connect to the MetaStore Server…
    3749 [main] INFO hive.metastore – Waiting 1 seconds before next connection attempt.
    4772 [main] ERROR org.apache.hadoop.hive.ql.Driver – FAILED: SemanticException [Error 10001]: Table not found TABLE1

    Then I saw these two posts:
    https://groups.google.com/a/cloudera.org/forum/#!topic/hue-user/5B77Pq5-uBE
    http://stackoverflow.com/questions/18709840/tables-created-by-oozie-hive-action-cannot-be-found-from-hive-client-but-can-fin

    So, if Hive is used through Oozie on a secured cluster, the metastore which is created by default would not work? I need to create a separate external Hive metastore?

    Is there any other workarounds available?

    Any information will be very helpful.

    Thanks again,
    Tanzir

The forum ‘Oozie’ is closed to new topics and replies.

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.