The Hortonworks Community Connection is now live. A completely rebuilt Q&A forum, Knowledge Base, Code Hub and more, backed by the experts in the industry.

You will be redirected here in 10 seconds. If your are not redirected, click here to visit the new site.

The legacy Hortonworks Forum is now closed. You can view a read-only version of the former site by clicking here. The site will be taken offline on January 31,2016

Hive / HCatalog Forum

Hive/Hbase integration

  • #31241


    I am attempting to extract data from our HDFS placed there using thrift/hbase into a Hive table, but I am struggling to find a solution to the following error message;

    ERROR client.HConnectionManager$HConnectionImplementation ( – Check the value configured in ‘zookeeper.znode.parent’. There could be a mismatch with the one configured in the master.

    followed by;

    FAILED: Error in metadata: MetaException(message:org.apache.hadoop.hbase.MasterNotRunningException

    These errors appear in the hive log after running the following query from a hive terminal;

    create external table d_table (id bigint, date string) stored by ‘org.apache.hadoop.hive.hbase.HBaseStorageHandler’ with serdeproperties (“hbase.columns.mapping” = “:key,a:date”) TBLPROPERTIES (“” = “demo_table”);

    The table exists, is populated and can be found from within a hbase shell, and I have checked that the HMASTER process is running. The ‘zookeeper.znode.parent’ is set to the default value, and hbase accepts this (if it is changed, hbase will complain, so I am fairly confident that it is correct). I am able to establish a ZK client with no problems.

    I have reached the point in researching this problem where I am told the solution is to recompile Hive against the version of HBase I am using, which sort of defies the point of using a distribution. ZK and HBase masters are on the same machine that I am running the HiveQL command from.

    If I run the query from another machine, I see the following;

    ERROR client.HConnectionManager$HConnectionImplementation ( – Check the value configured in ‘zookeeper.znode.parent’. There could be a mismatch with the one configured in the master.
    WARN zookeeper.ClientCnxn ( – Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect Connection refused
    at Method)
    at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(
    at org.apache.zookeeper.ClientCnxn$
    WARN zookeeper.RecoverableZooKeeper ( – Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid

    If I add a property to the hive-site.xml file for zookeeper.quorum, I get the same behaviour as the first machine. I have taken the values for the zookeeper.quorum from Ambari.

    I am using the release 1.3 distribution. We have successfully run Hive with HCatalog to create tables etc, and we have run a variety of other scenarios. If further information is needed to help, let me know, I wanted to keep this post clean.

    Thanks in advance,

    Andy Forrest

  • Author
  • #32797
    Enis Soztutar

    HBase clients discover the running HBase cluster using two properties, namely hbase.zookeeper.quorum and zookeeper.znode.parent. The former is used to connect to the zookeeper cluster, while the latter shows which znode keeps the data (and address for HMaster) for the cluster.

    Ambari and HDP installations use different zookeeper.znode.parent values than the default (/hbase). On secure setup, the value is configured as /hbase-secure, and in unsecure mode it is set as /hbase-unsecure.

    For Hive-HBase integration to work, thus Hive should be configured with these settings with either supplying these options via HIVE_OPTS=”-hiveconf hbase.zookeeper.quorum=server1,server2,server3 zookeeper.znode.parent=/hbase-unsecure” or inside the hive script with
    set hbase.zookeeper.quorum=server1,server2,server3
    set zookeeper.znode.parent=/hbase-unsecure

    Hope this helps.

The forum ‘Hive / HCatalog’ is closed to new topics and replies.

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.