Hive / HCatalog Forum

Hive HBase integration issues

  • #26407
    Arvind Kumar
    Member

    I’m currently using HDP sandbox. I’m trying to integrate Hive and HBase. When I create a table in Hive using the below script it fails.

    Script:
    CREATE TABLE atest(
    a string,
    b string,
    c int
    )
    STORED by
    ‘org.apache.hadoop.hive.hbase.HBaseStorageHandler’
    WITH SERDEPROPERTIES
    (“hbase.columns.mapping” = “:key, c1:a, c2:count”)
    TBLPROPERTIES
    (“hbase.table.name” = “hbase_tab_name”);

    Error:
    FAILED: Execution Error, return code -101 from org.apache.hadoop.hive.ql.exec.DDLTask
    13/05/27 19:58:48 ERROR ql.Driver: FAILED: Execution Error, return code -101 from org.apache.hadoop.hive.ql.exec.DDLTask
    13/05/27 19:58:48 INFO ql.Driver:
    13/05/27 19:58:48 ERROR beeswax.BeeswaxServiceImpl: Exception while processing query
    BeeswaxException(message:Driver returned: 9. Errors: Hive history file=/tmp/sandbox/hive_job_log_sandbox_201305271958_1717885251.txt
    FAILED: Execution Error, return code -101 from org.apache.hadoop.hive.ql.exec.DDLTask
    , log_context:033a7527-5f68-44d1-b48c-ca83a94db996, handle:QueryHandle(id:033a7527-5f68-44d1-b48c-ca83a94db996, log_context:033a7527-5f68-44d1-b48c-ca83a94db996), SQLState: )
    at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute(BeeswaxServiceImpl.java:319)
    at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:577)
    at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:566)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:337)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1118)
    at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1.run(BeeswaxServiceImpl.java:566)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
    at java.util.concurrent.FutureTask.run(FutureTask.java:138)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
    13/05/27 19:58:49 ERROR security.UserGroupInformation: PriviledgedActionException as:sandbox cause:BeeswaxException(message:Driver returned: 9. Errors: Hive history file=/tmp/sandbox/hive_job_log_sandbox_201305271958_1717885251.txt
    FAILED: Execution Error, return code -101 from org.apache.hadoop.hive.ql.exec.DDLTask
    , log_context:033a7527-5f68-44d1-b48c-ca83a94db996, handle:QueryHandle(id:033a7527-5f68-44d1-b48c-ca83a94db996, log_context:033a7527-5f68-44d1-b48c-ca83a94db996), SQLState: )
    13/05/27 19:58:49 ERROR beeswax.BeeswaxServiceImpl: Caught BeeswaxException
    BeeswaxException(message:Driver returned: 9. Errors: Hive history file=/tmp/sandbox/hive_job_log_sandbox_201305271958_1717885251.txt

    Can someone please help?

to create new topics or reply. | New User Registration

  • Author
    Replies
  • #26422
    Larry Liu
    Moderator

    Hi, Arvind

    I got the same error while running it from Hue. Can you please try to run it from the command line in HIVE?

    Larry

    #26588
    Arvind Kumar
    Member

    Larry-
    Thanks a lot for your reply.

    I did try it from the HIVE command line. I’m getting the below error now. Am I missing anything in my class path? Can you please help?

    java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/MasterNotRunningException
    java.lang.ClassNotFoundException: org.apache.hadoop.hbase.MasterNotRunningException

    Thanks,
    Arvind

    #26819
    Zeke Huang
    Member

    @Arvind, the ClassNotFoundException error indicates hbase jar is not in your classpath.
    Try either fix your classpath config or symlink the hbase jar (typically under /usr/lib/hbase) to hive lib directory /usr/lib/hive/lib.
    Other jars you may need include zookeeper, guava, protobuf-java.

    #26821
    Arvind Kumar
    Member

    Thanks Zake! I was able to resolve it. The tables are getting created in both Hive and HBase.

    When I load data via Hive, I’m able to see the data in HBase. I’m able to access the data via Hive too.

    But when I load the data in HBase, I’m not able to see the data in Hive. I did try creating external table in Hive too with the HBase column mapping. But, still no luck.

    #27708
    surbhi singh
    Member

    Hello,
    I am using Sandbox HDP. In this, our hive & hbase are working properly. We have integrated hive & hbase by adding some jar files like zookeeper-3.3.4.jar,guava-r09.jar,hadoop-core-1.0.0.jar,hbase-0.92.0.jar,hive-hbase-handler-0.9.0.jar to /usr/hive/lib & add aux path to our hive-site.xml. Our problem is that, when we execute this query then sometimes integration successfully done but sometimes this error came :

    Our query is this:
    CREATE TABLE hbase_test(key int, value string) STORED BY ‘org.apache.hadoop.hive.hbase.HBaseStorageHandler’ WITH SERDEPROPERTIES (“hbase.columns.mapping” = “:key,cf1:val”) TBLPROPERTIES (“hbase.table.name” = “xyz”);

    Error:

    hive> CREATE TABLE hbase_test(key int, value string)
    > STORED BY ‘org.apache.hadoop.hive.hbase.HBaseStorageHandler’
    > WITH SERDEPROPERTIES (“hbase.columns.mapping” = “:key,cf1:val”)
    > TBLPROPERTIES (“hbase.table.name” = “xyz”);
    java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/MasterNotRunningException
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at org.apache.hadoop.hive.ql.metadata.HiveUtils.getStorageHandler(HiveUtils.java:287)
    at org.apache.hadoop.hive.ql.metadata.Table.getStorageHandler(Table.java:279)
    at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:3570)
    at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:253)
    at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:138)
    at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
    at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1347)
    at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1130)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:940)
    at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
    at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:755)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:613)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
    Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.MasterNotRunningException
    at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
    … 21 more
    FAILED: Execution Error, return code -101 from org.apache.hadoop.hive.ql.exec.DDLTask

    Please provide me some solution about this error code -101

    #27774
    tedr
    Moderator

    Hi Surbhi,

    Check that HMaster, HRegionServer, and QuorumPeerMain are all listed in the java processes. The error you are getting seems to indicate that the HMaster is not running. You can check this by logging in to the Sandbox and running the following command:

    # /usr/jdk64/jdk1.6.0_31/bin/jps

    It is my guess that the QuorumPeerMain and HMaster will not be running. If not you will need to start Zookeeper and Hbase again.

    thanks,
    Ted.

You must be to reply to this topic. | Create Account

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.