Home Forums Hive / HCatalog Hive HBase integration issues

This topic contains 6 replies, has 5 voices, and was last updated by  tedr 1 year, 1 month ago.

  • Creator
    Topic
  • #26407

    Arvind Kumar
    Member

    I’m currently using HDP sandbox. I’m trying to integrate Hive and HBase. When I create a table in Hive using the below script it fails.

    Script:
    CREATE TABLE atest(
    a string,
    b string,
    c int
    )
    STORED by
    ‘org.apache.hadoop.hive.hbase.HBaseStorageHandler’
    WITH SERDEPROPERTIES
    (“hbase.columns.mapping” = “:key, c1:a, c2:count”)
    TBLPROPERTIES
    (“hbase.table.name” = “hbase_tab_name”);

    Error:
    FAILED: Execution Error, return code -101 from org.apache.hadoop.hive.ql.exec.DDLTask
    13/05/27 19:58:48 ERROR ql.Driver: FAILED: Execution Error, return code -101 from org.apache.hadoop.hive.ql.exec.DDLTask
    13/05/27 19:58:48 INFO ql.Driver:
    13/05/27 19:58:48 ERROR beeswax.BeeswaxServiceImpl: Exception while processing query
    BeeswaxException(message:Driver returned: 9. Errors: Hive history file=/tmp/sandbox/hive_job_log_sandbox_201305271958_1717885251.txt
    FAILED: Execution Error, return code -101 from org.apache.hadoop.hive.ql.exec.DDLTask
    , log_context:033a7527-5f68-44d1-b48c-ca83a94db996, handle:QueryHandle(id:033a7527-5f68-44d1-b48c-ca83a94db996, log_context:033a7527-5f68-44d1-b48c-ca83a94db996), SQLState: )
    at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute(BeeswaxServiceImpl.java:319)
    at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:577)
    at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:566)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:337)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1118)
    at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1.run(BeeswaxServiceImpl.java:566)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
    at java.util.concurrent.FutureTask.run(FutureTask.java:138)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
    13/05/27 19:58:49 ERROR security.UserGroupInformation: PriviledgedActionException as:sandbox cause:BeeswaxException(message:Driver returned: 9. Errors: Hive history file=/tmp/sandbox/hive_job_log_sandbox_201305271958_1717885251.txt
    FAILED: Execution Error, return code -101 from org.apache.hadoop.hive.ql.exec.DDLTask
    , log_context:033a7527-5f68-44d1-b48c-ca83a94db996, handle:QueryHandle(id:033a7527-5f68-44d1-b48c-ca83a94db996, log_context:033a7527-5f68-44d1-b48c-ca83a94db996), SQLState: )
    13/05/27 19:58:49 ERROR beeswax.BeeswaxServiceImpl: Caught BeeswaxException
    BeeswaxException(message:Driver returned: 9. Errors: Hive history file=/tmp/sandbox/hive_job_log_sandbox_201305271958_1717885251.txt

    Can someone please help?

Viewing 6 replies - 1 through 6 (of 6 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #27774

    tedr
    Moderator

    Hi Surbhi,

    Check that HMaster, HRegionServer, and QuorumPeerMain are all listed in the java processes. The error you are getting seems to indicate that the HMaster is not running. You can check this by logging in to the Sandbox and running the following command:

    # /usr/jdk64/jdk1.6.0_31/bin/jps

    It is my guess that the QuorumPeerMain and HMaster will not be running. If not you will need to start Zookeeper and Hbase again.

    thanks,
    Ted.

    Collapse
    #27708

    surbhi singh
    Member

    Hello,
    I am using Sandbox HDP. In this, our hive & hbase are working properly. We have integrated hive & hbase by adding some jar files like zookeeper-3.3.4.jar,guava-r09.jar,hadoop-core-1.0.0.jar,hbase-0.92.0.jar,hive-hbase-handler-0.9.0.jar to /usr/hive/lib & add aux path to our hive-site.xml. Our problem is that, when we execute this query then sometimes integration successfully done but sometimes this error came :

    Our query is this:
    CREATE TABLE hbase_test(key int, value string) STORED BY ‘org.apache.hadoop.hive.hbase.HBaseStorageHandler’ WITH SERDEPROPERTIES (“hbase.columns.mapping” = “:key,cf1:val”) TBLPROPERTIES (“hbase.table.name” = “xyz”);

    Error:

    hive> CREATE TABLE hbase_test(key int, value string)
    > STORED BY ‘org.apache.hadoop.hive.hbase.HBaseStorageHandler’
    > WITH SERDEPROPERTIES (“hbase.columns.mapping” = “:key,cf1:val”)
    > TBLPROPERTIES (“hbase.table.name” = “xyz”);
    java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/MasterNotRunningException
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at org.apache.hadoop.hive.ql.metadata.HiveUtils.getStorageHandler(HiveUtils.java:287)
    at org.apache.hadoop.hive.ql.metadata.Table.getStorageHandler(Table.java:279)
    at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:3570)
    at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:253)
    at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:138)
    at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
    at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1347)
    at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1130)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:940)
    at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
    at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:755)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:613)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
    Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.MasterNotRunningException
    at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
    … 21 more
    FAILED: Execution Error, return code -101 from org.apache.hadoop.hive.ql.exec.DDLTask

    Please provide me some solution about this error code -101

    Collapse
    #26821

    Arvind Kumar
    Member

    Thanks Zake! I was able to resolve it. The tables are getting created in both Hive and HBase.

    When I load data via Hive, I’m able to see the data in HBase. I’m able to access the data via Hive too.

    But when I load the data in HBase, I’m not able to see the data in Hive. I did try creating external table in Hive too with the HBase column mapping. But, still no luck.

    Collapse
    #26819

    Zeke Huang
    Member

    @Arvind, the ClassNotFoundException error indicates hbase jar is not in your classpath.
    Try either fix your classpath config or symlink the hbase jar (typically under /usr/lib/hbase) to hive lib directory /usr/lib/hive/lib.
    Other jars you may need include zookeeper, guava, protobuf-java.

    Collapse
    #26588

    Arvind Kumar
    Member

    Larry-
    Thanks a lot for your reply.

    I did try it from the HIVE command line. I’m getting the below error now. Am I missing anything in my class path? Can you please help?

    java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/MasterNotRunningException
    java.lang.ClassNotFoundException: org.apache.hadoop.hbase.MasterNotRunningException

    Thanks,
    Arvind

    Collapse
    #26422

    Larry Liu
    Moderator

    Hi, Arvind

    I got the same error while running it from Hue. Can you please try to run it from the command line in HIVE?

    Larry

    Collapse
Viewing 6 replies - 1 through 6 (of 6 total)