The Hortonworks Community Connection is now live. A completely rebuilt Q&A forum, Knowledge Base, Code Hub and more, backed by the experts in the industry.

You will be redirected here in 10 seconds. If your are not redirected, click here to visit the new site.

The legacy Hortonworks Forum is now closed. You can view a read-only version of the former site by clicking here. The site will be taken offline on January 31,2016

Hive / HCatalog Forum

Hive Hbase Integration

  • #31757
    surbhi singh
    Member

    Can anyone tell me the steps of inserting data into hbase table through hive table using insert query.

  • Author
    Replies
  • #31758
    surbhi singh
    Member

    I have integrated hive & hbase successfully. The below query run successfully & created table in both hive & hbase.

    CREATE TABLE hbase_test1(key int, value string)
    STORED BY ‘org.apache.hadoop.hive.hbase.HBaseStorageHandler’
    WITH SERDEPROPERTIES (“hbase.columns.mapping” = “:key,cf1:val”)
    TBLPROPERTIES (“hbase.table.name” = “test1”);

    Now i want to insert data into hbase table through hive. I have created a hive table name user.

    create table user(key int,value string)
    ROW FORMAT DELIMITED FIELDS TERMINATED BY ‘,’ LINES TERMINATED BY ‘\n’ STORED AS TEXTFILE;

    Successfully inserted data into hive table through this query.
    LOAD DATA LOCAL INPATH ‘/home/user.txt’ INTO TABLE user;

    Now i run this query into hive for inserting data into hbase table.
    INSERT OVERWRITE TABLE hbase_test1 SELECT * FROM user where key =1;

    Below error came when i run insert command:

    hive> INSERT OVERWRITE TABLE hbase_test3 SELECT * FROM user;
    Total MapReduce jobs = 1
    Launching Job 1 out of 1
    Number of reduce tasks is set to 0 since there’s no reduce operator
    Starting Job = job_201307292222_0015, Tracking URL = http://sandbox:50030/jobdet ails.jsp?jobid=job_201307292222_0015
    Kill Command = /usr/lib/hadoop/libexec/../bin/hadoop job -kill job_201307292222 _0015
    Hadoop job information for Stage-0: number of mappers: 1; number of reducers: 0
    2013-08-14 01:58:53,339 Stage-0 map = 0%, reduce = 0%
    2013-08-14 01:59:16,446 Stage-0 map = 100%, reduce = 100%
    Ended Job = job_201307292222_0015 with errors
    Error during job, obtaining debugging information…
    Job Tracking URL: http://sandbox:50030/jobdetails.jsp?jobid=job_201307292222_0015
    Examining task ID: task_201307292222_0015_m_000002 (and more) from job job_201307292222_0015

    Task with the most failures(4):
    —–
    Task ID:
    task_201307292222_0015_m_000000

    URL:
    http://sandbox:50030/taskdetails.jsp?jobid=job_201307292222_0015&tipid=task_201307292222_0015_m_000000
    —–
    Diagnostic Messages for this Task:
    java.lang.RuntimeException: Error in configuring object
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:425)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:365)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
    at org.apache.hadoop.mapred.Child.main(Child.java:249)

    #31759
    surbhi singh
    Member

    Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
    … 9 more
    Caused by: java.lang.RuntimeException: Error in configuring object
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
    at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
    … 14 more
    Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
    … 17 more
    Caused by: java.lang.RuntimeException: Map operator initialization failed
    at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
    … 22 more
    Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException
    at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:385)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:451)
    at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:407)
    at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:62)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:451)
    at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:407)
    at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:186)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
    at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:543)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
    at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:100)
    … 22 more
    Caused by: java.lang.NullPointerException
    at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:322)
    … 34 more

    FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
    MapReduce Jobs Launched:
    Job 0: Map: 1 HDFS Read: 0 HDFS Write: 0 FAIL
    Total MapReduce CPU Time Spent: 0 msec
    hive>

    #31760
    surbhi singh
    Member

    I have mentioned error into two parts.

    Please help me by providing solution as soon as possible.

    Thanks & Regards
    Surbhi Singh

The forum ‘Hive / HCatalog’ is closed to new topics and replies.

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.