Home Forums HBase Hive External Table Pointing to HBASE

Tagged: 

This topic contains 3 replies, has 3 voices, and was last updated by  abdelrahman 9 months, 1 week ago.

  • Creator
    Topic
  • #45119

    I have a Hbase table called

    Client_Txn with column famlies
    Txn (Txn_no)
    Dt (Txn_dt)
    Txn_Type (Type)
    Cf1 (val1, val2)
    Cf2 (val3, val4)

    Can I create a Hive Table pointing to this structure

    I basically want to create the Hive Table that refers to this external table and bring in only

    Txn_no
    Txn_dt
    Txn_Type
    Cf1_Val (from Cf1:val1)
    Cf2_Val (from Cf2:val3)

    Is it possible ?

Viewing 3 replies - 1 through 3 (of 3 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #45328

    abdelrahman
    Moderator

    Hi Paulie,

    What are the HBase/Hive jars that were added to the Hive query as well as Set values? Here are some steps to help resolve the issue:

    In the Hql do the following:
    – Change this to the Zookeeper servers
    set hbase.zookeeper.quorum=ZKHOST1,ZKHOST2,ZKHOST3;
    set hive.zookeeper.client.port=2181;
    – The following may vary
    set zookeeper.znode.parent=/hbase-unsecure ;

    In order to have all the Jars without the use of “add jar” operator, please follow these steps: (Change the jar versions to the latest)
    - Create a directory called auxlib
    #mkdir /usr/lib/hive/auxlib
    - Copy all of the following Jars to the previous directory:
    /usr/lib/hive/lib/hive-hbase-handler-0.11.0.1.3.0.0-107.jar;
    /usr/lib/zookeeper/zookeeper-3.4.5.1.3.0.0-107.jar ;
    /usr/lib/hbase/hbase-0.94.6.1.3.0.0-107-security.jar;
    /usr/lib/hbase/lib/guava-11.0.2.jar;

    Hope this helps

    Thanks
    -Rahman

    Collapse
    #45156

    Thanks. This is what I tried

    create external table hive_txn (
    id int,
    pol_no string,
    pol_amt int,
    bene_amt int
    )
    stored by
    ‘org.apache.hadoop.hive.hbase.HBaseStorageHandler’
    WITH SERDEPROPERTIES
    (“hbase.columns.mapping”=”:key,pol:pol_no,cf1:pol_amt,cf2:bene_amt”)
    TBLPROPERTIES
    (“hbase.table.name”=”txn”)

    Getting into the following error

    FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org/apache/hadoop/hbase/HBaseConfiguration

    Where cpi;d I be wrong ?

    Collapse
    #45125

    Nick Dimiduk
    Participant

    Yes, you can select which HBase column qualifiers are made available to Hive and using what name. This is the column mapping Serde property. See http://hortonworks.com/blog/using-hive-to-interact-with-hbase-part-2/ for a working (less complex) example.

    Collapse
Viewing 3 replies - 1 through 3 (of 3 total)